Thoughts

Orientalists of the East India Company

💬 1

The infamous East India Company, "the Company that Owned a Nation", is remembered harshly by history. And rightly so. On the whole, it was an exploitative venture, and the British individuals involved with it were ruthless opportunists. The Company's actions directly resulted in the impoverishment, the subjugation, and in several instances the death of countless citizens of the Indian Subcontinent.

Company rule, and the subsequent rule of the British Raj, are also acknowledged as contributing positively to the shaping of Modern India, having introduced the English language, built the railways, and established political and military unity. But these are overshadowed by its legacy of corporate greed and wholesale plunder, which continues to haunt the region to this day.

I recently read Four Heroes of India (1898), by F.M. Holmes, an antique book that paints a rose-coloured picture of Company (and later British Government) rule on the Subcontinent. To the modern reader, the book is so incredibly biased in favour of British colonialism that it would be hilarious, were it not so alarming. Holmes's four heroes were notable military and government figures of 18th and 19th century British India.

Clive, Hastings, Havelock, Lawrence; with a Concluding Note on the Rule of Lord Mayo.
Clive, Hastings, Havelock, Lawrence; with a Concluding Note on the Rule of Lord Mayo.
Image source: eBay.

I'd like to present here four alternative heroes: men (yes, sorry, still all men!) who in my opinion represented the British far more nobly, and who left a far more worthwhile legacy in India. All four of these figures were founders or early members of The Asiatic Society (of Bengal), and all were pioneering academics who contributed to linguistics, science, and literature in the context of South Asian studies.

Where is the official centre of Sydney?

There are several different ways of commonly identifying the "official centre point" of a city. However, there's little international consensus as to the definition of such a point, and in many countries and cities the definition is quite vague.

Most reliable and most common, is to declare a Kilometre Zero marker as a city's (and often a region's or even a country's) official centre. Also popular is the use of a central post office for this purpose. Other traditional centre points include a city's cathedral, its main railway station, its main clock tower (which may be atop the post office / cathedral / railway station), its tallest building, its central square, its seat of government, its main park, its most famous tourist landmark, or the historical spot at which the city was founded.

Satellite photo of Sydney CBD, annotated with locations of
Satellite photo of Sydney CBD, annotated with locations of "official centre" candidates.
Image source: Satellite Imaging Corp.

My home town of Sydney, Australia, is one of a number of cities worldwide that boasts most of the above landmarks, but all in different locations, and without any mandated rule as to which of them constitutes the official city centre. So, where exactly in Sydney does X mark the spot?

Where there be no roads

💬 1

And now for something completely different, here's an interesting question. What terra firma places in the world are completely without roads? Where in the world will you find large areas, in which there are absolutely no official vehicle routes?

A road (of sorts) slowly being extended into the vast nothingness of Siberia.
A road (of sorts) slowly being extended into the vast nothingness of Siberia.
Image source: Favourite picture: Road construction in Siberia – RoadStars.

Naturally, such places also happen to be largely bereft of any other human infrastructure, such as buildings; and to be largely bereft of any human population. These are places where, in general, nothing at all is to be encountered save for sand, ice, and rock. However, that's just coincidental. My only criteria, for the purpose of this article, is a lack of roads.

Running a real Windows install in VirtualBox on Linux

💬 1

Having a complete Windows (or Mac) desktop running within Linux has been possible for some time now, thanks to the wonders of Virtual Machine (VM) technology. However, the typical approach is to mount and boot a VM image, where the guest OS and hard disk are just files on the host filesystem. In this case, the guest OS can't be natively booted and run, because it doesn't occupy its own disk or partition on the physical hardware, and therefore it can't be picked up by the BIOS / boot manager.

I've been installing Windows and Linux on the same machine, in a dual-boot setup, for many years now. In this case, I boot natively into either one or the other of the installed OSes. However, I haven't run one "real" OS (i.e. an OS that's installed on a physical disk or partition) inside the other via a VM. At least, not until now.

At my new job this year, I discovered that it's possible to do such a thing, using a feature of VirtualBox called "Raw Disk Access". With surprisingly few hiccups, I got this running with Linux Mint 17.3 as the host, and with Windows 8.1 as the guest. Each OS is installed on a separate physical hard disk. I run Windows inside the VM most of the time, but I can still boot natively into the very same install of Windows at any time, if necessary.

Introducing Flask Editable Site

💬 1

I'd like to humbly present Flask Editable Site, a template for building a small marketing web site in Flask where all content is live editable. Here's a demo of the app in action.

Text and image block editing with Flask Editable Site.
Text and image block editing with Flask Editable Site.

The aim of this app is to demonstrate that, with the help of modern JS libraries, and with some well-thought-out server-side snippets, it's now perfectly possible to "bake in" live in-place editing for virtually every content element in a typical brochureware site.

This app is not a CMS. On the contrary, think of it as a proof-of-concept alternative to a CMS. An alternative where there's no "admin area", there's no "editing mode", and there's no "preview button". There's only direct manipulation.

"Template" means that this is a sample app. It comes with a bunch of models that work out-of-the-box (e.g. text content block, image content block, gallery item, event). However, these are just a starting point: you can and should define your own models when building a real site. Same with the front-end templates: the home page layout and the CSS styles are just examples.

Cookies can't be more than 4KiB in size

💬 2

Did you know: you can't reliably store more than 4KiB (4096 bytes) of data in a single browser cookie? I didn't until this week.

What, I can't have my giant cookie and eat it too? Outrageous!
What, I can't have my giant cookie and eat it too? Outrageous!
Image source: Giant Chocolate chip cookie recipe.

I'd never before stopped to think about whether or not there was a limit to how much you can put in a cookie. Usually, cookies only store very small string values, such as a session ID, a tracking code, or a browsing preference (e.g. "tile" or "list" for search results). So, usually, there's no need to consider its size limits.

However, while working on a new side project of mine that heavily uses session storage, I discovered this limit the hard (to debug) way. Anyway, now I've got one more adage to add to my developer's phrasebook: if you're trying to store more than 4KiB in a cookie, you're doing it wrong.

Robert Dawson: the first anthropologist of Aborigines?

The treatment of Aboriginal Australians in colonial times was generally atrocious. This is now well known and accepted by most. Until well into the 20th century, Aborigines were subjected to exploitation, abuse, and cold-blooded murder. They were regarded as sub-human, and they were not recognised at all as the traditional owners of their lands. For a long time, virtually no serious attempts were made to study or to understand their customs, their beliefs, and their languages. On the contrary, the focus was on "civilising" them by imposing upon them a European way of life, while their own lifestyle was held in contempt as "savage".

I recently came across a gem of literary work, from the early days of New South Wales: The Present State of Australia, by Robert Dawson. The author spent several years (1826-1828) living in the Port Stephens area (about 200km north of Sydney), as chief agent of the Australian Agricultural Company, where he was tasked with establishing a grazing property. During his time there, Dawson lived side-by-side with the Worimi indigenous peoples, and Worimi anecdotes form a significant part of his book (which, officially, is focused on practical advice for British people considering migration to the Australian frontier).

Robert Dawson of the Australian Agricultural Company.
Robert Dawson of the Australian Agricultural Company.
Image source: Wikimedia Commons.

In this article, I'd like to share quite a number of quotes from Dawson's book, which in my opinion may well constitute the oldest known (albeit informal) anthropological study of Indigenous Australians. Considering his rich account of Aboriginal tribal life, I find it surprising that Dawson seems to have been largely forgotten by the history books, and that The Present State of Australia has never been re-published since its first edition in 1830 (the copies produced in 1987 are just fascimiles of the original). I hope that this article serves as a tribute to someone who was an exemplary exception to what was then the norm.

Splitting a Python codebase into dependencies for fun and profit

When the Python codebase for a project (let's call the project LasagnaFest) starts getting big, and when you feel the urge to re-use a chunk of code (let's call that chunk foodutils) in multiple places, there are a variety of steps at your disposal. The most obvious step is to move that foodutils code into its own file (thus making it a Python module), and to then import that module wherever else you want in the codebase.

Most of the time, doing that is enough. The Python module importing system is powerful, yet simple and elegant.

But… what happens a few months down the track, when you're working on two new codebases (let's call them TortelliniFest and GnocchiFest – perhaps they're for new clients too), that could also benefit from re-using foodutils from your old project? What happens when you make some changes to foodutils, for the new projects, but those changes would break compatibility with the old LasagnaFest codebase?

What happens when you want to give a super-charged boost to your open source karma, by contributing foodutils to the public domain, but separated from the cruft that ties it to LasagnaFest and Co? And what do you do with secretfoodutils, which for licensing reasons (it contains super-yummy but super-secret sauce) can't be made public, but which should ideally also be separated from the LasagnaFest codebase for easier re-use?

Some bits of Python need to be locked up securely as private dependencies.
Some bits of Python need to be locked up securely as private dependencies.
Image source: Hoedspruit Endangered Species Centre.

Or – not to be forgotten – what happens when, on one abysmally rainy day, you take a step back and audit the LasagnaFest codebase, and realise that it's got no less than 38 different *utils chunks of code strewn around the place, and you ponder whether surely keeping all those utils within the LasagnaFest codebase is really the best way forward?

Moving foodutils to its own module file was a great first step; but it's clear that in this case, a more drastic measure is needed. In this case, it's time to split off foodutils into a separate, independent codebase, and to make it an external dependency of the LasagnaFest project, rather than an internal component of it.

This article is an introduction to the how and the why of cutting up parts of a Python codebase into dependencies. I've just explained a fair bit of the why. As for the how: in a nutshell, pip (for installing dependencies), the public PyPI repo (for hosting open-sourced dependencies), and a private PyPI repo (for hosting proprietary dependencies). Read on for more details.

Generating a Postgres DB dump of a filtered relational set

PostgreSQL is my favourite RDBMS, and it's the fave of many others too. And rightly so: it's a good database! Nevertheless, nobody's perfect.

When it comes to exporting Postgres data (as SQL INSERT statements, at least), the tool of choice is the standard pg_dump utility. Good ol' pg_dump is rock solid but, unfortunately, it doesn't allow for any row-level filtering. Turns out that, for a recent project of mine, a filtered SQL dump is exactly what the client ordered.

On account of this shortcoming, I spent some time whipping up a lil' Python script to take care of this functionality. I've converted the original code (written for a client-specific data set) to a more generic example script, which I've put up on GitHub under the name "PG Dump Filtered". If you're just after the code, then feel free to head over to the repo without further ado. If you'd like to stick around for the tour, then read on.

Storing Flask uploaded images and files on Amazon S3

💬 3

Flask is still a relative newcomer in the world of Python frameworks (it recently celebrated its fifth birthday); and because of this, it's still sometimes trailing behind its rivals in terms of plugins to scratch a given itch. I recently discovered that this was the case, with storing and retrieving user-uploaded files on Amazon S3.

For static files (i.e. an app's seldom-changing CSS, JS, and images), Flask-Assets and Flask-S3 work together like a charm. For more dynamic files, there exist numerous snippets of solutions, but I couldn't find anything to fill in all the gaps and tie it together nicely.

Due to a pressing itch in one of my projects, I decided to rectify this situation somewhat. Over the past few weeks, I've whipped up a bunch of Python / Flask tidbits, to handle the features that I needed:

I've also published an example app, that demonstrates how all these tools can be used together. Feel free to dive straight into the example code on GitHub; or read on for a step-by-step guide of how this Flask S3 tool suite works.