Thoughts

Thread progress monitoring in Python

💬 1

My recent hobby hack-together, my photo cleanup tool FotoJazz, required me getting my hands dirty with threads for the first time (in Python or otherwise). Threads allow you to run a task in the background, and to continue doing whatever else you want your program to do, while you wait for the (usually long-running) task to finish (that's one definition / use of threads, anyway — a much less complex one than usual, I dare say).

However, if your program hasn't got much else to do in the meantime (as was the case for me), threads are still very useful, because they allow you to report on the progress of a long-running task at the UI level, which is better than your task simply blocking execution, leaving the UI hanging, and providing no feedback.

As part of coding up FotoJazz, I developed a re-usable architecture for running batch processing tasks in a thread, and for reporting on the thread's progress in both a web-based (AJAX-based) UI, and in a shell UI. This article is a tour of what I've developed, in the hope that it helps others with their thread progress monitoring needs in Python or in other languages.

FotoJazz: A tool for cleaning up your photo collection

I am at times quite a prolific photographer. Particularly when I'm travelling, I tend to accumulate quite a quantity of digital snaps (although am still working on the quality of said snaps). I'm also a reasonably organised and systematic person: as such, I've developed a workflow for fixing up, naming and archiving my soft-copy photos; and I've also come to depend on a variety of scripts and little apps, that perform various steps of the workflow for me.

Sadly, my system has had some disadvantages. Most importantly, there are too many separate scripts / apps involved, and with too many different interfaces (mix of manual point-and-click, drap-and-drop, and command-line). Ideally, I'd like all the functionality unified in one app, with one streamlined graphical interface (and also everything with equivalent shell access). Also, my various tools are platform-dependent, with most of them being Windows-based, and one being *nix-based. I'd like everything to be platform-independent, and in particular, I'd like everything to run best on Linux — as I'm trying to do as much as possible on Ubuntu these days.

Plus, I felt in the mood for getting my hands dirty coding up the photo-management app of my dreams. Hence, it is with pleasure that I present FotoJazz, a browser-based (plus shell-accessible) tool built with Python and Flask.

On the Torah and the Word of G-d

The Torah — also known as the Old Testament, the Pentateuch, or the Five Books of Moses — is the foundation document of the Jewish religion (among others), and it's regarded by Orthodox Jews (among others) as the infallible Word of G-d. It's generally believed that the Torah in its entirety was conveyed orally to Moses on Mount Sinai, approximately 3,300 years ago; and that Moses transcribed the Torah exactly as it was revealed to him.

Most of the Torah is rock solid: sensible laws; moralistic stories; clear presentation of history; and other important information, such as geneaologies, rituals, and territorial boundaries. However, sometimes them five books throw some serious curve balls. I've selected here a few sections from the wonderful O.T, that in my opinion are so outrageously messed up, that they cannot possibly be the Divine Word. I believe in G-d, one hundred percent. But I see no reason to believe that these particular passages have a Divine source.

Five good reasons to not take drugs

💬 1

The use of narcotic substances isn't usually a topic about which I have strong feelings. I don't take drugs myself, and I tend not to associate myself with people who do; but then again, I see no reason to stop other members of society from exercising their liberties, in the form of recreational drug use. I've never before spoken much about this subject.

However, one of my best friends recently died from a drug overdose. On account of that, I feel compelled to pen a short article, describing what I believe are some good reasons to choose to not take drugs.

Highway border crossings between Chile and Argentina

💬 7

I've spent a fair bit of time, on several occasions, travelling in South America, including in Chile and Argentina. I've crossed the land border between these two countries several times, in several different places. It's an extremely long border, measuring 5,308km in total.

Recently, I was looking for a list of all the official crossings between the two countries. Finding such a list, in clear and authoritative form, proved more difficult than I expected. Hence, one thing led to another; and before I knew it, I'd embarked upon a serious research mission to develop such a list myself. So, here it is — a list of all highway border crossings between Chile and Argentina, that are open to the general public.

Travel is a luxury

International travel has become so commonplace nowadays, some people do it just for a long weekend. Others go for two-year backpacking marathons. And with good reason, too. Travelling has never been easier, it's never been cheaper, and it's never before been so accessible. I, for one, do not hesitate to take advantage of all this.

One other thing, though. It's also never been easier to inadvertently take it all for granted. To forget that just one generation ago, there were no budget intercontinental flights, no phrasebooks, no package tours, no visa-free agreements. And, of course, snail mail and telegrams were a far cry from our beloved modern Internet.

But that's not all. The global travel that many of us enjoy today, is only possible thanks to a dizzying combination of fortunate circumstances. And this tower (no less) of circumstances is far from stable. On the contrary: it's rocking to and fro like a pirate ship on crack. I know it's hard for us to comprehend, let alone be constantly aware of, but it wasn't like this all that long ago, and it simply cannot last like this much longer. We are currently living in a window of opportunity like none ever before. So, carpe diem — seize the day!

Solr, Jetty, and daemons: debugging jetty.sh

💬 7

I recently added a Solr-powered search feature to this site (using django-haystack). Rather than go to the trouble (and server resources drain) of deploying Solr via Tomcat, I decided instead to deploy it via Jetty. There's a wiki page with detailed instructions for deploying Solr with Jetty, and the wiki page also includes a link to the jetty.sh startup script.

The instructions seem simple enough. However, I ran into some serious problems when trying to get the startup script to work. The standard java -jar start.jar was working fine for me. But after following the instructions to the letter, and after double-checking everything, a call to:

sudo /etc/init.d/jetty start

still resulted in my getting the (incredibly unhelpful) error message:

Starting Jetty: FAILED

My server is running Ubuntu Jaunty (9.04), and from my experience, the start-stop-daemon command in jetty.sh doesn't work on that platform. Let me know if you've experienced the same or similar issues on other *nix flavours or on other Ubuntu versions. Your mileage may vary.

Jimmy Page: site-wide Django page caching made simple

💬 1

For some time, I've been using the per-site cache feature that comes included with Django. This site's caching needs are very modest: small personal site, updated infrequently, with two simple blog-like sections and a handful of static pages. Plus, it runs fast enough even without any caching. A simple "brute force" solution like Django's per-site cache is more than adequate.

However, I grew tired of the fact that whenever I published new content, nothing was invalidated in the cache. I began to develop a routine of first writing and publishing the content in the Django admin, and then SSHing in to my box and restarting memcached. Not a good regime! But then again, I also couldn't bring myself to make the effort of writing custom invalidation routines for my cached pages. Considering my modest needs, it just wasn't worth it. What I needed was a solution that takes the same "brute force" page caching approach that Django's per-site cache already provided for me, but that also includes a similarly "brute force" approach to invalidation. Enter Jimmy Page.

On and off

English is a language bursting with ambiguity and double meanings. But the words "on" and "off" would have to be two of the worst offenders. I was thinking about words that foreign-language speakers would surely find particularly hard to master, when learning to speak English. And I couldn't go past these two. From the most basic meaning of the words, which relates to position — e.g. "the book is on the table", and "the plane is off the ground" — "on" and "off" have been overloaded more thoroughly than an Indian freight train.

The Art of Sport

Soccer (or football) is the most popular sport in the world today; likewise, with over 700 million viewers, the 4-yearly FIFA World Cup is the most watched event in human history. Why is this so? Is it mere chance — a matter of circumstance — that soccer has so clearly and definitively wooed more devotees than any other sport? I think not. Team sports are an approximation of military battle. All such sports endeavour to foster the same strategies that can be used effectively in traditional warfare. Soccer, in my opinion, fosters those strategies better than any other sport; this is because, in tactical terms, it is the most pure sport in the modern world.