And it turns out, solutions to that problem don’t really exist. Yes, I can “roll my own” with duct tape and a bunch of shell scripts – but is this truly a system that nobody ever saw the need to create and share with the world?

In case it is shared, I’d really like to hear about it. If not, it’s probably ending up on my projects list sooner or later… And in case it does, here’s a set of links to get started.

OSS packages

Somewhat close to what I need, usually with lots of overkill in areas I don’t need:

• Condor
• TORQUE – this one looks closest to what I need, but again, is huge
• Maui Scheduler – a scheduling counterpart to TORQUE
• GNU Queue – seems pretty abandoned
• DrQueue – Mostly for scheduling render tasks, but can maybe be bent to my will

Discussions & articles on the net:

• A slashdot article
• Getting RabbitMQ on OSX
• Jeremy Zawodny on Open Source Queueing and Messaging Systems

So ideally, I’d like to have my code in an offsite repository, in addition to what I have locally. Putting my eggs into multiple baskets, so to speak.

The simplest solution would be to install GIT on my dreamhost account. (Shameless plug: If you sign up with them and use the promo code GROBY, you get $50 off your fees, and I’ll make $47 towards my hosting costs.) Autopragmatic has an excellent account on how to set up git on dreamhost. That’s not quite what I want though. First of all, quite a few git repositories are my actual web sites – hosting them with your service provider seems to defeat the purpose.

The next option coming up is hosting my git repositories on S3. Unfortunately, the only options I am seeing involve using S3 as a file system – i.e. using s3fs on OSX. Again, not a very appealing option to me – if I host my code in the cloud, I want to be able to get there from anywhere.

Next up on the cost scale: InDefero. It is an open source package that provides both git & svn hosting, and adds bug tracking/code review on top. And on top of all that, they give you free hosting with 250MB storage. Downside? 6 months of inactivity, and they drop your account. It’s probably good enough for most people like me, who have a local repo + backup, only small projects, and want additional cloud hosting, just to feel secure.

It’s probably a not-so-good idea to rely on them if that code is your livelihood, though – you do want some level of contractual agreement there.

The next step up then would be either going with a hosted provider – there is a GitHosting list on GitWiki, or installing indefero on your own hosts. (The cost is roughly the same, since you will need a second hosting provider to gain any real benefit – it depends if you want somebody focused on source control systems only, or if you need a second hosting provider anyways)

And whatever you do – if it’s code that makes you money, make sure who you are going with. Losing source code is painful enough if it’s just a hobby project.

Requirement #2 – I don’t want to manage my backups. After a couple of days, backups should automatically be deleted.

The easiest way to achieve all this is using crontab. Just add the following two lines

  0 3  *   *   *     mysqldump --defaults-extra-file=<CONFIG PATH>/backup.cnf -h <DB HOST> --comments -R --triggers --quote-names --allow-keywords --set-charset --quick --single-transaction --lock-tables <DATABASE> > <BACKUP PATH>/database_$(date +%Y_%m_%d).sql
  0 4  *   *   *     find <BACKUP PATH> -name '*.sql' -and -mtime +7 | xargs rm

and create a configuration file called backup.cnf which looks roughly like this:

[client]
user = <user name>
password = <password>

(And obviously, adjust all paths and site names to be useful for you)

The point of the .cnf file is to not announce your password via the command line. (Which everybody can take a look at should they happen to be online when your backup is running)

There are a few decent starting points, for sure. If that’s what you’re after, here’s the list:

• django-basic-apps
  The canonic starting point for a lot of Django apps. It takes care not only of blogs, but a lot of other fairly common things (movies, music, books, bookmarks, places, people,…) too. Seems to be maintained by a single person.

• Django Techblog
  Mostly worth mentioning for its integration of Microblogging. While it’s only a single person working at it, it looks like a fairly active application. There seem to be at least 10-15 check-ins per month.

• MightyLemon
  Fairly active, multiple committers. Nothing special about it, as far as I can tell. (Unless you’re a huge Leah Culver fan)

• BlogMaker
  Fairly large feature set, but very infrequent commits, even though the author only think it’s worthy of a 0.6. Definitely a good place to see how others solved a lot of blogging issues.

• ByteFlow
  Large feature list, single committer (mostly).If you want a feature complete solution that requires minimal work, this would probably be the best choice.

• Soturi
  Fairly minimal code, few changes, seems abandoned since December ‘08. Good project if you want to see a minimalist blog

• Banjo
  The most curious case of all of them. Somebody put quite a bit of design effort into having a nice home for this, clearly intending to make this a good reference implementation – but then fell short of providing much useful content on the page. From the looks of the repository, it seems to be under more or less active development by at least one person.

After thinking about all of this for a bit of time, I decided to roll my own – after all, the main purpose of my new blog is learning about Django, so I might as well do more than just download some code…

I’m still lazy enough to look for a decent starting point that takes care of a lot of grunt work – and that allows me to post fairly soon. (I really want to start writing, and not postpone it for another few months.)

As a result of all that, I decided to start based from django-basic-apps. It’s the canonic solution, it’s hosted on github (my vcs host of choice), and it seems to be under active development. I can always borrow from the other ones, should the need arise…

In Django’s case, I need to install flatpages. That solves the problem of storing static pages, and lets me create them right via the admin interface. But here’s the rub: I want some of those pages preloaded when I deploy – especially during the initial experimental phase, I anticipate wiping out and reinstalling my application quite a few times.

Thankfully, there’s a solution to that too – django-admin dumpdata/loaddata. dumpdata lets you export data from the database into JSON files. I.e. to dump all your flatpages:

And obviously, you reload via

This is almost all I need – I could certainly encode the re-loading of the data as another deployment step. But – here’s where being in a framework starts paying off – there’s a way to do this more easily. Just rename your dumped data to initial_data.json, and Django will load it into the database whenever you execute a syncdb

And since I felt like I need more experience with web frameworks in general, I’m not sticking with my trusty wordpress. It does all the things I want it to do, but it was time to try something new: Django.

It was surprisingly easy to get started – most of my troubles came from the facts that I’ve skipped steps in the tutorials here or there, and then got flabbergasted when things didn’t work. Well, it’s all sorted out so far, but there’s one nifty trick I’d really like to share. Django has all its settings in a settings.py file, and it needs to change depending where you deploy and what your backend database is. That doesn’t work well with the idea of version control – so I modified the settings so you can get settings per machine.

At the bottom of settings.py, add the following code:

Notes

1. I put all my configurations into a separate config module. I just like things clean – you can certainly keep them with the rest of your python.
2. I know that exec is evil, but I really couldn’t figure out how to do this with __import__ instead. Any tips appreciated
3. The mangling of gethostname() via replace() gets rid of invalid characters that import can’t handle. Also, it’s only split over two lines so I don’t get ugly line breaks.

Another layer of indirection

All those per-host files should only contain host-specific settings. With exactly one development and one production machine, that’s certainly not too much duplication. And yet, I split it up into production.py and development.py – again, just because it looks a little bit cleaner¹ (And I might get a new development machine at some point…)

So the config file for my local development machine right now only says

I’ve been tempted to add a base.py for shared settings into the mix, but at some point, I probably should go back to developing as opposed to housekeeping.

First stop: AquaMacs More or less it’s Emacs, adopted for OSX. I installed it – it was as easy as Mac apps should be – but it turns out it already has a built-in version of SLIME. And most SLIME docs recommend you always use the latest CVS version.

So back to the drawing board. We have to build the real McCoy. You get it like this:

Use your e-mail address as password, wait a while till everything’s downloaded, and

If you’re not familiar with emacs, I’d recommend you at least do a quick run through the emacs tutorial. Fire up emacs, press Control-h followed by t, and follow the on-screen instructions.

Next up – we’re installing SBCL

Once that is done,

, and it should bring up the SBCL prompt. Type

followed by enter to leave it again.

Now we’re ready to get SLIME

Finally, all you need is add a couple of lines to your .emacs

If you need more detailed instructions, go to http://www.david-steuber.com/Lisp/OSX/

The most recommended starting point for the language, at least according to Google: Practical Common Lisp

My verdict so far – I can’t stand that environment. SLIME is fragile, and Emacs is a prehistoric editor. As for Practical Common Lisp, it’s a decent intro, but the code examples bored me to tears. I really don’t want to write yet another MP3 song database…

More research reveals that there is a rather overwhelming collection of possible starting points. One of the best collections of references to Lisp environments and Lisp books is a thread on reddit – I really should’ve looked there sooner.

So the setup I ended up with for now? I stuck with SBCL, the editor is good old trusty TextMate, and for a starting set of problems to cut my teeth on, I went with Project Euler. (PCL left me with a good enough grasp of Lisp that I could at least start on problem one…)

Objective C in general:	Programming ObjectiveC
Intro to Cocoa:	Learning Cocoa with ObjectiveC
Getting your feet wet on the iPhone:	iPhone Open Application Development

And of course, when it comes to web pages, there’s the invaluable CocoaDev Wiki.

That’s rather annoying. Thankfully, that’s a solved problem. Many CPUs prefer non-branching code, too, and there’s a long history of creating non-branching version of code that technically should branch. The trick here is “creative” use of math and/or bit-logic functions. As a simple example, let’s say I have a threshold shader – values below the minimum value should have a brightness of 0, and values higher than the max value should have a brightness of 1.

step() function to the rescue: step(a,b) returns 0 if a < b, 1 if a > b. So to cut off the low range, we have

Very similarly, to check against the max value, we force brightness to be greater than 1 for pixels above the maximum threshold and then clamp to 1.0

That’s all there is to it. And in case you actually came for the filter code, here it is – with input sanitizing, nicer input values (midpoint/range), grayscale conversion, all the fun stuff.

It only seems that way, though. A quick look behind the scenes reveals that all that shell does is:

• start the ruby environment
• ‘require’ all included ruby scripts
• and runs NSApplicationMain

The first two, I can obviously do on my own. The third provides the main loop for all NSApplications. But you can’t simply call it from Ruby – it requires you to have an Info.plist. Again, too much baggage for quick & dirty prototypes. The goal is to have one ruby file to play around in.

That one is a bit trickier to find out, but after digging around in Apple’s documentation, it turns out there is a decent explanation in the overview for NSApplication

For future reference, this is what it does:

Now we’re getting somewhere. This can actually be completely duplicated in Ruby. Since I might want to quit my application, I’ve thrown in a window that shuts down the application when closed.

Or just download standalone.rb