Random Thoughts

Tech, words and musings from an Englishman in Seattle

LWB Is Now Open Source

I recently talked about how I implemented “LWB”, the software that powers this blog, using Go. Now, I'm happy to announce that I've now released it as open source - well, technically I released it a few weeks ago…

The code is all hosted on GitHub, along with a sample blog that looks very much like this one :-)

Enjoy and contribute!

LWB on GitHub.

Reinventing the Wheel

A few weeks ago I noted that I really wanted to rewrite the blogging platform that I use for “Random Thoughts” from scratch. A number of reasons prompted this:

  • It's way too slow. Page load times are awful.
  • It's way too complicated. MovableType is written in a mixture of perl and php and as a platform it has become way too bloated for my own personal use. I have little clue how it works and little desire to learn.
  • The future of the platform appears uncertain.
  • Upgrades can be painful.
  • It needs a full instance of MySQL behind it, with all the associated management headaches that come along with it.

But mostly I wanted to just write my own.

What language?

The first order of business was which language to use. I wanted to learn something new.

I first contemplated using Scala, and had actually gone down that route for a while. Unfortunately my brain exploded after a couple of nights coding.

Scala is pretty and all, but damn does it have a steep learning curve, even for someone moderately familiar with functional languages and very familiar with Java. I was spending way too long digging around in library code and documentation and still couldn't figure out how a simple generator pattern worked. Seriously, what breakOut is doing here?

scala> val s = Set(1, 2, 3)            
s: scala.collection.immutable.Set[Int] = Set(1, 2, 3)

scala> val m: Map[Int, Int] = s.map(k => (k, k * 10))(breakOut)

Anyhow, I gave up. To much reading, not enough coding.

Next up: Go.

I've been wanting to get my teeth into Go for a while and this seemed like the perfect project. After an initial ramp up reading the superb documentation and dicking around with a few toy samples I was off to the races. Code was figuratively dripping from my fingers.


One big choice was still to be made, though: What database system should I use to store the blog entries in?

On the face of it, there seems an obvious answer: Just use MySQL. The bindings exist and I know how to use it.

But then I stood back and thought about it.

  • There's only just over one thousand posts and it's not likely to increase by a factor of ten any time soon.
  • The indexes needed are well know ahead of time: by date, by tag and by category.

So why not just store them all as flat files? And that's what I did. Each post is a separate file that contains a single JSON object representing the post. I load them all up at startup and pre-build all the indexes. Nice and fast and works like a charm - no dependencies needed.

The only slightly tricky thing I needed to write was a python script that converts the MovableType export file to the JSON files.

Web framework?

When it comes to actually serving the pages, Go's built in http library was almost perfect, but I ended up using a framework developed by a friend of mine: Gary Burd's Twister. The library provides lots of fun stuff for URL handlers, pattern matching, default handlers including redirects, etc… Highly recommended.


I use Textile as the markup language for all my blog posts, so I needed a Go template formatter for it. Unfortunately none currently exist that I can find, so of course I had to write my own and this ended up taking more time than any other piece of this project, and it only supports the small subset of Textile that I actually use. I sincerely hope that someone else writes a full implementation of the textile parser for Go so that I can throw my own away…


A key goal of this whole project was to reduce the latency of my blog down to as close to nothing as possible. Unfortunately the textile parser and page rendering take time, especially for some of the larger archive pages. So I needed to cache, but what to use?

An obvious choice here is Memcached, but there's only one instance of the server and it adds another moving piece. So what the hell, I might as well just cache the contents of each page render and the conversion of each post from textile to html (as each post appears on multiple archive pages as well as it's own page). A quick back of the envelope calculation yields a few tens of megabytes needed. No biggy.


As I was rewriting the blogging platform, I might as well do a complete revamp of the layout of the blog. So the design is a complete rewrite using HTML5, using all the key buzzwords including web fonts. Take a look at the colophon for more details.

Was it worth it?

Absolutely. The text you're currently reading came from the platform in one form or another, so it seems to be working.

The only real missing piece is on the authoring side. Right now I just run a python script to create each new entry and emacs to edit the content. Not big deal for me, but it would be nice to provide API access so that tools such as MarsEdit (my favourite blog content creation tool) can be used with it.

And of course it needs a name. Software, I christen thee “Light Weight Blogging”.

© 2001 to present, Steve Lacey.