Random Thoughts

Tech, words and musings from an Englishman in Seattle

LWB Is Now Open Source

I recently talked about how I implemented “LWB”, the software that powers this blog, using Go. Now, I'm happy to announce that I've now released it as open source - well, technically I released it a few weeks ago…

The code is all hosted on GitHub, along with a sample blog that looks very much like this one :-)

Enjoy and contribute!

LWB on GitHub.

HTML5 Media Statistics

Recently I've been spending some time getting together a proposal for the HTML5 <video> and <audio> tags that enables web applications to gather statistics about how media playback is performing.

A lot of this was inspired by Mozilla's statistics support and the WebKit support I added at the beginning of the year (which shipped in Chrome 10).

So after a bunch of feedback, today I added a proposal to the WHATWG wiki — hopefully it won't get crushed too hard and we can get this into the HTML5 spec soon.

The key points are the ability to monitor download and decode bitrates as well as the presentation statistics and an interesting little metric called “jitter” that can give the developer feedback on perceived framerate quality.

Anyhow, take a look and give feedback on the list.


Over on his blog, Tim Bray describes his move to a more press-like hyphenation and justification model for the layout of his blog. I like it.

You should read his blog post for the full details, but overall, I agree. Ragged Right looks fairly icky — though you've probably never noticed it and there's certainly a lot of disagreement out there on the interwebs. However, after implementing exactly the same solution as Tim for fully hyphenated and justified text, I think my blog looks just that little bit prettier.

I hope you agree.

Reinventing the Wheel

A few weeks ago I noted that I really wanted to rewrite the blogging platform that I use for “Random Thoughts” from scratch. A number of reasons prompted this:

  • It's way too slow. Page load times are awful.
  • It's way too complicated. MovableType is written in a mixture of perl and php and as a platform it has become way too bloated for my own personal use. I have little clue how it works and little desire to learn.
  • The future of the platform appears uncertain.
  • Upgrades can be painful.
  • It needs a full instance of MySQL behind it, with all the associated management headaches that come along with it.

But mostly I wanted to just write my own.

What language?

The first order of business was which language to use. I wanted to learn something new.

I first contemplated using Scala, and had actually gone down that route for a while. Unfortunately my brain exploded after a couple of nights coding.

Scala is pretty and all, but damn does it have a steep learning curve, even for someone moderately familiar with functional languages and very familiar with Java. I was spending way too long digging around in library code and documentation and still couldn't figure out how a simple generator pattern worked. Seriously, what breakOut is doing here?

scala> val s = Set(1, 2, 3)            
s: scala.collection.immutable.Set[Int] = Set(1, 2, 3)

scala> val m: Map[Int, Int] = s.map(k => (k, k * 10))(breakOut)

Anyhow, I gave up. To much reading, not enough coding.

Next up: Go.

I've been wanting to get my teeth into Go for a while and this seemed like the perfect project. After an initial ramp up reading the superb documentation and dicking around with a few toy samples I was off to the races. Code was figuratively dripping from my fingers.


One big choice was still to be made, though: What database system should I use to store the blog entries in?

On the face of it, there seems an obvious answer: Just use MySQL. The bindings exist and I know how to use it.

But then I stood back and thought about it.

  • There's only just over one thousand posts and it's not likely to increase by a factor of ten any time soon.
  • The indexes needed are well know ahead of time: by date, by tag and by category.

So why not just store them all as flat files? And that's what I did. Each post is a separate file that contains a single JSON object representing the post. I load them all up at startup and pre-build all the indexes. Nice and fast and works like a charm - no dependencies needed.

The only slightly tricky thing I needed to write was a python script that converts the MovableType export file to the JSON files.

Web framework?

When it comes to actually serving the pages, Go's built in http library was almost perfect, but I ended up using a framework developed by a friend of mine: Gary Burd's Twister. The library provides lots of fun stuff for URL handlers, pattern matching, default handlers including redirects, etc… Highly recommended.


I use Textile as the markup language for all my blog posts, so I needed a Go template formatter for it. Unfortunately none currently exist that I can find, so of course I had to write my own and this ended up taking more time than any other piece of this project, and it only supports the small subset of Textile that I actually use. I sincerely hope that someone else writes a full implementation of the textile parser for Go so that I can throw my own away…


A key goal of this whole project was to reduce the latency of my blog down to as close to nothing as possible. Unfortunately the textile parser and page rendering take time, especially for some of the larger archive pages. So I needed to cache, but what to use?

An obvious choice here is Memcached, but there's only one instance of the server and it adds another moving piece. So what the hell, I might as well just cache the contents of each page render and the conversion of each post from textile to html (as each post appears on multiple archive pages as well as it's own page). A quick back of the envelope calculation yields a few tens of megabytes needed. No biggy.


As I was rewriting the blogging platform, I might as well do a complete revamp of the layout of the blog. So the design is a complete rewrite using HTML5, using all the key buzzwords including web fonts. Take a look at the colophon for more details.

Was it worth it?

Absolutely. The text you're currently reading came from the platform in one form or another, so it seems to be working.

The only real missing piece is on the authoring side. Right now I just run a python script to create each new entry and emacs to edit the content. Not big deal for me, but it would be nice to provide API access so that tools such as MarsEdit (my favourite blog content creation tool) can be used with it.

And of course it needs a name. Software, I christen thee “Light Weight Blogging”.

Fun With Home Wiring

When we moved into our place a few years ago I was happy to find that the place was wired with Cat5e, all run back to a central wiring closest in the house. Unfortunately though, every jack in the house was wired to phone (RJ11) jacks except for two - one in the office and one by the kitchen desk - I really want RJ45 everywhere.

This was OK, as I just put the WiFi router in the closest and a 5 port switch in the office and all was fine.

It still bugged me though. I want to put a PC in the playroom for the kids (Hi Bila!) and an AppleTV in the bedroom. Problem is that the WiFi upstairs is spotty. I really wanted some wired ethernet for those devices.

Now I was on a mission :-)

The first problem was that none of the cabling used for telephones was labelled. Sigh. Why don't the builders do this? So I needed to trace the wiring. I picked up cabling tester from Frys (the variety where you stick a remote on one end and the main testing unit on the other) along with a crimp tool and various RJ45 and RJ11 blanks1.

First step was to figure out the cabling. I wanted to avoid pulling all the sockets from the walls so I built an RJ45 to RJ11 cable (pretty simple really, the RJ11 is just missing the 2 outside wires from the standard 8 wires in the cable). This allowed me to connect the tester to the phone jacks.

Next I pulled each cable, one at a time, from the phone punchdown block in the closet, crimped on an RJ45 and attached the tester to it. Next I ran around the house testing each jack.

There were eight of them and of course the bedroom jack (which was the one I really wanted to find) was the last one. I got a lot of crimping practice.

And that was it really, all the cable runs are now labelled, the bedroom has a working ethernet jack and Netflix is available in the bedroom. A fun little project. Now I just need a real RJ45 patch panel for the wiring closet…

1 Btw, I really liked these EZ Jack RJ45 and RJ11 connectors that I found at Vetco. They make building the cables a lot easier. 


Every now and then, I think to myself “Steve, the blogging software you use on this site is horribly complicated. Why don't you write your own? I'd be a fun thing to build in some new language that you've been meaning to learn, like Scala, Ruby, etc… You could even host it on AppEngine!”

Then I start thinking about things like how comments are an absolute pain as you'd need to deal with XSS, etc… and I go off and think about something else.

Recently though, I thought “Comments are a pain. They currently look ugly, aren't threaded and I get an incredible amount of spam to deal with. Wouldn't it be great to offload that?”.

So I signed up for Disqus.

I had to fix a few things in the plugin to make it actually work with MoveableType, but now it appears to be up and running.


And one more barrier removed until I actually do write my own blogging system…

© 2001 to present, Steve Lacey.