Random Thoughts

Tech, words and musings from an Englishman in Seattle

LWB Is Now Open Source

I recently talked about how I implemented “LWB”, the software that powers this blog, using Go. Now, I'm happy to announce that I've now released it as open source - well, technically I released it a few weeks ago…

The code is all hosted on GitHub, along with a sample blog that looks very much like this one :-)

Enjoy and contribute!

LWB on GitHub.

HTML5 Media Statistics

Recently I've been spending some time getting together a proposal for the HTML5 <video> and <audio> tags that enables web applications to gather statistics about how media playback is performing.

A lot of this was inspired by Mozilla's statistics support and the WebKit support I added at the beginning of the year (which shipped in Chrome 10).

So after a bunch of feedback, today I added a proposal to the WHATWG wiki — hopefully it won't get crushed too hard and we can get this into the HTML5 spec soon.

The key points are the ability to monitor download and decode bitrates as well as the presentation statistics and an interesting little metric called “jitter” that can give the developer feedback on perceived framerate quality.

Anyhow, take a look and give feedback on the list.

Flow

Over on his blog, Tim Bray describes his move to a more press-like hyphenation and justification model for the layout of his blog. I like it.

You should read his blog post for the full details, but overall, I agree. Ragged Right looks fairly icky — though you've probably never noticed it and there's certainly a lot of disagreement out there on the interwebs. However, after implementing exactly the same solution as Tim for fully hyphenated and justified text, I think my blog looks just that little bit prettier.

I hope you agree.

Mini Film Review: SuckerPunch

As I haven't done one of these in a while, I thought I'd start again with SuckerPunch.

I saw the film late Saturday night and wasn't sure what I was expecting, suffice to say that the film delivered.

Overall, I liked it — but then I have a penchant for films about women kicking ass.

SuckerPunch grabs you from the outset with some pretty gritty scenes. I don't believe it's a spoiler to say that the premise of the film is of an abused girl who is sent by her abusive step-father to a mental asylum. At the moment of her awful final treatment she retreats into a fantasy world as a dancer in an environment layered over the institution that she has been sent to.

As she plots her escape with new-found friends, each “dance” is represented by an increasingly fantastical battle scene. Think an amalgam of SteamPunk, Scott Pilgrim vs. the World, Sky Captain and the World of Tomorrow and Kill Bill.

The plot mechanic of fantasy within fantasy as escape was interesting (think Inception), and at times some of the dialog got a little strained.

But overall, the film was a well worth the watch. At times sad and thoughtful, and at times thoroughly kick-ass.

Those expecting a simple “chicks with guns” film will get a little more than they bargained for…

Reinventing the Wheel

A few weeks ago I noted that I really wanted to rewrite the blogging platform that I use for “Random Thoughts” from scratch. A number of reasons prompted this:

  • It's way too slow. Page load times are awful.
  • It's way too complicated. MovableType is written in a mixture of perl and php and as a platform it has become way too bloated for my own personal use. I have little clue how it works and little desire to learn.
  • The future of the platform appears uncertain.
  • Upgrades can be painful.
  • It needs a full instance of MySQL behind it, with all the associated management headaches that come along with it.

But mostly I wanted to just write my own.

What language?

The first order of business was which language to use. I wanted to learn something new.

I first contemplated using Scala, and had actually gone down that route for a while. Unfortunately my brain exploded after a couple of nights coding.

Scala is pretty and all, but damn does it have a steep learning curve, even for someone moderately familiar with functional languages and very familiar with Java. I was spending way too long digging around in library code and documentation and still couldn't figure out how a simple generator pattern worked. Seriously, what breakOut is doing here?

scala> val s = Set(1, 2, 3)            
s: scala.collection.immutable.Set[Int] = Set(1, 2, 3)

scala> val m: Map[Int, Int] = s.map(k => (k, k * 10))(breakOut)

Anyhow, I gave up. To much reading, not enough coding.

Next up: Go.

I've been wanting to get my teeth into Go for a while and this seemed like the perfect project. After an initial ramp up reading the superb documentation and dicking around with a few toy samples I was off to the races. Code was figuratively dripping from my fingers.

Database?

One big choice was still to be made, though: What database system should I use to store the blog entries in?

On the face of it, there seems an obvious answer: Just use MySQL. The bindings exist and I know how to use it.

But then I stood back and thought about it.

  • There's only just over one thousand posts and it's not likely to increase by a factor of ten any time soon.
  • The indexes needed are well know ahead of time: by date, by tag and by category.

So why not just store them all as flat files? And that's what I did. Each post is a separate file that contains a single JSON object representing the post. I load them all up at startup and pre-build all the indexes. Nice and fast and works like a charm - no dependencies needed.

The only slightly tricky thing I needed to write was a python script that converts the MovableType export file to the JSON files.

Web framework?

When it comes to actually serving the pages, Go's built in http library was almost perfect, but I ended up using a framework developed by a friend of mine: Gary Burd's Twister. The library provides lots of fun stuff for URL handlers, pattern matching, default handlers including redirects, etc… Highly recommended.

Textile

I use Textile as the markup language for all my blog posts, so I needed a Go template formatter for it. Unfortunately none currently exist that I can find, so of course I had to write my own and this ended up taking more time than any other piece of this project, and it only supports the small subset of Textile that I actually use. I sincerely hope that someone else writes a full implementation of the textile parser for Go so that I can throw my own away…

Caching

A key goal of this whole project was to reduce the latency of my blog down to as close to nothing as possible. Unfortunately the textile parser and page rendering take time, especially for some of the larger archive pages. So I needed to cache, but what to use?

An obvious choice here is Memcached, but there's only one instance of the server and it adds another moving piece. So what the hell, I might as well just cache the contents of each page render and the conversion of each post from textile to html (as each post appears on multiple archive pages as well as it's own page). A quick back of the envelope calculation yields a few tens of megabytes needed. No biggy.

Design

As I was rewriting the blogging platform, I might as well do a complete revamp of the layout of the blog. So the design is a complete rewrite using HTML5, using all the key buzzwords including web fonts. Take a look at the colophon for more details.

Was it worth it?

Absolutely. The text you're currently reading came from the platform in one form or another, so it seems to be working.

The only real missing piece is on the authoring side. Right now I just run a python script to create each new entry and emacs to edit the content. Not big deal for me, but it would be nice to provide API access so that tools such as MarsEdit (my favourite blog content creation tool) can be used with it.

And of course it needs a name. Software, I christen thee “Light Weight Blogging”.

Fun With Home Wiring

When we moved into our place a few years ago I was happy to find that the place was wired with Cat5e, all run back to a central wiring closest in the house. Unfortunately though, every jack in the house was wired to phone (RJ11) jacks except for two - one in the office and one by the kitchen desk - I really want RJ45 everywhere.

This was OK, as I just put the WiFi router in the closest and a 5 port switch in the office and all was fine.

It still bugged me though. I want to put a PC in the playroom for the kids (Hi Bila!) and an AppleTV in the bedroom. Problem is that the WiFi upstairs is spotty. I really wanted some wired ethernet for those devices.

Now I was on a mission :-)

The first problem was that none of the cabling used for telephones was labelled. Sigh. Why don't the builders do this? So I needed to trace the wiring. I picked up cabling tester from Frys (the variety where you stick a remote on one end and the main testing unit on the other) along with a crimp tool and various RJ45 and RJ11 blanks1.

First step was to figure out the cabling. I wanted to avoid pulling all the sockets from the walls so I built an RJ45 to RJ11 cable (pretty simple really, the RJ11 is just missing the 2 outside wires from the standard 8 wires in the cable). This allowed me to connect the tester to the phone jacks.

Next I pulled each cable, one at a time, from the phone punchdown block in the closet, crimped on an RJ45 and attached the tester to it. Next I ran around the house testing each jack.

There were eight of them and of course the bedroom jack (which was the one I really wanted to find) was the last one. I got a lot of crimping practice.

And that was it really, all the cable runs are now labelled, the bedroom has a working ethernet jack and Netflix is available in the bedroom. A fun little project. Now I just need a real RJ45 patch panel for the wiring closet…

1 Btw, I really liked these EZ Jack RJ45 and RJ11 connectors that I found at Vetco. They make building the cables a lot easier. 

Experimentation

Every now and then, I think to myself “Steve, the blogging software you use on this site is horribly complicated. Why don't you write your own? I'd be a fun thing to build in some new language that you've been meaning to learn, like Scala, Ruby, etc… You could even host it on AppEngine!”

Then I start thinking about things like how comments are an absolute pain as you'd need to deal with XSS, etc… and I go off and think about something else.

Recently though, I thought “Comments are a pain. They currently look ugly, aren't threaded and I get an incredible amount of spam to deal with. Wouldn't it be great to offload that?”.

So I signed up for Disqus.

I had to fix a few things in the plugin to make it actually work with MoveableType, but now it appears to be up and running.

Enjoy…

And one more barrier removed until I actually do write my own blogging system…

Undo

A couple of weeks ago I left Facebook and returned to Google. I enjoyed working with the engineers there on fun and interesting stuff but for a number of reasons, mostly personal, I decided that the best place for me is Google. And here I am!

It's great to be back at Google - it's amazing how much can change when you're away for a even just a couple of months.

So what am I working on? Interestingly, this time I can actually talk about it! On rejoining Google, I had the chance to look around at a number of projects (there's lots of great stuff going on up here in Seattle/Kirkland), and I'm happy to say that I'm now working on Chrome and specifically on the video stack. It's going to be fun working on Open Source stuff as my day job!

So after a four year break, it's back to client-land for me!

Thank you Loon, whoever you are!

On Wednesday I flew down to San Francisco on Virgin America. After picking up my bags I headed to the car rental place, picked up the car and proceeded to attach the windshield mount for my Nexus One which I use for GPS navigation.

One problem. I couldn't find the phone.

Damn! It must have dropped out of my bag in the overhead storage.

So I proceeded to call the airline's lost and found, who said that they'd check the plane and call me back. Did I say call me? Yup. Handily I had another cell phone with me (never underestimate the usefulness of traveling with two cellphones!). I also use Google Voice, which means that everyone can still call me as it routes to both cellphones.

A little later I got a call from Loon, an employee at the airport, who had found my cell on the airport transit. He said he'd hand it in to the airline and asked when I was flying back. I gave him the details and thanked my lucky stars - I must have stored up a Karma surplus.

Then this evening when I checked in for my flight, Virgin America had my cellphone waiting for me!

So thank you Loon, whoever you are, and the wonderful staff at Virgin America!

Ch-Ch-Ch-Changes - Hello Facebook!

As most of you probably know by now, at the end of August I left Google to join Facebook, based out of their new Seattle office.

It was definitely a big change for me. I really enjoyed my time at Google working on some super cool tech with a bunch of super smart people and hadn't been thinking about leaving the big G. But when Facebook approached me with the idea of joining the fledgling Seattle office (and after a bunch of conversations, interviews, etc…) the opportunity and idea of the sheer impact that an engineer at Facebook can deliver grew on me to such an extent that I couldn't ignore it.

Of course it does mean traveling across the bridge every day…

Right now I've just finished week four of Bootcamp down at Facebook HQ in Palo Alto. Bootcamp is a really cool experience - six weeks packed full of lots of lectures; meeting potential teams that I might be working with; and lots of bug fixing. Indeed, there is an incredibly high likelihood that if you've used Facebook at all in the past month, you've come into contact with my code.

I've fixed a bunch of stuff now - both frontend and backend - it's a great way to get a solid grounding in the codebase and development environment. Right now I'm moving onto some work with a team I may well end up joining.

All told, I'm really happy and excited to be here. Onward!

OSCON

I'll be in Portland attending O'Reilly's OSCON next week.

I've never been before, so I'm quite looking forward to it! In addition, I'm driving there, so it'll be the first real long drive I've done in the car. I can't believe I've had it for over 6 years and never traveled any real distance…

Grey Goo

I was just browsing Hacker News and was intrigued by the link to Grey Goo.

Nanotechnology has been a fascination of mine since 1986 when I first read K. Eric Drexler's wonderful book Engines of Creation. If you haven't read it, I highly recommend that you do. It's a wild ride through our probable future.

Anyhow, I was particularly struck by one of the paragraphs in the wikipedia article that alludes to the class of bugs that all of us as programmers strive to avoid:

In a History Channel broadcast, grey goo is referred to in a futuristic doomsday scenario: “In a common practice, billions of nanobots are released to clean up an oil spill off the coast of Louisiana. However, due to a programming error, the nanobots devour all carbon based objects, instead of just the hydrocarbons of the oil. The nanobots destroy everything, all the while, replicating themselves. Within days, the planet is turned to dust.”

Indeed.

Let's not do that.

Episode V

What an astoundingly great review of The Empire Strikes Back

Their dialogue fairly crackles, from the first exchange in the corridor of Hoth (“Afraid I was going to leave without giving you a goodbye kiss?” “I'd sooner kiss a Wookie!”) to the famous last lines before Han is put into carbonite: “I love you.” “I know.” Incidentally, never ever say that to a girl when she utters those words unless you are actually about to be frozen in carbonite.

Recommended for nostalgia reasons alone.

Tip'O'Hat to John Gruber for the link.

I'm Living In The Future

I'm currently cruising at 35,000 feet on a Virgin America flight from San Francisco to Seattle. I'm surfing on my iPad whilst being served a Rum and Coke by a very nice flight attendant.

Being upgraded to first class didn't hurt either.

Seriously, Virgin America is the best airline I have ever flown. Friendly; a little bit irreverent; spotlessly clean. Recommended.

Did I mention the WiFi?

The Very Essence Of Fatherhood

As usual, Dadcentric sums it up.

I have slammed doors and stood behind them as you cried yourself to sleep.

I have slept in your bed, curled around you like a blanket and felt my legs grow slowly numb.

I heal your wounds and you fix me when I am broken. We meet in the middle and find much happiness there.

Seriously, Dadcentric has some of the best writing out there.

Magnetic Scrolls: A Interview with Michael Bywater

Here's a little gem I just found on the interwebs: an interview with Michael Bywater. Michael is an astounding wordsmith; writer of my favourite columns in Punch and allegedly slept with my boss. Not my current boss, mind you.

Some gems:

My main memories are of course of Anita. And her bull terrier, Murdoch. And of the chaotic offices in the ancient and haunted part of London called The Borough, a warren of streets and alleys on the south side of London Bridge…

and…

Even things like Myst - which, hell, was just a droopy post-hippie HyperCard stack with a rather good music loop -- were way below the level of Magnetic Scrolls or Infocom in narrative terms. So the era came to an end.

My time at Magnetic Scrolls was a wonderful time of my life and I enjoyed every moment of it. Hopefully this article will give you some sense of what it was like for the non-engineering folks who passed our way.

Read it over here.

Bringing Nothing To The Party

Paul Carr's wonderful tell-all book Bringing Nothing To The Party is now available as a free ebook.

You can read about the process about bringing it out as a free ebook at techcrunch, download it directly here, or read it online via Google Docs…

It's a great story about startup culture in the UK and the jounalist's involvement in it.

Taken directly from the Amazon editorial review:

Bringing Nothing To The Party, is the bizarre, hilarious and nauseatingly true story of a unique group of hard-partying high-achieving young entrepreneurs and one man's attempts to join them, whatever the cost.

As a journalist covering the first dot.com boom, Paul Carr spent his life meeting the world's most successful young Internet entrepreneurs. And in doing so he came to count many of them amongst his closest friends.

These friendships meant he was not only able to attend their press conferences and speak at their events, but also get invited to their ultra-exclusive networking events in London and New York, get drunk at their New Year parties in their luxury penthouse apartments and tag along when they threw impromptu parties at strip clubs after raising tens of millions of pounds in funding.

In 'Bringing Nothing to the Party', Paul uses his unparalleled (and totally uncensored) access to tell the real story of a unique group of hard-partying, high-achieving young entrepreneurs - and his attempts to join them, whatever the cost.

Recommended.

© 2001 to present, Steve Lacey.