Thursday, 4 February 2010

Goodbye Blogger

2010 has brought a few changes. I'm now a freelance software engineer with my own company, 1011 Ltd, based in London. This is also ideal opportunity to do something I've been planning for a while - stop using Blogger.

Blogger used to be fine, but it just hasn't kept up. Wordpress, for example, (and there are plenty of others) offers so much more in terms of usability, customizability and administration. Some real pain points for me are:

  • In Blogger you have to work with a minuscule editing window that you cannot enlarge. Wordpress has a fullscreen mode.
  • You can download and install Wordpress to your own domain, even modify it as you like, and have ownership of all your posts.
  • Blogger has doesn't have proper previews using the blog template, whereas the preview in Wordpress is an actual preview.
  • At this very moment the image feature is broken
  • The html in uneditable

It feels to me like Google bought Blogger and then just stop working on it. A bit like Microsoft disbanding the IE team in 2004.

Find me at my new blog,

[Update: And as you can see, the Spam protection is woeful]

Sunday, 23 August 2009

My Google Wave Client

Google released a reference implementation of a Wave client and server in July, written in Java, that you can download and experiment with. When Mickaël Rémond wrote about using the reference implementation with ejabberd, I decided that it was time to get my hands dirty.

I got the reference implementation working, and I could chat with multiple participants in the console clients. However, I wanted to know more about what was happening behind the scenes in the client, and more importantly, the server (and how operation tranforms work).

So I decided to write my own client in Erlang that communicates with the reference server. This way I could investigate what was happening in the client and the server, and gain more knowledge about Wave. It required a bit of reverse engineering, and messing around with protocol buffers (the client communicates with the server using these), but it works nicely.

I've created a video to demonstrate it. Happy watching!

Vimeo link

P.S. I used ngerakines's implementation of Erlang prototocol buffers, and the Nitrogen Web Framework.

P.P.S. I'm going on holiday for 2 weeks, so if I don't reply to comments, please be patient. I will do so when I get back...

Wednesday, 8 July 2009

Google OS - I believe I was right

disclosure: I own Google shares

I'm not one for blowing my own trumpet, but sometimes it feels good to be right. A month ago I blogged about HTML5, and I said the following:

"I believe it is part of Google's long term strategy to make the browser the de-facto application platform. To displace Windows as the dominant aplication platform. You can see it already with Chrome, how the tabs on the browser have become the new taskbar, and the integration of browser tabs as "applications" into the host OS. For example, features like local file storage and the canvas provide a path for Google Docs to become more like MS Office. You can be sure that some of the drive behind the HTML5 standard is coming from the Google docs team."

Yesterday Google announced Google OS: Google OS will be based on Chrome and a Linux kernel.

From the announcement:

"For application developers, the web is the platform. All web-based applications will automatically work and new applications can be written using your favorite web technologies. And of course, these apps will run not only on Google Chrome OS, but on any standards-based browser on Windows, Mac and Linux thereby giving developers the largest user base of any platform."

I will admit that I didn't think they would actually write an OS to support this strategy, but the strategy is clear. Unfortunately we'll have to wait at least a year to see it in action...

Thursday, 2 July 2009

Erlang Factory 2009 - New Kids on the Erlang Block

I went to Erlang Factory again this year thanks to my employer, and it was even more exciting than last year's event. There were some great speakers (Joe Armstrong - entertaining as ever, Simon Peyton Jones), I got to meet some interesting people (e.g. Rusty Klophaus, creator of the Nitrogen web framework) and I got to see some awesome projects. Although Erlang is experiencing a surge of interest, it is by no means "new". it has been used for years in robust and highly scalable applications. That is why I refer to the "new" - as opposed to the solid Erlang VM, tools, libraries and frameworks that have been around for at least a decade.

So, wo are these the New Kids on the Block? These guys?

Nice hair! No, not them. Here are some of the New Kids on the Erlang Block:
Heroku is a platform for deploying Ruby or Rails web apps. They've built their routing mesh in Erlang. This routing mesh is used to load-balance and route requests to their application work units (dynos). This is a really cool piece of cloud infrastructure.

HyperNumbers is a startup that is "doing for numbers what hypertext did for text". Think GoogleDocs, but each cell can be a resource that's located on another spreadsheet. Coming soon.

CouchDB is a non-relational document database written in Erlang, and it is going from strength to strength. Look at this graph from Google Trends:

If you don't know what CouchDB is, you should definitely have a look it. Even is it's just to make you challenge you preconceptions around databases, and relational ones in particular. There is also a book coming out soon. It's open source.

RabbitMQ is an open source, high-performance enterprise messaging system, and an implementation of the messaging standard AQMP. It is written in Erlang. I've heard rumors that ActiveMQ (Java) has 20 times more lines of code. (NB: If anyone can give me a reference or link to proof I would be grateful). Update: See comments for line counts.

This is my favourite web framework. It's a real Web 2.0 framework, and it's highly productive. Some of the main features are:
  • Event-Driven Development
  • Brainlessly Easy Ajax
  • Ridiculously Simple Comet
  • Complex Interfaces: Dragging, Dropping, and Sorting
  • Flexible Templating
  • Data Binding
  • Erlang Power
I've used this to create my own pet Erlang project, Check it out. It's also open source.

I didn't go to this talk, but I've been to one given last year at the Erlang London User group. Smarkets is an online betting exchange which allows users to bet on anything they wish. It is implemented in Erlang and close to launch.

Campfire is one of the applications from the guys at 37Signals. If you don't know who 37 Signals are, they created Ruby on Rails. The functional C polling for Campfire service that was being used was replaced by an Erlang implementation:

The Erlang implementation is as fast as the C implementation, modular and extensible, and much easier to administer with 1 OS process instead of 80.

ProcessOne - OneTeam Media Server
"OneTeam Media Server is a new project launched by ProcessOne which is a Media server for Flash clients implementing the FLV and RTMP protocol. The project is designed so that it can work hand in hand with ejabberd, ProcessOne Instant Messaging platform."

This one is very exciting. The media server has Flash in the front end, and Erlang in the back. It makes thing like writing video chat in the browser ridiculously easy. And it's open source! Look out for it in July.

Webmachine is the best way to implement a REST interface to your Erlang application that I've seen to date. I've blogged about REST and Erlang in the past, but instead of rolling your own, you should look at this. Especially considering how it handles all the response codes correctly, does all the correct caching responses etc. Another open source project.

And the point is...?

I think it's a very positive indicator for the future of Erlang that it is being used in these new and exciting areas and companies. It is performing very well in the field of high-scalability infrastructure. It is well suited to Cloud computing. But most importantly, very clever and innovative people are using it and loving it. That, above all else, means the future looks good for Erlang.

P.S. I'm sure I missed some great talks again this year, but lucky for me and you, you can find the talks and videos here.

Friday, 5 June 2009

Look out RIAs, HTML5 is coming

Google showing off Wave, has [re]ignited some discussion around HTML5. I read an article on InfoQ, Is Google Wave Going to Have an Impact on RIA/Silverlight? Which struck me as a bit of an odd question. Tim Heuer (in the Silverlight camp) is correct in asking "is HTML5 really what people are talking about here?" and towards the end of the article, the correct question is polled:

"What negative impact will HTML 5/Wave going to have on RIA/Silverlight?" [sic]

At the time the results of the poll were:

Major: 66
Significant: 115
Small: 112
None: 43
Total votes: 336

I was a bit surprised by the poll results. I voted "Major", and I find it hard to understand how someone could vote "None". None?! Really? I do realise that this is partly due to the fact that most people, including me, think they are above average (I just discovered that it is also called the Lake Wobegon effect. Cute). There is also some confusion about including Wave in the question, which has nothing to do with the issue. I'll try and convince you of the potential impact...

Why RIAs exist

The existence of Rich Internet Applications can be explained by the word "Rich" in the name. At some point in the past few years, people started to realise that HTML didn't keep up with the explosion in bandwidth and processing power available to users, and that the web experience could be "richer". This was especially obvious in areas such as not having the ability to to arbitrary on-screen animantions and rendering on a "canvas", the ability to store data locally on user's hard drives and drag-and-drop, amongst others. The lack of these features in HTML created a gap in the market, but only whilst browser technology continues to lag. This gap has been exploited by Flex, Silverlight and JavaFX.

This situation is changing.

Enter HTML5

Some of the features in HTML5 will add some really exciting possibilities to applications running on the browser platform:


Although I wouldn't categorise video as a major driver for RIA technologies, it is important to mention in relation to Adobe flash, which is the dominant technology for delivering internet video. YouTube is already delivering non-flash internet video in mobile devices. The introduction of the tag will reduce the pressure of deploying flash on every platform, which could have an effect on the usage of Flex. Consider that Flash is not available on the iPhone: if you have a choice of developing your application on just one technology (HTML5) for both desktop and mobile, or choosing 2, which would cost less?

Canvas drawing (immediate mode drawing)

Canvas drawing will enable application developers to fill a gaping hole in native browser capabilities - gaming. The gaming industry is bigger than the movie industry, and the availability of a virtually universally available standard platform will not got unnoticed by game developers. Of course this capability doesn't only apply to gaming.

Local storage

As long as there's no capability for storing data offline, web applications continue to be considered the poor cousin of desktop applications. Google documents are not availble offline without Google Gears. Having local storage available in a standardised way is a big attraction for application developers. Gears might work for Gooogle, but not everyone is so keen on having a dependency on a non-standard technology.

Some other disavantages to RIAs which could favour HTML5 applications

- RIAs are usually everything or nothing. They don't play well with embedding RIA functionality into existing web pages. HTML5 allows you to grow your application, creating a richer and richer experience as you go along.
- Some RIAs have proprietary technologies.
- RIAs require an extra deployment step on top of deploying the browser. Browsers implementing HTML5 will provide a more standardised way of deploying applications to users, also in the corporate environment where locked-down systems are common. Choosing a technology that doesn't require another installation step is a big attraction for corporate IT departments.

In the long term

There another significant point to make.

I believe it is part of Google's long term strategy to make the browser the de-facto application platform. To displace Windows as the dominant aplication platform. You can see it already with Chrome, how the tabs on the browser have become the new taskbar, and the integration of browser tabs as "applications" into the host OS. For example, features like local file storage and the canvas provide a path for Google Docs to become more like MS Office. You can be sure that some of the drive behind the HTML5 standard is coming from the Google docs team.

In closing

I'm not trying to say that HTML5 will eliminate RIA technologies, but I do believe that thinking that its impact on RIA will be "small" or "none" is absolutely ridiculous.

There are some big players pushing the browser to become the dominant application platform. HTML5 is a big step in that direction.

If you disagree, I look forward to your comments!

Wednesday, 20 May 2009

The Law of Conservation of Misery

At university I had a professor who would jokingly refer to his Law of Conservation of Misery:

The total amount of misery in a system is constant

which implies that when you try and decrease the misery in one aspect of a system, you will increase the misery in some of the other aspects.

There are many types of misery. In software engineering projects these could include non-maintainability, non-performance and budget constraints.

Personally, I think people are too prone to think that there are zero-sum monsters lurking under each bed, so my own take on the "law" would be:

The total amount of misery in a system is > 0

Which implies that you can make tradeoffs that decrease the total amount of misery, but you can never eliminate it completely. A bad implementation of a system will have a higher total amount of misery, and a good one will have less, but you will always have at least a little bit!

Tuesday, 19 May 2009

How to send email via Gmail using Erlang

One of my pet projects,, integrates with email to send notifications to users.

I use Google Apps for email infrastructure, so you need an SMTP client that supports TLS. At the time, I couldn't find a simple Erlang SMTP client that could handle TLS, so I used a command-line SMTP client.

For my new pet project, for want of a better name temporarily called The Isabelle Project, I need to add some email functionality. This time I would prefer to use an Erlang solution with proper error handling and logging.

I looked at the SMTP protocol on Wikipedia, and it didn't seem to difficult. Erlang's built-in ssl module also seemed to support TLS. So, with a bit of trial and error, here's the result:


connect() ->
{ok, Socket} = ssl:connect("", 465, [{active, false}], 1000),
send(Socket, "HELO localhost"),
send(Socket, "AUTH LOGIN"),
send(Socket, binary_to_list(base64:encode(""))),
send(Socket, binary_to_list(base64:encode("johngalt"))),
send(Socket, "MAIL FROM: <>"),
send(Socket, "RCPT TO:<>"),
send(Socket, "DATA"),
send_no_receive(Socket, "From: <>"),
send_no_receive(Socket, "To: <>"),
send_no_receive(Socket, "Date: Tue, 15 Jan 2008 16:02:43 +0000"),
send_no_receive(Socket, "Subject: Test message"),
send_no_receive(Socket, ""),
send_no_receive(Socket, "This is a test"),
send_no_receive(Socket, ""),
send(Socket, "."),
send(Socket, "QUIT"),

send_no_receive(Socket, Data) ->
ssl:send(Socket, Data ++ "\r\n").

send(Socket, Data) ->
ssl:send(Socket, Data ++ "\r\n"),

recv(Socket) ->
case ssl:recv(Socket, 0, 1000) of
{ok, Return} -> io:format("~p~n", [Return]);
{error, Reason} -> io:format("ERROR: ~p~n", [Reason])

And the output from the Erlang shell:

3> application:start(ssl).
4> smtp:connect().
"220 ESMTP y37sm613282mug.19\r\n"
"250 at your service\r\n"
"334 VXNlcm5hbWU6\r\n"
"334 UGFzc3dvcmQ6\r\n"
"235 2.7.0 Accepted\r\n"
"250 2.1.0 OK y37sm613282mug.19\r\n"
"250 2.1.5 OK y37sm613282mug.19\r\n"
"354 Go ahead y37sm613282mug.19\r\n"
"250 2.0.0 OK 1242683885 y37sm613282mug.19\r\n"
"221 2.0.0 closing connection y37sm613282mug.19\r\n"

The only tricky bit is that for the AUTO LOGIN, the received text and the username and password you send is base-64 encoded. By default the connect is active=false, which means the responses are send to the creating process directly. Using passive mode requires explicit receiving of the response using ssl:recv/2

You'll have to handle errors better if you use this in production, but the basic protocol is pretty straightforward...