An adventure into learning a bit of erlang

It is common knowledge that Heroku has a fair number of Erlang components, we even published a book.
When I joined, 3 years ago, I had barely heard about it.
But as I tried to get a better grasp on how the platform works, I wanted to learn how some of those erlang components work.

Better understanding erlang took me roughly those 3 years (on and off obviously, not full time). I started by reading learn you some erlang.

At that point, I was amazed by what the language offered, and yet I didn’t understand it all.
But I kind of fell in love (don’t tell my wife that) with functional programming and pattern matching.

Then I didn’t practice and stopped caring about it for almost two years.
Last fall, I went back to it, successfully this time. This is what I want to share in that article.

Getting started

It seems all erlang documentation assumes either an already working application, or a simple console. But getting started building a new app is barely documented.
When in fact it is fairly simple. Just use rebar3.

This definitely didn’t come as an obvious thing. Managers have a rebar package available. But it seems to be rebar 2, which isn’t even on the same github organization.
Rebar3 on the other end is not available in my package manager (homebrew), and required manually compiling it and moving it to my PATH. I ended up storing the compiled binary inside my dotfiles repository.

Hello, World

Once I was able to install rebar3 and bootstrap a new app, things went a lot more smoothly.
Within about a week of working for about 30 minutes to 1 hour each week day on it, I was able to boot a very simple erlang TCP server.
That TCP server linked above doesn’t do much. It responds with exactly the same content the request sent to it. But it works!

At that point, I was in love the language again. But this time, it was the processes and supervisors mechanisms which attracted my attention.
Realizing that I was able to write the equivalent of ruby’s unicorn in about 10 lines of erlang opened a lot more possibilities.

Building something useful

My idea here wasn’t actually to build something useful, but something which did something (other than just a mime artist which answers what you just told it).
So I started building a key-value store. But to make it more fun, and because I was missing my mime artist, I made it mimick redis.

So the idea is an erlang TCP server which can understand a redis client, with only three commands: GET, SET and FLUSHALL. But built so that adding more commands would be straightforward.
I won’t go through the entire code here. It is available here: github/dmathieu/sayagain.

The best jokes aren’t explained. But I’ll do an exception with that one.
In french, “redis” means “say again”. “redis moi ├ža” can be translated into “say that again”, hence the name sayagain.

The processes started by that app look like the following:

sayagain observer

There are apps with nicer processes graph. But that one is quite self-explanatory on how the app works (says the guy who wrote the app, so that’s absolutely not biased).

say_tcp_sup is a supervisor which boots TCP listeners. It has a pool of 20 by default. So when our app idles, we will always have 20 processes waiting for new connections.
When a new connection comes in, it will be assigned to one of those listeners.

That minion will start by booting a new listener so we can process another request.
Then it will parse the data sent, expecting RESP.

Once the data is parsed, we will get a tuple similar to this:

get hello

From there, we can execute that command, to get and/or mutate the data and return the value redis expects for that command.
Finally, we go back to our command parser, this time to parse the erlang tuples back to being readable by redis.

All that in under 200 lines of code (including the TCP server). I haven’t spent time benchmarking it yet. But the architecture should allow it to handle thousands of clients in parallel.

What’s next?

This has been an awesome learning experience, and the more I play with erlang, the more I love it.
I get that the syntax can be a factor preventing people from trying it, and I am the first one to get that feeling.

For that reason, I have started playing with elixir too. This time, I am building a small API which you might hear about in the coming months (or, not, if I get bored before).

What’s sure though is that learning some Erlang is giving me better inspiration for the day-to-day work apps I’m building in Go and Ruby.
In other words, it’s making me a better programmer.