An experiment with sidekiq, ruby and go

Go has been getting some visibility these past few months. It has already proven to be highly scalable in production environments.

Sidekiq being a background job processing I’ve been wanting to try a bit, and go-workers having just landed into the Open Source world a few weks ago, let’s use them to compare running sidekiq jobs with ruby and go.

Running the jobs

We’re going to try a very simple benchmark, just to demonstrate enqueuing jobs in ruby and processing them both in ruby and in go.

require 'sidekiq'
1000.times do
  Sidekiq::Client.push('queue' => 'goqueue', 'class' => 'myJob', 'args' => [])
  Sidekiq::Client.push('queue' => 'rubyqueue', 'class' => 'MyJob', 'args' => [])
end

We have two different queues, one for go and one for ruby. And for each of these queues, we have a job MyJob.

You can already run this code. It will enqueue our jobs. But they won’t be processed, as we don’t have any worker.

About the “benchmark”

Talking of a “benchmark” is a bit abusive. Ruby and Go both have their advantages. Go will have the big advantage of paralellism. While Ruby will have the advantage of a pleasant API and a lot of open source libraries.

And that’s all right. A language can’t be perfect everywhere, and while ruby remains awesome for building a web application, it’s not ideal for the time-consuming jobs which need to happen in the background.

So this benchmark is indeed biased. Sidekiq will be just as fast between both languages.
Therefore, we’re just running a benchmark between ruby and go, which won’t ever mean anything, as it’s too dependent of the kind of job you need to do.

However, the fact that we cannot just say “yay, go is faster” means we have to get both of the awesomeness of both languages. That’s the real idea behind this article.

The Ruby Worker

Since Sidekiq is written in ruby, it’s going to be very easy to run a worker in that language.

require 'sidekiq'
require 'open-uri'

class MyJob
  include Sidekiq::Worker
  def perform(*args)
    puts fibonacci(30)
  end

  def fibonacci(n)
    return n if n <= 1
    fibonacci(n - 1) + fibonacci(n - 2)
  end
end

That’s all. We’re calculating the fibonacci sequence of 30 in each job.

The Go Worker

For the go worker, we need to use the go-workers library, which we already mentioned before.

package main

import (
  "github.com/jrallison/go-workers"
)

func fibonacci(n int) int {
  if n <= 1 {
    return n
  }
  return fibonacci(n - 1) + fibonacci(n - 2)
}

func myJob(args *workers.Args) {
  println(fibonacci(30))
}

func main() {
  workers.Configure(map[string]string{
    "server":  "localhost:6379",
    "pool":    "25",
    "process": "1",
  })

  workers.Process("goqueue", myJob, 25)

  go workers.StatsServer(8080)
  workers.Run()
}

This looks a bit more complicated since we need to manually start the worker. The worker content is, however the same, calculating the fibonacci sequence of 30 every time the job is run.

Conclusion

As I already mentioned, this is not a reliable benchmark, since we’re comparing two very different languages and approaches.
However, it’s interesting to mention that on a 2-years old iMac, each Go job takes around 15ms, while each ruby job takes around 70ms.

The conclusion of this, if there is any except showing a code sample of Go is that building polyglot applications will allow you to much more finely use the advantage of each language and become very profitable to the overall speed of your platform.