HTTP Streaming in Golang

For the past few months, I’ve been experimenting with go quite a bit.

Finally, I’m coming up with an actually useful library and not something just built for the purpose of tests.
Let me introduce gofire.

Gofire is a library for talking to Campfire. Currently, you can send messages to a room, and listen to incoming messages.

While sending a message is pretty straightforward, since it’s just about sending a POST request, opening a streaming connection is relatively more advanced.
So I thought I would walk through it here.

Establishing the streaming connection

We need to use net/http/httputil to open a persisted HTTP connection, which requires that we first create a new TLS connection.

tcpConn, err := net.Dial("tcp", c.path.Host+":443")
cf := &tls.Config{Rand: rand.Reader}
ssl := tls.Client(tcpConn, cf)

c.clientConn = httputil.NewClientConn(ssl, nil)

We’re opening a new connection to the server on port 443 (since the connection is encrypted with SSL).

We’re then creating the tls configuration object.
Here, we’re manually specifying the random number used for encryption. But if we didn’t provide it, golang would generate one itself.

Finally, we’re generating the new client object.
And, of course, we use this client to open the new connection.

The next part of the connection is quite similar to what we’ve considered being straightforward earlier in sending messages.

req, err := http.NewRequest("GET", c.path.String(), nil)
resp, err := c.clientConn.Do(req)

We create an http request.
And we retrieve a response by actually executing this request with the client previously defined.

Listening for new messages

Now that we are connected to the streaming server, we can start listening to incoming message.
In order to do so, we’re going to use the bufio library.

reader := bufio.NewReader(resp.Body)
line, err := reader.ReadBytes('\r')
line = bytes.TrimSpace(line)
fmt.Println("I just received the message %s", line)

bufio opens a new I/O connection. Inside which we’re using ReadBytes to wait for any new line and treat it accordingly.

This portion of code will wait until the streaming connection send a new paragraphe (\r), continue the program at that moment.
So we’re waiting until we get an entire line, and display it.

Since we want to be waiting for new lines as long as the program is working though, we’re gonna loop indefinitely inside that part.

reader := bufio.NewReader(resp.Body)
for {
  if c.stale {
    c.clientConn.Close()
    break
  }

  line, err := reader.ReadBytes('\r')
  line = bytes.TrimSpace(line)

  fun(line)
}

We’re basically doing the same as above here, except we’re looping indefinitely when receiving new lines.
The only thing which could break our loop is marking the connection as stale, which will close it and stop the execution.

The fun method is a callback used internally to send each new line to the room object.
We could definitely be using a channel there. But since we’re already using one in the public API to send the final message object, I wanted to avoid creating too many threads.

Parsing the message and sending it back

Now that we have received a message, it’s being transmitted to our callback.
Inside which we retrieve the content of the message, in byte format.

var message Message
err := json.Unmarshal(content, &message)

We’re then generating a new message object and unmarshaling the byte content inside it.
Starting now, we have access to a Message object with it’s values filled from the content the API gave us.

We can send this object to the public channel.

channel <- message

The public API

Now that we’ve seen how the streaming works under the table, we can more easily understand the public API.

client := gofire.NewClient("<your API token>", "<your subdomain>", true)
room := client.NewRoom("<your room id (not it's name)>")

In which we create a new client with our subdomain and API token, along with a room.

channel := room.Listen()

Inside our room, we can start listening for new messages.
After starting listening, the connection to the streaming server is established and we’re ready to receive messages.

for {
  msg := <-channel
  fmt.Println(msg.Body)
}

So we loop indefinitely, listening for new messages in the channel.
Whenever one is received, go will automatically continue to the next line, which prints the message.

Conclusion

In this example, I’ve taken for granted the golang basics.
If you got lost and just scrolled down here, I recommand that you read the golang book.