Visualizing Parallel Requests in Elixir

We have been evaluating Elixir at Codemancers and today I was learning how to spin up a minimal HTTP API endpoint using Elixir. Like Rack in Ruby land, Elixir comes with Plug, a swiss army knife for dealing with HTTP connections.

Using Plug to build an HTTP endpoint

First, let's create a new Elixir project:

$ mix new http_api --sup

This creates a new Elixir OTP app. Let's add :cowboy and :plug as hex and application dependencies:

# Change the following parts in mix.exs

  def application do
    [applications: [:logger, :cowboy, :plug],
     mod: {HttpApi, []}]
  end

  defp deps do
    [
      {:cowboy, "~>1.0.4"},
      {:plug, "~>1.1.0"}
    ]
  end

Plug comes with a router which we can use to build HTTP endpoints with ease. Let's create a module to encapsulate the router:

# lib/http_api/router.ex
defmodule HttpApi.Router do
  use Plug.Router

  plug :match
  plug :dispatch

  get "/" do
    send_resp(conn, 200, "Hello Plug!")
  end

  match _ do
    send_resp(conn, 404, "Nothing here")
  end
end

If you have worked with sinatra-like frameworks, this should look familiar to you. You can read the router docs to understand what everything does if you are curious.

To start the server, we'll tell the application supervisor to start the Plug's Cowboy adapter:

# lib/http_api.ex

defmodule HttpApi do
  use Application

  def start(_type, _args) do
    import Supervisor.Spec, warn: false

    children = [
      # `start_server` function is used to spawn the worker process
      worker(__MODULE__, [], function: :start_server)
    ]
    opts = [strategy: :one_for_one, name: HttpApi.Supervisor]
    Supervisor.start_link(children, opts)
  end

  # Start Cowboy server and use our router
  def start_server do
    { :ok, _ } = Plug.Adapters.Cowboy.http HttpApi.Router, []
  end
end

The complete code for the above example can be found here. You can run the server using:

$ iex -S mix

This starts the interactive Elixir shell and runs your application on the Erlang VM. Now comes the fun part.

Visualizing processes using :observer

In the iex prompt, start the Erlang :observer tool using this command:

iex> :observer.start

This opens a GUI tool that looks like this:

On the left hand side of the Applications panel, you can see a list of all the applications currently running on the Erlang VM - this includes our app (http_api) and all its dependencies, the important ones being cowboy and ranch.

Cowboy and Ranch

Cowboy is a popular HTTP server in the Erlang world and it uses Ranch , another Erlang library, to handle TCP connections behind the scenes. When we start the Plug router, we pass on the router module to Plug's Cowboy adapter. Now when Cowboy receives a connection, it passes it over to Plug, and Plug runs it through it's plug pipeline and sends back the request.

Concurrent Requests

Plug by default asks cowboy to start 100 TCP connection acceptor processes in Ranch. You can see the 100 acceptor processes for yourself if you see the application graph of ranch using :observer.

Does this mean that we can only have 100 concurrent connections? Let's find out. We'll change the number of acceptors to 2 by passing it as a parameter to Plug's Cowboy adapter:

Plug.Adapters.Cowboy.http HttpApi.Router, [], [acceptors: 2]

Let's see the how the processes look like now:

Okay, so we've got only 2 TCP connection acceptor processes running. Let's try making 5 long running concurrent requests and see what happens.

# lib/http_api/router.ex

# Modify router to add some sleep
defmodule HttpApi.Router do
  use Plug.Router

  plug :match
  plug :dispatch

  # Sleep for 100 seconds before sending the reponse
  get "/" do
    :timer.sleep(100000)
    send_resp(conn, 200, "Hello Plug!")
  end

  match _ do
    send_resp(conn, 404, "Nothing here")
  end
end

Let's make 5 requests now by running this in the iex prompt:

for n <- 1..5, do: spawn(fn -> :httpc.request('http://localhost:4000') end)

Start :observer from iex using :observer.start and see the process graph:

We can see that there are only 2 acceptor processes still, but 5 other processes were spawned somewhere else. These are connection processes which hold accepted connections. What this means is that, acceptor processes do not dictate how many processes we can run at a time, it just restricts how many new processes can be accepted at a time. Even if you want to serve 1000 concurrent requests, it's safe to leave the number of acceptor processes at the default value of 100.

Summary

• You can build simple HTTP endpoints using Plug router
• Ranch can handle multiple TCP connections at a time by spawning processes
• Erlang :observer is a great way to visualize concurrency in your apps
• Acceptor processes only accept connections. You only need 100 of them.