File based routing with Plug

A lot of frontend frameworks take the design decision for the project file structure to dictate the structure of the API. Let’s mirror that with Elixir’s Plug library.

What is Plug

Plug is an extremely popular web application specification library. Its used in Phoenix (along with A LOT of custom macros) to build up their router. That funny DSL which you will have written if you’ve ever played with a Phoenix application… yep that’s plug.

Plug asks that you write modules and functions in a specific format to abide by it’s specification. Its really simple to implement a module which can be deemed a plug, but the power comes from plugs’ composability.

Since the library knows modules and functions are of a particular shape, composing them together into pipelines becomes possible.

Project setup

Let’s get started by creating a new mix project with a supervision tree (that’s the --sup option).

mix new faucette --sup
cd faucette

We’ll need to add 2 dependencies

• Plug
• Bandit

Let’s add them to our mix.exs file

# mix.exs
def deps do 
	[ 
		{:plug, "~> 1.14.2"},
		{:bandit, "~> 0.6"} 
	] 
end

And install them locally

mix deps.get

Why Bandit

Bandit actually isn’t the default server for Plug to use, most applications will use the Erlang Cowboy library. Bandit, however, is a web server library written purely in Elixir, specifically built for Plug.

It’s also faster, as they state in their README

When comparing HTTP/2 performance, Bandit is up to 1.5x faster than Cowboy. This is possible because Bandit has been built from the ground up for use with Plug applications; this focus pays dividends in both performance and also in the approachability of the code base

Starting our web server

Now we have our chosen libraries installed in our project, we need to tell our Elixir Application to start the server and add it to our supervision tree.

# lib/faucette/application.ex
defmodule Faucette.Application do
  @moduledoc false

  use Application

  @impl true
  def start(_type, _args) do
    children = [
      {Bandit, scheme: :http, plug: FaucetteWeb.Router}
    ]

    opts = [strategy: :one_for_one, name: Faucette.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

Here we’ve told Bandit to startup and pass any HTTP requests to our plug FaucetteWeb.Router, so next up we need to create that router module.

# lib/faucette_web/router.ex
defmodule FaucetteWeb.Router do
	@moduledoc false
	def init(options), do: options

    def call(conn, _opts) do
	    conn
        |> Plug.Conn.put_resp_header("content-type", "text/html")
        |> Plug.Conn.send_resp(200, "<h1>Hello world</h1>")
    end
end

And there we have it. A complete Plug application!

Let’s take it for a spin 🏎️

# we use the --no-halt flag to keep the server alive 
# ready to receive http requests
mix run --no-halt
# >> [info] Running FaucetteWeb.Router with Bandit 0.7.7 at 0.0.0.0:4000 (http)

# in another terminal
curl http://localhost:4000
# >> <h1>Hello world</h1>

Plugs are just modules/functions which abide to the Plug behaviour

Here we have the simplest Plug imaginable. We have an init/1 function which allows for configuring options for the plug (called once at the start of the application). And we have a call/2 function which is called on every request.

File based routes

To begin our file based router, lets create a first module with a get function.

# lib/faucette_web/routes/hello.ex
defmodule FaucetteWeb.Routes.Hello do
  @moduledoc false

  def get(_conn) do
    "<h2>Hello</h2>"
  end
end

So we’re making a couple of design decisions straight out of the gate:

• File based routes will live under lib/faucette_web/routes/*
• Functions in handlers will mirror http methods (GET requests will call get/1)

Let’s go ahead and call our handler from the router we implemented earlier

# lib/faucette_web/router.ex
...
require Logger 

def call(conn, _opts) do
	route(conn.method, conn.request_path, conn)
end

def route("GET", "/hello", conn) do
	body = FaucetteWeb.Routes.Hello.get(conn)

	conn
	|> Plug.Conn.put_resp_header("content-type", "text/html")
	|> Plug.Conn.send_resp(200, body)

end

# catch all
def route(_method, request_path, conn) do
	Logger.debug(request_path)
	send_resp(conn, 404, "not found")
end

Here we’ve also added a catch all route which will log out any request paths which aren’t handled by our route/3 function.

curl http://localhost:4000/hello
# >> <h1>Hello world</h1>

So this is a great start, but we want that NextJS/Javascript framework style of routing - defined by the file structure of the project.

Defining routes at compile time

Elixir has a powerful mechanism for running compilation and generating code before runtime: Macros!

Starting the macro

Let’s begin by moving all of our current logic into the macro

# lib/faucette.ex
defmodule Faucette do
	@moduledoc false

	defmacro __using__(:router) do
		quote do
			require Logger
			
			def init(options), do: options

			def call(conn, _opts) do
				route(conn.method, conn.request_path, conn)
			end

			# catch all
			def route(_method, request_path, conn) do
				Logger.debug(request_path)
				send_resp(conn, 404, "not found")
			end
	    end
	end
end

And we’ll call that macro like so:

# lib/faucette_web/router.ex
defmodule FaucetteWeb.Router do
	use Faucette, :router
end

Nothing else is needed in the consuming module, since everything is defined in the __using__ macro.

Reading from the file system to get paths

So we’ll need a recursive function which will delve into a given directory and pull out all the Elixir modules

defp get_routes(base_path, path) do
	base_path
	|> Path.join(path)
	# list out all files under this base_path + path
	|> File.ls!()
	# iterate over them, reducing them into a new list of maps
	|> Enum.reduce([], fn file, acc ->
		cond do
			# we only care about Elixir files
			Path.extname(file) == ".ex" ->
				full_path = base_path |> Path.join(path) |> Path.join(file)
				# we'll define this below
				module = get_module_from_file_path(full_path)
		
				module
				# list all the functions exported from this module
				|> Kernel.apply(:__info__, [:functions])
				|> Enum.any?(fn {name, _} -> name in ["get"] end)
				|> if do
					# construct a map to be used later
					route = %{
					  path: Path.join(path, Path.basename(file, ".ex")),
					  module: module
					}
		
					[route | acc]
				else
					acc
				end

			# if a directory is found, recurse into this function
			base_path
			|> Path.join(path)
			|> Path.join(file)
			|> File.dir?() ->
				result = get_routes(base_path, Path.join(path, file))
			
				acc ++ result
		
			true ->
				acc
		  end
	end)
end

We’ll then need to use that function, starting at the current path of the calling module and looking in the /routes directory beside it.

# in our __using__ macro
routes_code =
	__CALLER__.file
	|> Path.dirname()
	|> Path.join("routes")
	|> get_routes("/")
	|> Enum.map(fn %{module: module, path: path} ->
		# generate a `route/3` definition using the path 
		# and calling the module's `get/1` to get the body
		quote do
		  def route("GET", unquote(path), conn) do
			Logger.info("GET #{unquote(path)} \n>> calling #{unquote(module)} get/1")
		
			body = unquote(module).get(conn)
		
			conn
			|> Plug.Conn.put_resp_header("content-type", "text/html")
			|> Plug.Conn.send_resp(200, body)
		  end
		end
	end)

The unquote parts in the above let us access the derived values of those variables. For instance path will resolve to “/hello” when the Enum.map hits that module in the list, hence creating

def route("GET", "/hello", conn) do

just like before.

Getting the module name from the file

Above we called a get_module_from_file_path/1 function, let’s define that

defp get_module_from_file_path(file_path) do
    {:ok, contents} = File.read(file_path)

    # \S is equivalent to [^\s]
    pattern = ~r{defmodule \s+ (\S+) }x

    Regex.scan(pattern, contents, capture: :all_but_first)
    |> Enum.map(fn [name] ->
	    String.to_existing_atom("Elixir.#{name}")
    end)
    |> hd()
end

We read the file and apply a regex to get all module definitions returning the first one as an atom.

There are functions in the Code module of the Elixir standard library which allow you to require_file’s and do all sorts of other magic, however they will error if the file/modules have already been defined. This is due to the BEAM generated files which Elixir outputs have no knowledge on their source Elixir files.

A bit annoying but this parsing and regexing is functional, that’s the most important this for this learning experience.

Testing the finished product

With these changes to the macro complete, we should be able to define nested directories of Elixir modules which export handlers.

# lib/faucette_web/routes/nest/bird.ex
defmodule FaucetteWeb.Routes.Nest.Bird do
  @moduledoc false

  def get(_conn) do
    "<h1>Nest bird</h1>"
  end
end

Now lets fire up our server again and check that route works

mix run --no-halt
# >> [info] Running FaucetteWeb.Router with Bandit 0.7.7 at 0.0.0.0:4000 (http)

# in another terminal
curl http://localhost:4000/hello
# >> <h1>Hello world</h1>
curl http://localhost:4000/nest/bird
# >> <h1>Nest bird</h1>

Amazing!

Of course this is quite a contrived example, our router would require a heck of a lot more functionality to be production-ready. BUT this is pretty cool we got it working so easily, a true testament to the flexibility of Elixir!