Run, Dynamo, Run!

Dynamo is a web framework that runs on Elixir. It leverages the power of the Erlang VM to build highly performant and concurrent web applications. Dynamo goals are performance, robustness and simplicity.

Dynamo shows excellent performance out of the box beating similar frameworks like Sinatra and Express, available in other languages, in a callback-free programming fashion. By using the Erlang VM all of the network I/O is asynchronous but your code appears to be synchronous. The Erlang VM also allows you to use all cores available by default, without a need to start many instances of your web server, and performs well under heavy load with many concurrent open connections.

On the developer side, Dynamo focuses on simplicity by shipping with a bare stack, allowing a team to get started quickly while making it easy to extend the application as and when they see fit.

It is currently alpha software and it supports:

• Light-weight connections with streaming.
• Code organization in environments.
• Code reloading for fast development cycles.
• Lazy parsing and handling of cookies, sessions, params and headers.
• Handling of static assets.
• Template rendering.
• An exceptional exception handler for development.
• Integration with Erlang OTP applications.

Before becoming beta, we want to add the following to Dynamo:

• Logging.
• Websockets support.
• Built-in JSON encoding.
• Database adapters.

This README will go into Dynamo installation and a basic walk-through.

Documentation for Dynamo is available at elixir-lang.org/docs/dynamo.

As an alpha-software, Dynamo installation is a bit manual but can be done in few steps:

1. Ensure you are on Elixir v0.8.1 and you have rebar installed (both available on homebrew)

2. Clone this repository and go to its directory

3. Get Dynamo dependencies and run tests with: MIX_ENV=test mix do deps.get, test

4. Create a project: mix dynamo path/to/your/project

Congratulations! You created your first project with Dynamo! Let's run it:

1. Go to your app

2. Get dependencies with: mix deps.get

3. Run it: mix server

Check lib/ and web/ folders for more information. Changes done in the web directory are picked up without a need to reload the server.

Static content is served from priv/static/ folder and from the /static route.

Your project can be compiled and used in production with: MIX_ENV=prod mix do compile, server.

This walk-through is going to put you in touch with the four main elements in Dynamo: routers, connection, OTP and the Dynamo itself. This walk-through requires you to be familiar with Elixir, so please read Elixir's getting started guide if you haven't yet.

A Dynamo is organized in routers. Routers are kept in the web/routers directory and by default a project contains an ApplicationRouter defined at web/routers/application_router.ex, which is your Dynamo main entry point:

defmodule ApplicationRouter do
  use Dynamo.Router

  prepare do
    # Pick which parts of the request you want to fetch
    # You can comment the line below if you don't need
    # any of them or move them to a forwarded router
    conn.fetch([:cookies, :params])
  end

  # It is common to break your Dynamo in many
  # routers forwarding the requests between them
  # forward "/posts", to: PostsRouter

  get "/" do
    conn = conn.assign(:title, "Welcome to Dynamo!")
    render conn, "index.html"
  end
end

All routers must use the Dynamo.Router module. By using this module, we have access to the macros get, post, put, patch and delete that allows developers to generate routes. Here is a simple route:

get "/hello/world" do
  conn.resp(200, "Hello world")
end

The conn value above represents a connection which we are going to get into more details soon. In the example above in particular, we are setting the connection to have response status 200 with the body "Hello world". Routes can also contain dynamic segments:

put "/users/:user_id" do
  conn.resp 200, "Got user id: #{conn.params[:user_id]}"
end

Each route is compiled down to a function clause, this makes routes matching extremely fast and also allow the use of guard constraints:

get "/section/:section" when section in ["contact", "about"] do
  render conn, "#{section}.html"
end

In general, Dynamos are broken into many routers which forwards the request in between them. Furthermore, routers also provide hooks, as the prepare hook seen above.

Hooks are a mechanism to encapsulate actions that are shared in between many routes. For example, one may write a hook that only allow requests with a given parameter to proceed:

prepare do
  unless conn.params[:user_info] do
    halt! conn.status(400)
  end
end

Another common functionality in hooks is to prepare the connection, sometimes by setting a layout:

prepare do
  conn.assign :layout, "home.html"
end

Or by fetching information used throughout all routes:

prepare do
  conn.fetch :params
end

There is also a finalize hook which is used after the request is processed.

Besides adding hooks to a router as whole, hooks can also be added per specific route using the @prepare and @finalize attributes before each route:

@prepare :authenticate_user
get "/users/:user_id" do
  # ...
end

defp authenticate_user(conn) do
  unless conn.session[:user_id] do
    halt! conn.status(401)
  end
end

As your Dynamo grows, it would be impractical to have all routes in a single router. For this reason, Dynamo allows you to easily compose and forward requests in between routers. Let's consider the router generated by default:

defmodule ApplicationRouter do
  use Dynamo.Router

  prepare do
    conn.fetch([:cookies, :params])
  end

  forward "/posts", to: PostsRouter

  get "/" do
    conn = conn.assign(:title, "Welcome to Dynamo!")
    render conn, "index.html"
  end
end

In the example above, all routes starting with /posts will be automatically routed to the PostsRouter. Furthermore, the PostsRouter will receive on the trailing part of the url. For instance, a request to /posts/recent, it will be seen by the PostsRouter as /recent:

defmodule PostsRouter do
  use Dynamo.Router

  get "/recent" do
    # Get all recent posts
  end
end

Finally, in the ApplicationRouter above, notice we fetch cookies and the request params for every request. However, if just the PostsRouter is using such information, the prepare hook could be moved to inside the PostsRouter, so it will be fetched only when it is needed.

This forwarding mechanism makes it very easy to split and organize your code by its logical sections while also having control on which features is required by each router.

The connection plays a huge part when building Dynamos. As we have seen above, each route have access to the connection as the variable conn. The authenticate_user prepare hook defined above also receives the connection as argument:

defp authenticate_user(conn) do
  unless conn.session[:user_id] do
    halt! conn.status(401)
  end
end

The connection is a direct interface to the underlying web server. In the examples above, we have used conn.resp(status, body) to set up our response. This will set a status and a response that will be sent back to the client after the route finishes processing.

However, a developer could also use conn.send(status, body), which will send the response immediately:

conn.send(200, "Hello world")

In the example above there is no wait and the response is sent immediately. The fact the connection is a direct interface makes streaming equally trivial:

conn = conn.send_chunked(200)
conn.chunk("Hello World")

There are no dirty hacks nor work-around required. You get a fast and clean API to interact with the web server.

One common confusion for developers starting with Dynamo (and to certain extent functional programming too) is immutability.

Elixir data structures are immutable, this means that, if you have a tuple, you can't modify this tuple in place. Adding or removing elements will actually return a new tuple:

first  = { 1, 2, 3 }
second = setelem(first, 0, :hello)

first  #=> { 1, 2, 3 }
second #=> { :hello, 2, 3 }

Notice how the elements of the first tuple remain intact. The same happens with the connection data structure. Consider this route:

get "/" do
  conn.assign(:title, "Welcome to Dynamo!")
  render conn, "index.html"
end

The first line conn.assign(:title, "Welcome to Dynamo!") has no effect whatsoever because we don't store the result of the operation anywhere. In fact, if someone writes IO.inspect conn.assigns[:title] in the next line, the value of the :title assign will still be nil. That said, as we do with tuples, we need to store the new connection in the updated variable:

get "/" do
  conn = conn.assign(:title, "Welcome to Dynamo!")
  render conn, "index.html"
end

Almost all functions through Dynamo will receive and return a connection. For example, the render function above will return a new connection with the render template as a body. Similarly, all routes and hooks must return the connection, you will get a gentle exception if you don't.

The connection data structure keeps the low-level API to interact with web servers. You can set status codes and response bodies, handle headers and parse request information as :params, :cookies, :session and so forth.

Furthermore, the connection is responsible to carry out the information in between routers and between routers and templates via assigns. For example:

get "/" do
  conn = conn.assign(:title, "Welcome to Dynamo!")
  render conn, "index.html"
end

For example, a prepare hook may also set the a current_user assign which could then be retrieved in any router as conn.assigns[:current_user]. You can find more information about assigns and other connection functions in Dynamo.Connection.

Finally, Dynamo also builds many functionalities on top of this low-level connection API:

• Dynamo.HTTP.Cookies - conveniences for working with cookies
• Dynamo.HTTP.Halt - conveniences for halting a connection, as the function halt! we saw in some examples
• Dynamo.HTTP.Redirect - conveniences for redirecting a connection
• Dynamo.HTTP.Render - conveniences for rendering templates
• Dynamo.HTTP.Session - conveniences for working with session

All those functions in Dynamo.HTTP.* are imported by default into your Dynamo.Router.

The next two sections will focus on the Dynamo integration with OTP, which is a set of tools and libraries for building robust, fault-tolerant applications. Talking about "applications", you may have noticed throughout this tutorial we haven't used the word "application" a lot so far. This is intentional as there is no such thing as a "Dynamo application"!

I will explain. When we first invoked mix dynamo path/to/your/project, it generated a project. This project may contain one or more OTP applications (by default, just one) and the Dynamo is always part of an OTP application.

In this section we are going to detail what it means to be an OTP application. Before continuing, make sure you have the guide about building OTP apps with Mix, because we will build on top of it.

To recap, when we invoke mix dynamo path/to/your/project, we have:

• a project - the project is everything that was generated and its backbone is the mix.exs file that contains the project dependencies, tasks to compile and run tests and much more;

• one or more applications - a project may contain one or more application, but most of the cases, it contains just one. In general, an application is an artifact generated by a project. For example, when you compile your code, you will have a bunch of .beam files and an .app file inside the ebin directory. The priv directory is also part of the application and it contains everything that is needed at runtime but is not source code, like assets, database configuration, etc. Your application also has dependencies and they are a subset of your project dependencies (in general, test and compile-time dependencies are not application dependencies);

• one or more dynamos - one application may contain one or more dynamos. Yes, you heard it right: Dynamo was designed from the ground up to be isolated, so you can have many Dynamos, listening to and serving different ports in the same project and running in the same OS process.

A project generated with mix dynamo contains the same elements as a basic OTP application. Assume we generate the same project as we did in building OTP apps with Mix but now using the dynamo command:

$ mix dynamo stacker

In the mix.exs file, we will see the project definition, containing the main application, the version and the project dependencies. We will also see a function named application, this function returns the application specification:

def application do
  [ applications: [:cowboy, :dynamo],
    mod: { Stacker, [] } ]
end

This application specifiation declares it depends on other two applications, in this case :cowboy (the web server) and :dynamo (this web framework). It also specifies an application callback. The application callback is the module Stacker and it will be initialized receiving an empty list as argument.

If we open up the lib/hello.ex file, we can see the application callback implementation:

defmodule Stacker do
  use Application.Behaviour

  @doc """
  The application callback used to start this
  application and its Dynamos.
  """
  def start(_type, _args) do
    Stacker.Dynamo.start_link([max_restarts: 5, max_seconds: 5])
  end
end

As explained in the building OTP apps with Mix guide, the application callback must return the PID of an OTP Supervisor. In this case, we are returning the PID of the Dynamo supervisor. In this case, if our Dynamo fails, the Erlang VM will act on it.

In case your application has other workers and supervisors, you can create your own supervisor tree and simply embed the Dynamo supervisor in your own tree. For example, imagine we copy both Stacker.Supervisor and Stacker.Server from the Mix guide, the supervisor looks like this:

defmodule Stacker.Supervisor do
  use Supervisor.Behaviour

  # A convenience to start the supervisor
  def start_link(stack) do
    :supervisor.start_link(__MODULE__, stack)
  end

  # The callback invoked when the supervisor starts
  def init(stack) do
    children = [ worker(Stacker.Server, [stack]) ]
    supervise children, strategy: :one_for_one
  end
end

Notice this supervisor has its own worker, called Stacker.Server, that should be part of the supervision tree. We can embed the Dynamo supervisor in that tree by changing the init function:

def init(stack) do
  children = [
    worker(Stacker.Server, [stack]),
    supervisor(Stacker.Dynamo, [])
  ]
  supervise children, strategy: :one_for_one
end

Now, all we need to do is to change the Stacker module to initialize the new supervisor tree instead of the Dynamo directly:

defmodule Stacker do
  use Application.Behaviour

  def start(_type, _args) do
    Stacker.Supervisor.start_link([])
  end
end

This shows that your application is the one in control and a Dynamo is simply embeded into it.

So what is a Dynamo after all? A Dynamo is nothing more than a module which can be found at lib/*/dynamo.ex of a newly generated project. Here is an example:

defmodule Stacker.Dynamo do
  use Dynamo

  config :dynamo,
    env: Mix.env,
    otp_app: :stacker,
    endpoint: ApplicationRouter,
    static_route: "/static"
end

When you run mix compile, this Dynamo is compiled and becomes functional when you start your application supervision tree, as we just discussed. The Dynamo is responsible to handle requests, compile and manage the files under the web directory and each Dynamo also has its own supervisor tree.

Finally, notice that in the same directory you find your Dynamo, you can also find an environments directory which contains configuration specific for each environment. Take a look at them for examples on the available configuration options.

Summing up, when you run mix dynamo stacker to create a project and then fetch its dependencies, this is the final structure you get:

- deps                        # Your project dependencies
- lib                         # Contains your application files
  - lib/stacker.ex            # Your application callback
  - lib/stacker/dynamo.ex     # The Dynamo definition
  + lib/stacker/environments  # Dynamo environment related configuration
- mix.exs                     # Your project specification
- priv                        # Application files that are needed on runtime
  + priv/static               # Static assets (they are needed on runtime!)
- test                        # Your test files
  + test/features             # End-to-end tests
  + test/routers              # Routers unit tests
- web                         # Files managed and compiled by the Dynamo
  + web/routers               # Your application routers

That's all. If you haven't built an OTP application before, you may be a bit overwhelmed but there is nothing stop you from diving into the web directory and learning about OTP just when you need it. Before you see it, you be leveraging the power of the Erlang VM to build robust, high performance and concurrent web applications!

In the guides directory, we contain a bunch of small, simple guides that teach you how to achieve something in Dynamo. We have the following guides:

• How to create single file Dynamos, although not recommended for production, the examples directory usually use single file Dynamos to easily show how to achieve something;

• How to render templates with layouts

• How to import functions (helpers) into templates

• How to use SSL

Dynamo source code is released under Apache 2 License. Check LICENSE file for more information.