elixir-plug / plug Goto Github PK

refs: phoenixframework/phoenix#281

Working on this now

@josevalim, can we have new stuff - esp. excellent Plug.Conn.halt/1 - rolled out to Hex, please?

From josevalim here phoenixframework/phoenix#338:

Two more notes:

It would be nice if we have a mechanism to skip the CSRF token if something is set inside the conn.private. For that, we could check if conn.private[:plug_skip_csrf_protection] is set to true and skip it. This would be useful for testing.

We should also add protection to this style of cross requests: rails/rails@1650bb3

Feel free to ping me if you need more info!

Currently WIP here: #136

I have a Router plug

defmodule Rest do
  use Plug.Router
  import Plug.Conn

  plug :match
  plug :dispatch

  get "/hello" do
    send_resp(conn, 200, "Hello, world!")
  end

  match _ do
    send_resp(conn, 404, "oops")
  end

  def start do
    Plug.Adapters.Cowboy.http Rest, [], port: 80
  end

  def stop do
    Plug.Adapters.Cowboy.shutdown Rest.HTTP
  end
end

However, when calling Rest.start I get

{:error,
 {{:shutdown,
   {:failed_to_start_child, :ranch_acceptors_sup,
    {{:badmatch, {:error, :eacces}},
     [{:ranch_acceptors_sup, :init, 1,
       [file: 'src/ranch_acceptors_sup.erl', line: 30]},
      {:supervisor, :init, 1, [file: 'supervisor.erl', line: 243]},
      {:gen_server, :init_it, 6, [file: 'gen_server.erl', line: 306]},
      {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 239]}]}}},
  {:child, :undefined, {:ranch_listener_sup, Rest.HTTP},
   {:ranch_listener_sup, :start_link,
    [Rest.HTTP, 100, :ranch_tcp, [port: 200], :cowboy_protocol,
     [env: [dispatch: [{:_, [],
         [{:_, [], Plug.Adapters.Cowboy.Handler, {Rest, []}}]}]],
      compress: false]]}, :permanent, :infinity, :supervisor,
   [:ranch_listener_sup]}}}

If I remove the port: 80, there is no problem calling Rest.start, and the server listens on port 4000.

I am using Elixir v0.15.0.

The main reason is consistency. We can do conn.assigns, conn.private, conn.cookies but we can't do conn.session.

On the other hand, session is not really a thing that would matter to all requests, so we need to balance how first class it should or should not be.

/cc @ericmj @chrismccord

Hi,
I get an error when I try to build the Plug dependency with mix.
I beleive that this issue is not fully resolved, because nested directories are still spawned. Here's a complete, minimal, buggy example:

  def application do
    [ applications: [:cowboy, :plug],
      mod: {MyApp, []} ]
  end
  defp deps do
    [ {:cowboy, github: "extend/cowboy"},
      {:plug, "~> 0.4.2"} ]
  end

'mix deps.get' seems to go through fine

* Getting cowboy (git://github.com/extend/cowboy.git)
Cloning into '/path/to/project/MyApp/deps/cowboy'...
remote: Reusing existing pack: 6046, done.
remote: Total 6046 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (6046/6046), 3.64 MiB | 2.39 MiB/s, done.
Resolving deltas: 100% (3549/3549), done.
* Getting cowlib (git://github.com/extend/cowlib.git)
Cloning into '/path/to/project/MyApp/deps/cowlib'...
remote: Reusing existing pack: 170, done.
remote: Total 170 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (170/170), 77.61 KiB, done.
Resolving deltas: 100% (98/98), done.
* Getting ranch (git://github.com/extend/ranch.git)
Cloning into '/path/to/project/MyApp/deps/ranch'...
remote: Reusing existing pack: 803, done.
remote: Total 803 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (803/803), 255.52 KiB, done.
Resolving deltas: 100% (464/464), done.
Running dependency resolution
Unlocked:   plug
Dependency resolution completed successfully
  plug: v0.4.2
* Getting plug (package)
Fetching package (http://s3.hex.pm/tarballs/plug-0.4.2.tar)
Unpacked package tarball (/home/user/.mix/.package-cache/plug-0.4.2.tar)

But compilation fails at conn.ex.

* Compiling ranch
==> ranch (compile)
Compiled src/ranch_protocol.erl
Compiled src/ranch_transport.erl
Compiled src/ranch_acceptors_sup.erl
Compiled src/ranch_server.erl
Compiled src/ranch_conns_sup.erl
Compiled src/ranch_listener_sup.erl
Compiled src/ranch_app.erl
Compiled src/ranch_acceptor.erl
Compiled src/ranch_tcp.erl
Compiled src/ranch.erl
Compiled src/ranch_sup.erl
Compiled src/ranch_ssl.erl
* Compiling cowlib
==> cowlib (compile)
Compiled src/cow_http_hd.erl
Compiled src/cow_spdy.erl
Compiled src/cow_mimetypes.erl
Compiled src/cow_date.erl
Compiled src/cow_cookie.erl
Compiled src/cow_http.erl
Compiled src/cow_qs.erl
Compiled src/cow_http_te.erl
Compiled src/cow_multipart.erl
* Compiling cowboy
==> cowboy (compile)
Compiled src/cowboy_sub_protocol.erl
Compiled src/cowboy_middleware.erl
Compiled src/cowboy_http_handler.erl
Compiled src/cowboy_websocket_handler.erl
Compiled src/cowboy_loop_handler.erl
Compiled src/cowboy_clock.erl
Compiled src/cowboy_handler.erl
Compiled src/cowboy_spdy.erl
Compiled src/cowboy_static.erl
Compiled src/cowboy.erl
Compiled src/cowboy_protocol.erl
Compiled src/cowboy_router.erl
Compiled src/cowboy_app.erl
Compiled src/cowboy_rest.erl
Compiled src/cowboy_websocket.erl
Compiled src/cowboy_sup.erl
Compiled src/cowboy_bstr.erl
Compiled src/cowboy_req.erl
Compiled src/cowboy_http.erl
* Compiling plug
Compiled lib/plug.ex
Compiled lib/plug/adapters/cowboy.ex
Compiled lib/plug/builder.ex
Compiled lib/plug/conn/query.ex
Compiled lib/plug/conn/adapter.ex
Compiled lib/plug/conn/cookies.ex
Compiled lib/plug/conn/unfetched.ex
Compiled lib/plug/head.ex
Compiled lib/plug/method_override.ex
lib/plug/mime.ex:51: lc is deprecated, please use for comprehensions instead
lib/plug/mime.ex:52: lc is deprecated, please use for comprehensions instead
lib/plug/mime.ex:76: lc is deprecated, please use for comprehensions instead
Compiled lib/plug/conn/utils.ex
Compiled lib/plug/exceptions.ex
== Compilation error on file lib/plug/adapters/cowboy/conn.ex ==
could not compile dependency plug, mix compile failed. You can recompile this dependency with `mix deps.compile plug` or update it with `mix deps.update plug`
** (ArgumentError) cannot access module File.Stat because it is not a record
    (elixir) expanding macro: Kernel.access/2
    lib/plug/adapters/cowboy/conn.ex:33: Plug.Adapters.Cowboy.Conn.send_file/4
    (elixir) src/elixir.erl:157: :elixir.erl_eval/2
    (elixir) src/elixir.erl:150: :elixir.eval_forms/4
    (elixir) src/elixir_lexical.erl:17: :elixir_lexical.run/2
    (elixir) lib/kernel/parallel_compiler.ex:91: anonymous fn/3 in Kernel.ParallelCompiler.spawn_compilers/8

I use a freshly compiled version of otp_src_17.0 (no java compiler and wxwidget, but it does not seem to be necessary here) as well as a recent version of elixir that passes all tests.

Today, there is logic in Phoenix to automatically send the response if nobody sent it yet but I feel we could push the responsibility to Plug adapters. The question is what to do if nothing was set.

/cc @chrismccord

This issue discusses how plug stacks will be formed.

Plugs

There are two kind of plugs: function plugs and module plugs. A function plug is any function that receives a connection and a set of options and returns a connection. Its type signature is:

@spec function(Plug.Conn.t, term) :: Plug.Conn.t

A module plug is an extension of the function plug. It must export a call/2 function, with the signature defined above, but it also must provide a init/1 function, for initialization of the options:

@type arg :: tuple | atom | integer | float | [arg]
@spec init(arg) :: arg
@spec call(Plug.Conn.t, arg) :: Plug.Conn.t

The result returned by init/1 is the one given to call/2. Note init/2 may be called during compilation and as such it must not return pids, ports or values that are not meant to live across Operating System processes.

Wrappers

A wrapper is a module that exports two functions: init/1 and wrap/3:

@spec init(arg) :: arg
@spec wrap(Plug.Conn.t, arg, (Plug.Conn.t -> Plug.Conn.t)) :: Plug.Conn.t

wrap/3 is similar to call/2 except it receives a third argument as a function containing the remaining of the plug stack.

The builder

Plug will ship with a builder that provides the following DSL:

defmodule MyPlug do
  use Plug.Builder

  plug Plug.Parsers, parsers: [:urlencoded]
  plug Plug.Session, storage :ets
  plug :my_function

  def my_function(conn, _) do
    conn.send_body(200, "hello world")
  end
end

The plug/2 macro accepts the an atom as argument, which means a function plug, or a module, which means either a (module) plug or a wrapper. The given module must be available at compilation time and export one of the plug or the wrapper APIs.

Design constraints

There are two major design constraints in Plugs. Most abstractions like Plug, like Rack and Ring, provide the concept of middleware (middle of the stack) and application/handler (endpoints). Those usually have pre-defined positions in the stack. In order to use an application in a stack, you need to first wrap it in a middleware.

This requires new abstractions to be added further down the stack. For example, most Rack web frameworks end up adding their own idea of filters/callbacks later in the application, because writing middleware for controllers/applications is too verbose.

Plug disappears with the concept of applications in favor of just plugs. Plugs may exist in two flavors, as a simple function, or as a module. Besides, plugs also provide wrappers, which can be built in the same stack as plugs themselves.

The second constraint comes from how the Erlang VM (and therefore Elixir) handles anonymous functions on code reloading. Ring has a beautiful abstraction where a middleware is just about high-order functions:

(defn wrap-content-type [handler content-type]
  (fn [request]
    (let [response (handler request)]
      (assoc-in response [:headers "Content-Type"] content-type))))

On boot, the stack boils down to anonymous functions invoking anonymous functions which is than handled to the web server handler. However, keep in mind that anonymous functions in Elixir does not survive code reloading! If someone reloads the code of the middleware above, after 2 reloads, the web server would hold a reference to an anonymous function that no longer exists and be killed.

That's the reason why Plug explicitly favors the module approach with callback functions instead of relying on high-order functions.

We believe this proposal achieves Plug's goals of working well with the underlying VM constraints and being composable, by allowing stacks where both wrappers and plugs can be mixed.

/cc @devinus @chrismccord @ericmj @batate

Add peer field to connection record to retrieve IP address of peer.

I am new to Elixir and Phoenix and trying to experiment with running a website on Heroku. I have the site up and running there but I would like to eliminate the storage of my Cookie store session secret in my Git repository for security reasons. I would instead like to do something like:

// config/prod.ex
...
config :plugs, cookies: true
# Generate secret easily with 'openssl rand -hex 64'
# Set with heroku config:set SESSION_SECRET=foo
config :cookies, key: "_my_site", secret: System.get_env("SESSION_SECRET")
...

However, taking this approach is seemingly not possible since there is an explicit check at compile time that the cookie is larger than 64 bytes which raises an exception if it is not.

See:
https://github.com/elixir-lang/plug/blob/master/lib/plug/session/cookie.ex#L32

I would also like to be able to make the repository for my web app public without exposing production secrets. Can you tell me if there is a way around this, and if not please consider allowing this to be lazily loaded from the environment.

Thanks.

This is a first draft and I intend to change and alter it from feedback

I've been doing a little research into making websockets a part of plug, and having built the websocket support for phoenix I figured I'd do an RFC for Plug.Websocket.

Why?

• Plug supports every other major form of HTTP/Browser style specs outside of SPDY and Websockets.
• Websockets are supported on every current browser and many former versions http://caniuse.com/websockets.
• They are a great Usecase for Elixir.

How?
Because Plug has goals of being multi-backend we need to be cognizant of how possible backends might work. Currently we only support cowboy, but I assume we have plans to support:

• Mochiweb
• Yaws
• Webmachine
• inets
• elli

Unfortunately only a subset of these natively support websockets, but all is not lost! For those that DONT support it out of the box we can implement connection hijacking as shown here by simple_bridge. I will be references simple_bridge often as they tackled the concept of supporting many of these backends with websockets.

Is there a common interface?

• Mochiweb: doesn't support would require highjacking, meaning we can define our own interface.
• Yaws: does support, check for a header and upgrade the connection, defing a custom handler module.

Interface

init([_Arg, State={Handler, Bridge}]) ->
handle_message(Data, State) ->j
handle_info(Data, State) ->
terminate(Reason, State) ->

• Webmachine: doesn't support, would require highjacking.
• Inets: doesn't support, would require highjacking.
• elli: does support with third party lib, which implements highjacking.

Interface

websocket_init(Req, Opts) ->
websocket_handle(_Req, {text, Data}, State) ->
websocket_handle(_Req, {binary, Data}, State) ->
websocket_handle(_Req, _Frame, State) ->
websocket_info(Req, Message, State) ->
websocket_handle_event(Name, EventArgs, State) -> #open, close etc

• Cowboy: does support, return upgrade to request and current module handles rest.

Interface

websocket_init(_Transport, Req, _Opts) ->
websocket_handle(Data, Req, State) ->
websocket_info(Data, Req, State) ->
websocket_terminate(Reason, _Req, State) ->

Proposed Plug Interface
Plug.Conn.upgrade_to_websocket(conn, HandlerModule) this function will call the adaper.upgrade_to_websocket which has one responsibility to verify and upgrade a connection. I see this working two ways: for built in functions it would attach a value to conn.assigns and match on it in init or it would attempt to hijack the request similar to how simple_bridge and elli do. It would be up to the adapter to make the decision.

The HandlerModule would be a module that implements the behavior based on the common interfaces from above.

#initalizing the socket. 
defcallback init(request, options)
#handling client data
defcallback data(data, req, state)
#handling messages from erlang process
defcallback info(data, req, state)
#handling terminations from users or erlang
defcallback terminate(reason, req state)

There would be another plug module called Plug.WebSocket which defines reply, terminate, hibernate and info.

In each case the adapter would call the HandlerModule function when appropriate. Plug.WebSocket also calls the adapter module/s definitions to help build the proper response for reply/hibernate/info/terminate.

With minimal changes to the current plug adapter interface, and a new method on Plug.Conn I believe we can have Websocket support.

The first step is adding cowboy support since thats the only backend we currently support. If we can decide on a common interface I could do it.

Hi,

I'm trying to get phoenix/plug to be able to properly decrypt rails session cookies. The differences in the defaults are:
rails: sha1 digest, 64 byte keys, 1000 iterations
plug: sha256 digest, 32 byte keys, 1000 iterations

Rails allow customization of the digest fun by setting the "action_dispatch.cookies_digest" config, but not the length or the iterations. A quick solution (and breaking change) would be to default the key length to 64 in phoenix. The other option would be to expose the 3 options in the session settings?

What would be your preference?

Thanks,
Chris

Hi,

docs published at http://elixir-lang.org/docs/plug/ happen to be lagging couple of versions - they're 0.5.1 now..

I did a bit of work on getting plug to correctly decrypt the rails session cookies. Here are the differences between plug and rails:

• MessageEncryptor fails if salt size if higher than 32 bytes. Ruby/OpenSSL trims the key to 32 bytes.
• MessageEncryptor uses a custom padding scheme, while rails/openssl uses PKCS7 padding.
• MessageVerifier uses base64 encoding for digest, while rails uses base16.
• Plug does not encode the cookie name or value, however rack encodes both name and value https://github.com/rack/rack/blob/master/lib/rack/utils.rb#L303

All but first are breaking changes. Rails compatibility is probably not a goal for plug, and it shouldn't be, but it would definitely make the migration path easier to be able to interoperate with rails.

Do you think it makes sense to make these changes in plug?

I have made the changes on a fork, and validated that I can decode/encode rails session cookies:
https://github.com/cconstantin/plug/commits/rails-compat

Cheers

Today if you call collect_body multiple times, you will get the body the first time and an empty body in the following ones. It would be nice if collect_body/3 instead returned {:error, :already_collected, conn} instead. In order for this to work, we will probably need to store this data or in the connect or in the adapter.

In the adapter is preferable although I am not sure if possible. Does cowboy gives us this information in the new API?

@scrogson, could you please take a look at this? :D After we get this in, we can release v0.5.0 with collect_body/3.

It would be nice to able to hibernate plug processes and re-enter the plug stack created by Plug.Builder.compile after hibernation at the current position. This is required to use all features of cowboy sub_protocols, such as cowboy_websocket, without delegating some dispatch duties to cowboy.

I've been playing around with Plug.Session on and off now for about a week, attempting to get it working. However, I seem to be running into a series of issues.

I started out trying to set up an ETS store like this:

defmodule Plugger.Router do
  import Plug.Conn
  use Plug.Router

  plug :match
  plug :dispatch

  plug Plug.Session, store: :ets, key: "_my_app_session", secure: true, table: :session

  get "/" do
    conn = fetch_session(conn)
    conn = put_session(conn, :foo, "bar")
    foo = get_session(conn, :foo)
    send_resp(conn, 200, foo)
  end

  match _ do
    send_resp(conn, 404, "oops")
  end
end

When I run this I get the message "cannot fetch session without a configured session plug". So I add the following:

get "/" do
    opts = Plug.Session.init(store: :ets, key: "_my_app_session", secure: true, table: :session)
    conn = Plug.Session.call(conn, opts)
    conn = fetch_session(conn)
    conn = put_session(conn, :foo, "bar")
    foo = get_session(conn, :foo)
    send_resp(conn, 200, foo)
  end

but that doesn't appear to work either because I receive the following error:

{stacktrace,[{ets,insert_new,[session,{<<"NYlHoyEdRsMceu9jDiEALhZhiBggTTFQ7IU21lB9mZsLcG2mNwOCOAkTic4/6QcT8tkveH3KyMAT0ZMR5OYHRlfDYEJ5M9oeAyLPmO7GL0GHdE3UQZnakRpU8bFjxYFq">>,#{foo => <<"bar">>}}],[]}

Now, this may fall back to Erlang but it if you've seen it and know what's happening , please let me know.

I decided to move on to cookie storage based on the examples:

defmodule Plugger.Router do
  import Plug.Conn
  use Plug.Router

  plug :match
  plug :dispatch

  @valid_secret String.duplicate("abcdef0123456789", 8)

  plug Plug.Session, store: :cookie, key: "_plugger", secret: @valid_secret

  get "/" do
    conn = fetch_session(conn)
    conn = put_session(conn, :foo, "baz")

    foo = get_session(conn, :foo)

    send_resp(conn, 200, foo)
  end

  match _ do
    send_resp(conn, 404, "oops")
  end
end

Once again, I got the message "cannot fetch session without a configured session plug". So I add the following:

 get "/" do
    opts = Plug.Session.init(store: :cookie, key: "_plugger", secret: @valid_secret)
    conn = Plug.Session.call(conn, opts)
    conn = fetch_session(conn)
    conn = put_session(conn, :foo, "baz")

    foo = get_session(conn, :foo)

    send_resp(conn, 200, foo)
  end

And then everything appears to work but it can't be right.

The plug macro appears to be working for other modules so it's hard to say there is an issue there but it seems that there may be. Before I dive to much deeper, I'd like to run it past everyone to see if I'm doing something obviously wrong.

On Elixir 0.13.1, an empty mix project with the following mix.exs will not compile:

defmodule Foobar.Mixfile do
  use Mix.Project

  def project do
    [app: :foobar,
      version: "0.0.1",
      elixir: "~> 0.13.1",
      deps: deps]
  end

  # Configuration for the OTP application
  #
  # Type `mix help compile.app` for more information
  def application do
    [applications: [:cowboy, :plug],
      mod: {Foobar, []}]
  end

  # Dependencies can be hex.pm packages:
  #
  # {:mydep, "~> 0.3.0"}
  #
  # Or git/path repositories:
  #
  # {:mydep, git: "https://github.com/elixir-lang/mydep.git", tag: "0.1"}
  #
  # Type `mix help deps` for more examples and options
  defp deps do
    [
      {:cowboy, github: "extend/ranch"},
      {:plug, "~> 0.4.2"},
      ]
  end
end

Here is the output I get:

bu2b:foobar thmzlt$ mix deps.clean --all
* Cleaning cowboy
* Cleaning plug
bu2b:foobar thmzlt$ mix deps.get
* Getting cowboy (git://github.com/extend/ranch.git)
Cloning into '/Users/thmzlt/Code/foobar/deps/cowboy'...
remote: Reusing existing pack: 803, done.
remote: Total 803 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (803/803), 255.52 KiB | 351.00 KiB/s, done.
Resolving deltas: 100% (464/464), done.
Checking connectivity... done.
Running dependency resolution
Unlocked:   plug
Dependency resolution completed successfully
  plug: v0.4.2
* Getting plug (package)
Fetching package (http://s3.hex.pm/tarballs/plug-0.4.2.tar)
Unpacked package tarball (/Users/thmzlt/.mix/.package-cache/plug-0.4.2.tar)
bu2b:foobar thmzlt$ mix deps.compile
* Compiling plug
Compiled lib/plug.ex
== Compilation error on file lib/plug/conn/adapter.ex ==
could not compile dependency plug, mix compile failed. You can recompile this dependency with `mix deps.compile plug` or update it with `mix deps.update plug`
** (RuntimeError) cannot compile Plug because the :cowboy application is not available
    lib/plug/conn/adapter.ex:4: (file)
    (elixir) src/elixir_lexical.erl:17: :elixir_lexical.run/2
    (elixir) lib/kernel/parallel_compiler.ex:91: anonymous fn/3 in Kernel.ParallelCompiler.spawn_compilers/8

bu2b:foobar thmzlt$ mix deps.compile cowboy
* Compiling cowboy
 ERLC   ranch.erl ranch_acceptor.erl ranch_acceptors_sup.erl ranch_app.erl ranch_conns_sup.erl ranch_listener_sup.erl ranch_protocol.erl ranch_server.erl ranch_ssl.erl ranch_sup.erl ranch_tcp.erl ranch_transport.erl
 APP    ranch.app.src
bu2b:foobar thmzlt$ mix deps.compile plug
* Compiling plug
Compiled lib/plug.ex
Compiled lib/plug/adapters/cowboy.ex
== Compilation error on file lib/plug/conn/adapter.ex ==
could not compile dependency plug, mix compile failed. You can recompile this dependency with `mix deps.compile plug` or update it with `mix deps.update plug`
** (RuntimeError) cannot compile Plug because the :cowboy application is not available
    lib/plug/conn/adapter.ex:4: (file)
    (elixir) src/elixir_lexical.erl:17: :elixir_lexical.run/2
    (elixir) lib/kernel/parallel_compiler.ex:91: anonymous fn/3 in Kernel.ParallelCompiler.spawn_compilers/8

Also tried Elixir and Plug from master running with both OS X and Ubuntu 14.04. Same issue.

Getting this when installing dependencies, here is my mix.exs

defmodule Koncur.Mixfile do
  use Mix.Project

  def project do
    [ app: :koncur,
      version: "0.0.1",
      elixir: "~> 1.0.0-rc1",
      elixirc_paths: ["lib", "web"],
      deps: deps ]
  end

  # Configuration for the OTP application
  def application do
    [
      mod: { Koncur, [] },
      applications: [:phoenix, :cowboy, :logger, :postgrex, :ecto]
    ]
  end

  # Returns the list of dependencies in the format:
  # { :foobar, git: "https://github.com/elixir-lang/foobar.git", tag: "0.1" }
  #
  # To specify particular versions, regardless of the tag, do:
  # { :barbat, "~> 0.1", github: "elixir-lang/barbat" }
  defp deps do
    [
      {:phoenix, github: "phoenixframework/phoenix"},
      {:postgrex, ">= 0.0.0"},
      {:ecto, "~> 0.2.0"},
      {:cowboy, "~> 1.0.0"},
      {:oauth2ex, github: "parroty/oauth2ex"}
    ]
  end
end

Full backtrace

* Updating oauth2ex (git://github.com/parroty/oauth2ex.git)
From git://github.com/parroty/oauth2ex
 + 56683ab...060ef38 master     -> origin/master  (forced update)
Running dependency resolution
Unlocked:   cowboy, plug
Dependency resolution completed successfully
  plug: v0.6.0
  cowboy: v1.0.0
  cowlib: v1.0.0
  ranch: v1.0.0
* Updating plug (package)
Checking package (http://s3.hex.pm/tarballs/plug-0.6.0.tar)
Using locally cached package
Unpacked package tarball (/Users/justin/.hex/packages/plug-0.6.0.tar)
==> plug
warning: the dependency plug requires Elixir "~> 0.15.0" but you are running on v1.0.0-rc1

== Compilation error on file lib/plug/conn/adapter.ex ==
** (RuntimeError) cannot compile Plug because the :cowboy application is not available. Please ensure it is listed as a dependency before the plug one.
    lib/plug/conn/adapter.ex:4: (file)
    (elixir) src/elixir_lexical.erl:17: :elixir_lexical.run/3
    (elixir) lib/kernel/parallel_compiler.ex:97: anonymous fn/4 in Kernel.ParallelCompiler.spawn_compilers/8

could not compile dependency plug, mix compile failed. You can recompile this dependency with `mix deps.compile plug` or update it with `mix deps.update plug`

Trying to use plug (0.5.3) using cowboy (1.0.0) with Elixir 0.15.0 and did hello world in the doc but getting this error:

Running with Cowboy on http://localhost:4000
== Compilation error on file lib/plug_try.ex ==

** (UndefinedFunctionError) undefined function: PlugTry.init/1
PlugTry.init([])
lib/plug/adapters/cowboy.ex:158: Plug.Adapters.Cowboy.dispatch_for/2
lib/plug/adapters/cowboy.ex:36: Plug.Adapters.Cowboy.args/4
lib/plug/adapters/cowboy.ex:125: Plug.Adapters.Cowboy.run/4
(stdlib) erl_eval.erl:657: :erl_eval.do_apply/6

It would be nice to allow users to choose an encode/decode option for session cookies.

I imagine a Plug.Session.Coder behaviour with a simple API of encode and decode. The coder can just be specified in the plug opts. Something like:

plug Plug.Session, store: :cookie, key: "key", secret: "secret", coder: :json

Then users can create a coder

defmodule Plug.Coder.JSON do
  use Jazz

  def encode(data) do
    JSON.encode!(data)
  end

  def decode(data) do
    JSON.decode!(data)
  end
end

The default coder (Plug.Coder.ELIXIR?) could just use :erlang.term_to_binary and :erlang.binary_to_term

Thoughts?

/?array[][foo]=foo1&array[][foo]=foo2&array[][bar]=bar2
result of rails parser is
{"array"=>[{"foo"=>"foo1"}, {"foo"=>"foo2", "bar"=>"bar2"}]}
result of plug parser is
%{"array"=>[%{"foo"=>"foo1"}, %{"foo"=>"foo2"}, %{"bar"=>"bar2"}]}

Is this a bug?

I'm trying to polish the documentation for Plug.Builder. Throughout all the source code, guards are mentioned: there's even a compile_guard/2 function defined roughly as:

defp compile_guard(call, true), do: call
defp compile_guard(call, guard), do: # do 'call' if 'guard' evaluates to true

However, as far as I understand, the addition of new plugs follows this logic:

1. use the plug/2 macro to store a new plug
2. this macro adds a tuple like {plug_name, plug_opts, true} to the @plugs module attribute
3. this tuple gets passed around to initialise the options, call the plugs and so on, but the third element of the tuple (which should be the guard) always remains untouched, hence true

First, did I get everything right? Second, is this code there in order to allow for future changes? What made me wonder is that the plug/2 macro only accepts two arguments and no guards.

If the code can be removed, I'd be super happy to take care of that in the general polish I'm trying to give to Plug.Builder. :) Thanks!

Any reason not having the request body easily accessible in the Plug.Conn struct? The only pain point with this I can think of at the moment would be in the case of streaming requests, but surely an elegant solution for this could be developed.

Hi, how can access cowboy onrequest and onresponse.
I need it to add some monitoring to my Plug based application.

thanks,
Ariel

When switching between 0.5.3 and 0.6.0, there's deps issue:

% mix deps.get 
* Updating plug (package)
Checking package (http://s3.hex.pm/tarballs/plug-0.6.0.tar)
Using locally cached package
Unpacked package tarball (/home/wk/.hex/packages/plug-0.6.0.tar)
% mix          
==> plug
Compiled lib/plug.ex
Compiled lib/plug/adapters/cowboy.ex
Compiled lib/plug/builder.ex

== Compilation error on file lib/plug/conn/adapter.ex ==
** (RuntimeError) cannot compile Plug because the :cowboy application is not available. Please ensure it is listed as a dependency before the plug one.
    lib/plug/conn/adapter.ex:4: (file)
    (elixir) src/elixir_lexical.erl:17: :elixir_lexical.run/3
    (elixir) lib/kernel/parallel_compiler.ex:93: anonymous fn/3 in Kernel.ParallelCompiler.spawn_compilers/8

could not compile dependency plug, mix compile failed. You can recompile this dependency with `mix deps.compile plug` or update it with `mix deps.update plug`
% mix deps                                      1
* ranch 1.0.0 (package)
  locked at 1.0.0
  ok
* reprise 0.2.6 (package)
  locked at 0.2.6
  ok
* cowlib 1.0.0 (package)
  locked at 1.0.0
  ok
* cowboy 1.0.0 (package)
  locked at 1.0.0
  ok
* plug (package)
  locked at 0.6.0
  the dependency build is outdated, please run `mix deps.compile`
% mix deps.compile plug
==> plug
Compiled lib/plug.ex
Compiled lib/plug/adapters/cowboy.ex
Compiled lib/plug/builder.ex

== Compilation error on file lib/plug/conn/adapter.ex ==
** (RuntimeError) cannot compile Plug because the :cowboy application is not available. Please ensure it is listed as a dependency before the plug one.
    lib/plug/conn/adapter.ex:4: (file)
    (elixir) src/elixir_lexical.erl:17: :elixir_lexical.run/3
    (elixir) lib/kernel/parallel_compiler.ex:93: anonymous fn/3 in Kernel.ParallelCompiler.spawn_compilers/8

could not compile dependency plug, mix compile failed. You can recompile this dependency with `mix deps.compile plug` or update it with `mix deps.update plug`

Now this is remedied by full deps clean, but such things become an issue on Heroku, where you can't easily run mix deps.clean --all :\

Is there some potential how it could be improved?

Reproduced with several trivial and non-trivial Plug-based apps.

Hello @josevalim,

Would be good to have following cowboy wrappers in plug:

• :cowboy_req.path(req)/1
• :cowboy_req.body/1
• ::cowboy_req.cookie/2
• :cowboy_req.cookies/1

If there are already in plug, sorry, and let me know please where to find it.

Possibly accepting an array of string secrets or an :old_secret option for graceful degradation of cookie secrets.

EX. http://rubydoc.info/github/rack/rack/master/Rack/Session/Cookie

I have a basic, naive code reloading Plug that simply runs mix compile in process. This seems to work reasonably well for the "save/refresh" cycle of development. The only outstanding issue I can see is restarting the application in addition to compilation, but some things may just require a standard restart. Thoughts?

POC implementation:
https://github.com/phoenixframework/phoenix/blob/master/lib/phoenix/plugs/code_reloader.ex

Hey there, great work with Plug! I'm really liking it.

I have one question: I have the need to match routes based on the subdomain (e.g., an API hosted at api.domain.com which I want to match on the api. subdomain). Is there a clean way to do that using Plug?

As of now, I'm using a plain if Regex.match(~r/^api\./, conn.host) but I feel it's a hack since I should have to do it in every api.-prefixed route.

I think something along the lines of forward route, to: Plug would be cool but I'm not sure if there's an existing way which includes matching on the host. Phoenix does something like this using the :host option with the scope macro.

Thanks in advance ❤️

I was wondering if we should make a similar change to the hello world example as we did to the router one. In particular, it presently kicks things off with:

Plug.Adapters.Cowboy.http MyPlug, []

But I had the impression from our other discussion that it might be better to do:

Plug.Adapters.Cowboy.http MyPlug, MyPlug.init([])

Would that be better or is it not needed in this case? Certainly happy to make this tiny pull if it makes sense. Thanks!

Currently adapters provide the raw, clear-text request headers to Plug. [1] I posit that instead, adapters should provide already parsed request headers to Plug.

To do so would require some a minor implementation changes to Plug that break backwards compatibility:

1. Plug.Conn.req_headers would be filled with :cowboy_req.parse_headers/{2.3}, with a fallback implementation provided by Plug for unparsed headers or other adapters that do not provide similar functionality. See RFC 2616 § 14 for a list of well-defined headers.
2. Plug's matcher for Plug.Parsers and alike would have to be updated to understand the new format of the Accept field (which includes weighting!)

That would result in more standards complaint server behaviour, and in my opinion a better API. Furthermore, it allows better introspection, e.g. failed language matches (any of the requested languages are not returned) could be noted more easily.

I can submit a pull-request implementing the necessary changes described.

Hi all,

I'd like to discuss a plug for dealing with session data with an ability to have a variety of adapters (stores) for storing the session data. I have already mocked up a prototype that takes notes from the Dynamo session filter, but it was suggested by @josevalim (with whom I agree) that I open an issue for discussing an implementation for this project.

Proposed adapter behaviour

Below is a proposed behaviour for a session adapter.

defmodule Plug.Session.Adapter do
  use Behaviour

  @type opts   :: Keyword.t
  @type sid    :: binary
  @type data   :: iodata
  @type reason :: String.t

  defcallback init(opts) :: opts
  defcallback get(sid) :: {:ok, data} | {:error, reason}
  defcallback put(sid, data) :: :ok | {:error, reason}
end

Reason for not passing a conn is that I believe a storage adapter should only be concerned with accessing and mutating data for a session. Is this a valid thought? Should we keep the adapter from dealing with the conn directly?

Proposed plug for using adapters

Below is a propsed wrapper plug.

defmodule Plug.Session do
  import Plug.Connection

  @behaviour Plug.Wrapper

  def init(opts) do
    # Require a session adapter
    # Require a name for the session id cookie / adapter identifier

    adapter = opts[:adapter]
    adapter.init(opts)
  end

  def wrap(conn, opts, fun) do
    adapter = opts[:adapter]

    # Fetch the session id and store if necessary

    # Get session data
    data = adapter.get(session_id)

    # Append session data to conn for use elsewhere in application
    conn = conn |> assign(:session, data)

    # Progress in plug stack
    conn = fun.(conn)

    # Store current state of session data
    adapter.put(sid, conn.assigns[:session])

    conn
  end
end

init/1 passes of to the adapter's init/1 function to allow for it to have its own set up at compile time. There is section in wrap/3 where the current session id is fetched. My prototype has the plug create the session id if one isn't present, while Dynamo's session filter delegates this responsibility to the session store. One benefit of delegating to the adapter is if a session id is required to be in a particular format/data type for the adapter.

I can provide a link to my prototype implementation if requested, but it would probably be best if we decide upon a defined specification regardless of what I have already done.

Hello!

What do you think about adding code coverage to plug? I have recently started using Elixir but already found a couple of code coverage tools which seem to do the job:

• https://github.com/alfert/coverex
• https://github.com/parroty/excoveralls

The first generates an HTML report of the coverage which looks like this:

while the second one is terminal based. Both feature the option to push coverage to https://coveralls.io.

I'm testing Plug with coverex and already wrote a couple of tests in order to cover untested sections of the code, and coverex is helping me out really well.

Thoughts?

When I try to compile plug, I receive the following:

lib/plug/conn.ex:124: warning: unused alias Conn

== Compilation error on file lib/plug/conn.ex ==
** (CompileError) lib/plug/conn.ex:87: undefined function ::/2
(stdlib) lists.erl:1352: :lists.mapfoldl/3
(stdlib) lists.erl:1353: :lists.mapfoldl/3

I've attempted to compile this against the phoenix framework and just plug alone with the same error.

== Compilation error on file lib/plug/parsers/multipart.ex ==
** (CompileError) lib/plug/parsers/multipart.ex:46: Plug.Upload.__struct__/0 is undefined, cannot expand struct Plug.Upload
    (elixir) src/elixir_map.erl:55: :elixir_map.translate_struct/4
    (stdlib) lists.erl:1352: :lists.mapfoldl/3
    (stdlib) lists.erl:1353: :lists.mapfoldl/3
    (elixir) src/elixir.erl:157: :elixir.erl_eval/2

I tried to find the definition of Plug.Upload in the code, but I couldn't seem to find it in the elixir-0.13.1 branch. Any ideas to get this compiling on 0.13.2-dev?

Maybe we have this documented somewhere but I haven't had luck finding it. Anyway, it would be nice to have a reference that explains how the Plug.Router handles a request and shows all the different transformations the conn goes through up until sending a response back to the client.

What do you think?

Hi,
I am encountering this error in Windows 7 with most current master on Elixir (VERSION=0.12.2-dev) .

I ran in the Windows Command Shell prompt:

mix new test_plug
cd test_plug

modified mix.exs to have the following snippets:

def application do
[ applications: [:cowboy, :plug],
mod: { TestPlug, [] }]
end

defp deps do
[ { :cowboy, github: "extend/cowboy" },
{ :plug, github: "elixir-lang/plug" } ]
end

then ran:

mix deps.get

And get the following error:

== Compilation failed ==
Compilation failed on the following files:

• lib/plug/adapters/cowboy/connection.ex is missing module :cowboy_req
• lib/plug/adapters/cowboy/handler.ex is missing module :cowboy_req

The first failure is shown below...
== Compilation error on file lib/plug/adapters/cowboy/connection.ex ==
could not compile dependency plug, mix compile failed. You can recompile this dependency with mix deps.compile plug or update it with mix deps.update plug
** (CompileError) deps/plug/lib/plug/adapters/cowboy/connection.ex:5: module :cowboy_req is not loaded and could not be found
(elixir) src/elixir_exp.erl:95: :elixir_exp.expand/2
(stdlib) lists.erl:1339: :lists.mapfoldl/3
(stdlib) lists.erl:1340: :lists.mapfoldl/3
(elixir) src/elixir_exp.erl:43: :elixir_exp.expand/2
(elixir) src/elixir.erl:150: :elixir.quoted_to_erl/3

Thanks!
Tina

I am running into an issue, where a header like Accept:application/json, text/javascript, */*; q=0.01 (generated by jQuery.ajax) can't be properly processed by software using Plug.Conn.Utils.media_type.

Master cowlib can do this without breaking:

:cow_http_hd.parse_accept("application/json, text/javascript, */*; q=0.01")
# =>
[{{"application", "json", []}, 1000, []},
 {{"text", "javascript", []}, 1000, []}, {{"*", "*", []}, 10, []}]```

Thoughts?

I'm seeing this compile error using 0.14.2. Any ideas?

lib/plug/conn/unfetched.ex:7: warning: type t/0 already exported
== Compilation error on file lib/plug/conn/unfetched.ex ==
** (CompileError) lib/plug/conn/unfetched.ex:7: type t() already defined
    (stdlib) lists.erl:1336: :lists.foreach/2
    (stdlib) erl_eval.erl:657: :erl_eval.do_apply/6
    (elixir) src/elixir.erl:163: :elixir.erl_eval/2
    (elixir) src/elixir.erl:156: :elixir.eval_forms/4
    (elixir) src/elixir_lexical.erl:17: :elixir_lexical.run/2
    (elixir) src/elixir.erl:163: :elixir.erl_eval/2

See https://travis-ci.org/elixir-web/weber/builds/29202180