Git Product home page Git Product logo

yada's Introduction

yada

CircleCI

yada is a web library for Clojure, designed to support the creation of production services via HTTP.

It has the following features:

  • Standards-based, comprehensive HTTP coverage (content negotiation, conditional requests, etc.)
  • Parameter validation and coercion, automatic Swagger support
  • Rich extensibility (methods, mime-types, security and more)
  • Asynchronous, efficient interceptor-chain design built on manifold
  • Excellent performance, suitable for heavy production workloads

yada is a sibling library to bidi - whereas bidi is based on routes as data, yada is based on resources as data.

The user-manual for the latest (1.x) release is available at https://juxt.pro/yada and offline (see below).

The user-manual is also available as an e-book or PDF, at Leanpub.

Installation

For the latest stable release, add the following dependency to your project.clj or build.boot file:

[yada "1.2.15"]

For the latest alpha release, add the following dependency to your project.clj or build.boot file:

[yada "1.3.0-alpha9"]

Build Status

Create a yada handler

Typically, yada handlers are created from a configuation expressed in data.

(require '[yada.yada :as yada])

(yada/handler
  {:methods
    {:get
      {:produces "text/html"
       :response "<h1>Hello World!</h1>"}}})

This is a simple example, there are a lot more options in yada than can be expressed here, but the approach is the same. The data configuration can be hand-authored, or generated programmatically leading enabling creation of consistent APIs at an industrial scale.

Dependencies

yada requires the following :-

  • a Java JDK/JRE installation, version 8 or above
  • Clojure 1.8.0

Support for other web-servers, such as undertow, are on the road-map.

Future compatibility

If you want to ensure that your code will not break with future releases of yada, you should only use functions from the yada.yada namespaces.

You are free to use other public functions in yada, but please be warned that these can and do change between releases.

Lean yada

By default, yada is batteries-included, bringing in a large number of dependencies.

However, a leaner version of yada is available which cuts out Swagger, swagger-ui, JSON (cheshire), Transit, buddy, core.async, SSE and other fat.

The following differences apply:

  • yada doesn't automatically encode/decode JSON bodies, or render JSON as HTML
  • no parameter validation or coercion
  • no Swagger
  • no SSE (currently)
  • no JWT
  • no Transit

To use the lean (or any other) variant of yada, specify the appropriate classifier in your project.clj or build.boot file:

[yada/lean "1.2.15"]

Running documentation and examples offline

Although yada is a library, if you clone this repo you can run the documentation and examples from the REPL.

cd yada
lein repl

Once the REPL starts, type in and run the following :-

user> (dev)
dev> (go)

Now browse to http://localhost:8090.

Troubleshooting FAQ

Q. I'm migrating from a version before yada 1.1 and my async multipart and other uploads are not working, sometimes throwing NullPointerExceptions or other errors.

A. Either use yada's built-in yada.server function or make sure you have started aleph's server with the option raw-stream? :true. Previous versions of yada left these settings up to the user but it's very important in yada 1.1 that raw-stream? is set.

Support

yadarians mostly chat in the Slack channel plus there is also a dedicated Gitter channel channel

Join the chat at https://gitter.im/juxt/yada

Also, there is a discussion group yada-discuss to discuss ideas.

Contributing

Feel free to raise GitHub issues on this repository.

Pull requests are welcome. Please run the test suite and check that all tests pass prior to submission.

$ lein test

If you want to build and test your own version of yada, you need to be aware how to locally install your own version. Since yada is broken into multiple Maven jars, each with their own version declaration, there is a script that allows you to set the version to whatever you need it to be.

$ ./set-version 1.3.0-MS-SNAPSHOT

Rather than use lein install, you should replace lein with ./treelein.

For example:

$ ./treelein install

This will install all the yada jars into your local Maven repository.

Acknowledgments

Thanks to the following people for inspiration, contributions, feedback and suggestions.

  • Malcolm Sparks
  • Martin Trojer
  • Philipp Meier
  • David Thomas Hume
  • Zach Tellman
  • Stijn Opheide
  • Frankie Sardo
  • Jon Pither
  • Håkan Råberg
  • Ernestas Lisauskas
  • Thomas van der Veen
  • Leandro Demartini
  • Craig McCraig of the clan McCraig
  • Imre Kószó
  • Luo Tian
  • Joshua Griffith
  • Joseph Fahey
  • David Smith
  • Mike Fikes
  • Brian Olsen
  • Stanislas Nanchen
  • Nicolas Ha
  • Eric Fode
  • Leon Mergen
  • Greg Look
  • Tom Coupland
  • Mikkel Gravgaard
  • Lucas Lago
  • Johannes Staffans
  • Michiel Borkent
  • James Laver
  • Marcin Jekot
  • Daniel Compton
  • Yoshito Komatsu
  • Bor Hodošček
  • Ivar Refsdal
  • Josh Graham
  • Joshua Ballanco
  • Steven Harms
  • Ryan Smith
  • Alexis Lee

Also, see the dependency list. In particular, yada would certainly not exist without the considerable efforts of those behind the following libraries.

  • Manifold & Aleph - Zach Tellman
  • Prismatic Schema - Jason Wolfe (and others)
  • Ring-swagger - Tommi Riemann (and others)

Copyright & License

The MIT License (MIT)

Copyright © 2015-2016 JUXT LTD.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

yada's People

Contributors

alxdb avatar borh avatar borkdude avatar boxxxie avatar danielcompton avatar elzibubble avatar ernestas avatar grav avatar greglook avatar griff avatar imrekoszo avatar josf avatar jstaffans avatar lvh avatar malcolmsparks avatar martinklepsch avatar mccraigmccraig avatar mfikes avatar mike706574 avatar naartjie avatar nha avatar rosado avatar roxolan0 avatar runejuhl avatar severeoverfl0w avatar sharms avatar spinningarrow avatar tangrammer avatar tanzoniteblack avatar tcoupland avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

yada's Issues

Swagger bidi match problem

Hi,

Is this a bug or am I doing something wrong?

(def route ["" {"/api" (y/swaggered
{:info {:title "test"
:version "0.1"
:description "Bug"}
:basePath "/api"}
["/v1" (yada.yada/yada "a")])}])

(bidi.bidi/match-route route "/api") -> nil

(bidi.bidi/match-route route "/api/") -> {:handler #object[yada.swagger.Swaggered$reify__41225 0x35792c1 "yada.swagger.Swaggered$reify__41225@35792c1"]}

Regards,

Timur

Swagger tags

In an older version of Yada, it was possible to give tags using the 'swagger' metadata.

In 1.1.0, it doesn't seem to have an effect.

If I try to it the :tags into the resource definition, it results in an error "Cannot turn resource-model into resource, because it doesn't conform to a resource-model schema..."

Is there another way to provide tags for swagger?

websocket supported

Do we have websocket supported within the lib ? Or is there any other way we can get websocket working with the lib ( using aleph websocket maybe ) ?

Thanks for creating a beautiful lib.

Could not locate clojure/core/cache

Yada cannot be included in a project with the following dependencies:

                 [[org.clojure/clojure "1.8.0"]
                 [org.clojure/clojurescript "1.7.170"]
                 [yada "1.1.0-20160202.093502-16"]
                 [aleph "0.4.1-beta3"]
                 [hiccup "1.0.5"]
                 [reagent "0.5.1"]
                 [re-frame "0.5.0"]
                 [bidi "1.25.0"]
                 [kibu/pushy "0.3.6"]
                 [cljsjs/react-bootstrap "0.27.3-0"]
                 [cljsjs/jquery "2.1.4-0"]
                 [cljsjs/lodash "3.10.1-0"]
                 [puppetlabs/trapperkeeper "1.2.0"]]

Exception is:

Could not locate clojure/core/cache__init.class or
clojure/core/cache.clj on classpath.

Current solution: Include [org.clojure/core.cache "0.6.4"] in dependencies.

Regards,

Timur

Parsing request body

I can find stuff that e.g. serializes return values to JSON, but there doesn't appear to be anything that puts stuff in the ctx the same way that stuff from the path or query parameters gets put in.

Similarly, headers; but I could see how that's a separate ticket.

Swagger seems to fail with the new version

With yada:
[yada "1.1.0-20160210.093249-22"]

The following example from the tutorial results in an exception:
["/api"
(swaggered
{:info {:title "Hello World!"
:version "1.0"
:description "A greetings service"}
:basePath "/api"}
["/greetings"
[
["/hello" (yada "Hello World!\n")]
["/goodbye" (yada "Goodbye!\n")]
]
])]

Exception:

               RT.java:  947  clojure.lang.RT/nthFrom
               RT.java:  897  clojure.lang.RT/nth
             bidi.cljc:  387  bidi.bidi$route_seq/invokeStatic
             bidi.cljc:  387  bidi.bidi$route_seq/invoke
             bidi.cljc:  391  bidi.bidi$route_seq/invokeStatic
             bidi.cljc:  387  bidi.bidi$route_seq/invoke
           swagger.clj:   89  yada.swagger/swagger-spec
           swagger.clj:   87  yada.swagger/swagger-spec
           RestFn.java:  425  clojure.lang.RestFn/invoke
           swagger.clj:  172  yada.swagger/swaggered
           swagger.clj:  171  yada.swagger/swaggered
           RestFn.java:  423  clojure.lang.RestFn/invoke

Is there any documentation, usage examples or sample code?

From reading the readme yada sounds really cool and something I feel I'd be very interested in, but I'm having a hard time figuring it out from the tests and source. I see there is a bunch of code in /dev/src/yada/dev and I'm slowly piecing it together, but as its quite dense with little commenting and few docstrings its quite a bit of detective work to figure out whats what and why.

Is there any other sample/example code or other documentation available anywhere?

Any and all help appreciated!

Nested arity exception

Problem described in comment in yada.methods/apply-response-fn

Needs a failing test.

500: Internal Server Error

There was an internal server error.

clojure.lang.ExceptionInfo: Error on GET {:response #yada.context.Response{:headers {}, :vary #{}, :produces {:media-type #yada.media_type.MediaTypeMap{:name "text/html", :type "text", :subtype "html", :parameters {}, :quality 1.0}}}, :resource yada.resource.Resource, :error #error {
 :cause "Wrong number of args (2) passed to: template/new-page-resource/responsefn--30958"
 :via
 [{:type clojure.lang.ArityException
   :message "Wrong number of args (2) passed to: template/new-page-resource/responsefn--30958"
   :at [clojure.lang.AFn throwArity "AFn.java" 429]}]
 :trace
 [[clojure.lang.AFn throwArity "AFn.java" 429]
  [clojure.lang.AFn invoke "AFn.java" 36]
  [clojure.lang.AFn applyToHelper "AFn.java" 156]
  [clojure.lang.AFn applyTo "AFn.java" 144]
  [clojure.core$apply invoke "core.clj" 632]
  [yada.methods$apply_response_fn invoke "methods.clj" 43]
  [yada.methods$eval23870$fn__23877$fn__23882 invoke "methods.clj" 186]

yada throws exception when media-type is not parsable

When a client sends an invalid Accept header e.g. 'Accept: application/json, text/plain, /', yada.media-type/string->media-type will throw an exception, resulting in a 500 response.

This should be 400 Bad Request.

Document dependency on Clojure 1.7?

Might be worth mentioning somewhere that this requires Clojure 1.7 (at least until 1.7 is out of beta) as it throws an exception on require otherwise. Took a little while to realise and might save others some time.

yada.bidi also crashes with a rather obscure exception if a version of bidi < 1.19.0 is being used (I was accidentally pulling in 1.18.9 through juxt.modular/bidi). Again, might be worth mentioning minimum version requirements.

error handler

I would like to be able to define what happens to internal server errors. Currently they return a 500 with the full stack trace, I want to not show the stack trace for example.

Support for other Web Servers

Hi,

Are you planning to support any other web servers in the near future?

The library looks quite promising but usage with only Aleph limits its adoption. (I have nothing against Aleph but probably its not the most commonly used one)

Regards,

Timur

:consumes should be available everywhere :produces is

By what I can generate for swagger-ui, :consumes is only accepted in the top level option map for swaggered:

(swaggered {:info {:title "Some API"
                   :version "1.0.0"
                   :description "Bla bla bla"}
            :consumes #{....}  ;;  <-----------------
            :basePath "/api/v1"} ...

(A quick scan of the Yada source doesn't reveal anything related to :consumes, as opposed to :produces)

text/plain Content-Type should specify charset when rendering a string

It seems to be UTF-8, but I don't know if something is specifying this explicitly somewhere or if that's just an accidental default somewhere.

Example of this going awry using httpie and a UTF-8 term:

➜  ~ http GET http://localhost:8090/user-guide/examples/ResourceStateWithBody/17382343
HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 24
Content-Type: text/plain
Date: Thu, 14 May 2015 03:37:25 GMT
Server: Aleph/0.4.0

Your balance is ฿1300

cannot treat nil as http response

I have this error when trying out yada and aleph

(ns components.yada
  (:require
   [clojure.pprint :refer [pprint]]
   [bidi.ring :refer [make-handler] :as bidi]
   [yada.yada :refer [yada] :as yada]
   [aleph.http :refer [start-server]]))

(def api
  ["/" {"hello" (yada "hello world" {:id :hello})
        "hello-atom" (yada (atom "hello world"))}])

(def server
  (start-server
   (bidi/make-handler api)
   {:port 3000}))
#_(.close server)

when I visit `http://localhost:3000/

java.lang.IllegalArgumentException: cannot treat nil as HTTP response for request to ''
    at aleph.http.server$invalid_value_response.invoke(server.clj:131)
    at aleph.http.server$handle_request$fn__23992$fn__23993.invoke(server.clj:184)
    at manifold.deferred$eval5912$chain_SINGLEQUOTE____5933.invoke(deferred.clj:750)
    at manifold.deferred$eval5912$subscribe__5913$fn__5918.invoke(deferred.clj:716)
    at manifold.deferred.Listener.onSuccess(deferred.clj:219)
    at manifold.deferred.Deferred$fn__5755.invoke(deferred.clj:396)
    at manifold.deferred.Deferred.success(deferred.clj:396)
    at manifold.deferred$success_BANG_.invoke(deferred.clj:243)
    at manifold.deferred$catch_SINGLEQUOTE_$fn__5996.invoke(deferred.clj:959)
    at manifold.deferred.Listener.onSuccess(deferred.clj:219)
    at manifold.deferred.Deferred$fn__5755.invoke(deferred.clj:396)
    at manifold.deferred.Deferred.success(deferred.clj:396)
    at manifold.deferred$success_BANG_.invoke(deferred.clj:243)
    at manifold.deferred$eval5912$chain_SINGLEQUOTE____5933.invoke(deferred.clj:737)
    at manifold.deferred$eval5912$subscribe__5913$fn__5914.invoke(deferred.clj:710)
    at manifold.deferred.Listener.onSuccess(deferred.clj:219)
    at manifold.deferred.Deferred$fn__5755.invoke(deferred.clj:396)
    at manifold.deferred.Deferred.success(deferred.clj:396)
    at manifold.deferred$success_BANG_.invoke(deferred.clj:243)
    at aleph.http.server$handle_request$fn__23977$f__5358__auto____23978.invoke(server.clj:152)
    at clojure.lang.AFn.run(AFn.java:22)
    at io.aleph.dirigiste.Executor$Worker$1.run(Executor.java:62)
    at manifold.executor$thread_factory$reify__5240$f__5241.invoke(executor.clj:36)
    at clojure.lang.AFn.run(AFn.java:22)
    at java.lang.Thread.run(Thread.java:745)

I'm very new to yada, so I'm sure I might miss something, but according to the manual this should work.

Demo site doesn't compile

The repository has a dependency not met on clojars, namely juxt.modular:co-dependency:jar:0.3.0.

Still if a revert it to [juxt.modular/co-dependency "0.2.2"] (or revert to a few commits backs), issuing lein run in the repository fails with:

Exception in thread "main" java.lang.IllegalArgumentException: Can't define method not in interfaces:
walker, compiling:(modular/component/co_dependency/schema.clj:7:1) 

HTTP status 418 requires BREW method

RFC 2324 states for the status code 418:

Any attempt to brew coffee with a teapot should result in the error
code "418 I'm a teapot". The resulting entity body MAY be short and
stout.

http://tools.ietf.org/html/rfc2324#section-2.3.2

To enable proper HTCPCP for yada the BREW method must be supported.

Additionally, the mandatory media type for a response to the BREW method is message/coffeepot

http://tools.ietf.org/html/rfc2324#section-4

By having a correct implementation for the HTCPCP yada could show it's full power and flexibility.

Yada-discuss should be open

You must be a member of this group to view and participate in it.

I think users should be able to at least see what's being discussed... if only to increase the number of potential posters.

Problems with CORS

CORS seems to have a few issues right now. Going by this comment in the source[1], I understand this is still a work in progress, but here are the two issues I've encountered so far anyway in case its helpful:

  1. I get this error when doing a cross-origin request from Chrome: Request header field Content-Type is not allowed by Access-Control-Allow-Headers. I was using cljs-ajax to perform the requests and it automatically sends Content-Type, I believe. Maybe the headers list used in https://github.com/juxt/yada/blob/master/src/yada/core.clj#L129 should be pulled from the context, if provided?
  2. Preflight requests trigger the authorization function (which then fails for me because the requests do not contain the credentials my auth function looks for). Perhaps OPTION requests should bypass the :authorization step?

[1] CORS support: build this in, make allow-origin first-class, which headers should be allowed should be a hook (with default)

[juxt/iota "0.2.1"] not in clojars

Hi Malcolm again!
I tried to start the repl with a fresh cloned yada repo but I got this error
Caused by: org.sonatype.aether.resolution.DependencyResolutionException: Could not find artifact juxt:iota:jar:0.2.1 in central (https://repo1.maven.org/maven2/)
and after checking in clojars last version is still 0.2.0 ....
could you deploy last required version?

thanks in advance!
Juan

PS: anyway i'll downgrade iota version and i'll try again hopefully it's not a breaking change

Availability of parsed parameters in yada.protocols/Properties

We already discussed this on slack, but maybe I should add it here too.

When calculating the properties on request (properties [_ ctx]) arity, it would be nice to have parameters parsed already. This would mean that the get-properties interceptor would be executed after the malformed? interceptor.

As far as I can see,

method-allowed?
process-request-body
malformed?

do not need anything that is processed in get-properties, at least not the properties on request version. But I might be wrong on that one.

On the other hand if we have the output of malformed? already in the ctx, get-properties would be much easier to implement, especially if you need to query e.g. a database / filesystem to answer :exists?, :representations, :last-modified, ...

NPE handling large multipart upload

An NPE will occur in yada.multipart/finish-up if you upload a large file to a Yada-based server.

To repro on OS X, clone the yada repo and start up lein repl in its top level directory. Then

(require '[aleph.http :refer [start-server]]
         '[bidi.ring :refer [make-handler]]
         '[yada.yada :refer [resource]])

and

(start-server
  (make-handler
    ["/upload"
     (resource
       {:methods
        {:post
         {:consumes "multipart/form-data"
          :response (constantly nil)}}})])
  {:port 3000})

Then create a large file:

dd if=/dev/zero of=file.txt count=1024 bs=1024

and use cURL to upload the file to the server:

curl -F [email protected] http://localhost:3000/upload

You should get an error:

20:02:42.021 [manifold-pool-2-7] ERROR yada.multipart - Failed: null
java.lang.NullPointerException: null
    at clojure.lang.Numbers.ops(Numbers.java:1013)
    at clojure.lang.Numbers.add(Numbers.java:128)
    at yada.multipart$finish_up.invoke(multipart.clj:313)
    at yada.multipart$process_chunk.invoke(multipart.clj:334)
    at yada.multipart$parse_multipart$this__2646__auto____24204$fn__24205$fn__24206.invoke(multipart.clj:420)
    at manifold.deferred$eval2555$chain___2576.invoke(deferred.clj:860)
    at yada.multipart$parse_multipart$this__2646__auto____24204$fn__24205.invoke(multipart.clj:414)
    at yada.multipart$parse_multipart$this__2646__auto____24204.invoke(multipart.clj:402)
    at clojure.lang.AFn.applyToHelper(AFn.java:156)
    at clojure.lang.AFn.applyTo(AFn.java:144)
    at clojure.core$apply.invoke(core.clj:632)
    at yada.multipart$parse_multipart$this__2646__auto____24204$fn__24232.invoke(multipart.clj:402)
    at manifold.deferred.Listener.onSuccess(deferred.clj:219)
    at manifold.deferred.Deferred$fn__2357$fn__2358.invoke(deferred.clj:396)
    at clojure.lang.AFn.run(AFn.java:22)
    at io.aleph.dirigiste.Executor$Worker$1.run(Executor.java:62)
    at manifold.executor$thread_factory$reify__1758$f__1759.invoke(executor.clj:36)
    at clojure.lang.AFn.run(AFn.java:22)
    at java.lang.Thread.run(Thread.java:745)

Dependencies are not available

Hi yada guys!
I'm trying to get a try to this code but this is the result of cider-jack-inor lein repl

...
error in process sentinel: Could not start nREPL server: Retrieving prismatic/schema/0.3.5/schema-0.3.5.pom from clojars

...
Could not find artifact juxt.modular:aleph:jar:0.0.3 in central (https://repo1.maven.org/maven2/)
Could not find artifact juxt.modular:aleph:jar:0.0.3 in clojars (https://clojars.org/repo/)

cheers!
Juan

support schema Any

When using prismatic/schema Any yada.coerce throws an error :

For instance :
:parameters {:query { (sch/optional-key sch/Any) sch/Any}} will throw :

java.lang.IllegalArgumentException: No implementation of method: :to-key of protocol: #'yada.coerce/ParameterKey found for class: schema.core.AnythingSchema

500 Status Error using yada.resources.misc/just-methods

Hi yada guys!
all was working fine when I was using a yada/yada with a fn to have :get by default

(ns dev-system
  "Dev Components and their dependency reationships"
  (:require
...
   [yada.yada :as yada]
   [yada.resources.misc :refer (just-methods)]
   [bidi.bidi :refer [RouteProvider]]
   [bidi.ring :refer (make-handler)]
...
  ))
(reify
    RouteProvider
    (routes [_]
     ["/api/v1"
      (yada/swaggered
       {:info {:title "api backend"
               :version "0.0"
               :description "having good times with clojure and rest"}
        :basePath "/api/v1"}
       ["/" {"masterdata/" {[:uuid] (yada/yada
                                     (fn [ctx] (str "UUID: " (get-in ctx [:parameters :uuid])))
                                     {:parameters {:get {:path {:uuid String}}}})}}])]))

screen shot 2015-10-23 at 13 09 12

But i get 500 status error when i use just-methods to specify the same definition

(reify
    RouteProvider
    (routes [_]
     ["/api/v1"
      (yada/swaggered
       {:info {:title "api backend"
               :version "0.0"
               :description "having good times with clojure and rest"}
        :basePath "/api/v1"}
       ["/" {"masterdata/" {[:uuid] (yada/yada
                                     (just-methods
                                      :get {:response (fn [ctx]
                                                        (str "UUID: " (get-in ctx [:parameters :uuid])))
                                            :parameters {:path {:uuid String}}}))}}])]))

screen shot 2015-10-23 at 13 10 18

Any ideas about, maybe I'm missing something??
Thanks in advance!
Juan

yada/swaggered seems to think this regex is a part of the resource path

I have a resource where an email address is part of the path like "/foo/by-email/[email protected]"

A bidi route accepting this with :email as a route param would look like:

[["/foo/" [#".+\@.+\..+" :email]] :foo]

Note the regex format for the :email param which is required by bidi to parse the email as a route param. This is explained in juxt/bidi#98 (comment)

When I created a yada/swaggered api on top of such a routing I got an error saying Pattern does not implement the encode method of yada.swagger/SwaggerPath which makes me think yada/swaggered thinks the regex in the above example is a separate path element from :email. For now I fixed it in my project with:

(extend-protocol yada.swagger/SwaggerPath
  java.util.regex.Pattern (encode [_] nil))

But I doubt this is the right thing to do in the long term.

Direct memory exhaustion large multipart upload

If you do a large multipart upload with your own Partial and PartConsumer implementations (which could stream to storage, but instead simply discard the uploaded data), then the system will exhaust direct memory within a few minutes.

To reproduce, first do lein new foo to create a blank project and add the following to the :dependencies vector in project.clj:

[aleph "0.4.1-beta5"]
[bidi "2.0.1"]
[yada "1.1.2"]

then, start a REPL in that project and issue the following forms in the REPL:

(require '[aleph.http :refer [start-server]]
         '[yada.yada :refer [resource]]
         '[yada.aleph :refer [listener]]
         '[yada.request-body :as request-body])

(import (yada.multipart Partial PartConsumer))

(defrecord MyPartial []
  Partial
  (continue [this _] (prn (java.util.Date.)) this)
  (complete [_ state _] state))

(defrecord MyPartConsumer []
  PartConsumer
  (consume-part [_ _ _])
  (start-partial [_ _] (->MyPartial))
  (part-coercion-matcher [_] {}))

;; Hook in our own part consumer
(let [original-impl (get-method request-body/process-request-body "multipart/form-data")]
  (defmethod request-body/process-request-body "multipart/form-data"
    [ctx body-stream media-type & args]
    (let [new-ctx (assoc-in ctx [:options :part-consumer] (->MyPartConsumer))]
      (apply original-impl new-ctx body-stream media-type args))))

(listener
  ["/upload"
   (resource
     {:methods
      {:post
       {:consumes "multipart/form-data"
        :response (constantly nil)}}})]
  {:port 3000})

(Note that the process-request-body multimethod is modified in lieu of :options setting.)

Now, make a large file:

dd if=/dev/zero of=file.txt count=10240000 bs=1024

and upload it to the server started in the REPL:

curl -F [email protected] http://localhost:3000/upload

Timestamps will be displayed in the REPL as chunks come in:

#inst "2016-03-18T02:03:40.141-00:00"
#inst "2016-03-18T02:03:40.188-00:00"

after a few minutes things will fail. On Linux, the exception will look like the following:

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 3653632 bytes for committing reserved memory.
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000ff815000, 3653632, 0) failed; error='Cannot allocate memory' (errno=12)
# An error report file with more information is saved as:
# /home/vagrant/yada-dm/hs_err_pid3621.log
Exception in thread "Thread-3" clojure.lang.ExceptionInfo: Subprocess failed {:exit-code 1}
    at clojure.core$ex_info.invokeStatic(core.clj:4617)
    at clojure.core$ex_info.invoke(core.clj:4617)
    at leiningen.core.eval$fn__5625.invokeStatic(eval.clj:271)
    at leiningen.core.eval$fn__5625.invoke(eval.clj:267)
    at clojure.lang.MultiFn.invoke(MultiFn.java:233)
    at leiningen.core.eval$eval_in_project.invokeStatic(eval.clj:373)
    at leiningen.core.eval$eval_in_project.invoke(eval.clj:363)
    at leiningen.repl$server$fn__11767.invoke(repl.clj:243)
    at clojure.lang.AFn.applyToHelper(AFn.java:152)
    at clojure.lang.AFn.applyTo(AFn.java:144)
    at clojure.core$apply.invokeStatic(core.clj:646)
    at clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1881)
    at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1881)
    at clojure.lang.RestFn.invoke(RestFn.java:425)
    at clojure.lang.AFn.applyToHelper(AFn.java:156)
    at clojure.lang.RestFn.applyTo(RestFn.java:132)
    at clojure.core$apply.invokeStatic(core.clj:650)
    at clojure.core$bound_fn_STAR_$fn__4671.doInvoke(core.clj:1911)
    at clojure.lang.RestFn.invoke(RestFn.java:397)
    at clojure.lang.AFn.run(AFn.java:22)
    at java.lang.Thread.run(Thread.java:745)

SocketException The transport's socket appears to have lost its connection to the nREPL server
    clojure.tools.nrepl.transport/bencode/fn--10128/fn--10129 (transport.clj:95)
    clojure.tools.nrepl.transport/bencode/fn--10128 (transport.clj:95)
    clojure.tools.nrepl.transport/fn-transport/fn--10100 (transport.clj:42)
    clojure.core/binding-conveyor-fn/fn--4676 (core.clj:1938)
    java.util.concurrent.FutureTask.run (FutureTask.java:266)
    java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1142)
    java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:617)
    java.lang.Thread.run (Thread.java:745)

And on OS X you will see:

Mar 17, 2016 6:46:51 PM clojure.tools.logging$eval420$fn__424 invoke
WARNING: error in HTTP server
java.lang.OutOfMemoryError: Direct buffer memory
    at java.nio.Bits.reserveMemory(Bits.java:693)
    at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
    at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
    at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:645)
    at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:229)
    at io.netty.buffer.PoolArena.allocate(PoolArena.java:213)
    at io.netty.buffer.PoolArena.allocate(PoolArena.java:133)
    at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:262)
    at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)
    at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:170)
    at io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:131)
    at io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:73)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:111)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:510)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:467)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:381)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:353)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:742)
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
    at java.lang.Thread.run(Thread.java:745)

Produces doesn't result in charset in content type

Hello, I can't say I'm certain about this, but I'm wondering why, when I specify a charset in produces for a resource, the response doesn't include the charset in the content type of the response? My example is this (probably not the best place for it, but it shows the problem):

I changed the hello-world example to have a new resource:

...
(defn say-hello-json [ctx]
  {:say "hello" :p (get-in ctx [:parameters :query :p])})
....
(defn hello-json []
  (yada
   (resource
    {:methods
     {:get
      {:parameters {:query {:p String}}
       :produces [{:media-type "application/json"
                   :charset "utf-8"}]
       :response say-hello-json}}}))) 
...
(routes
   ...
   ["hello-json" (tag (hello-json) ::hello-json)]
   ...
)

and I created a new test in helloworld_test.clj like this:

(deftest json-test
  (let [resource (hello/hello-json)
        response @(resource (request :get "/?p=bob"))
        headers (:headers response)]
    (is (= 200 (:status response)))
    (is (= (to-string (:body response)) "{\"say\":\"hello\",\"p\":\"bob\"}"))
    (is (= (get-in response [:headers "content-type"]) "application/json;charset=utf-8"))))

My thinking is that, as I specified the resource to have :produces with :charset "utf-8, the response should have a content-type of application/json;charset-utf-8. However the response only has the content-type application/json.

And my question is: is this a bug? Or not?

Also, another point, I wonder why the actual body has a trailing \n?

If you look at my branch https://github.com/mdaley/yada you'll be able to see my check-in.

If this is a problem, if you give me some pointers, I'd be more than willing to be involved in fixing it.

Invalid json gives 500

sending {"x":"1} to an endpoint that accepts json will result in a 500 where cheshire is unable to parse it, this should be a 400.

Return body from PUT or POST

Currently it's impossible to return a body that contains e.g. a map of data from a PUT or POST method.

SEVERE: error sending body of type  clojure.lang.PersistentArrayMap
java.lang.IllegalArgumentException: Don't know how to convert class clojure.lang.PersistentArrayMap into (stream-of io.netty.buffer.ByteBuf)
    at byte_streams$convert.invoke(byte_streams.clj:187)
    at aleph.netty$to_byte_buf_stream.invoke(netty.clj:170)
    at aleph.http.core$send_streaming_body.invoke(core.clj:277)
    at aleph.http.core$eval24414$send_message__24421$fn__24422.invoke(core.clj:353)
    at aleph.http.core$eval24414$send_message__24421.invoke(core.clj:352)
    at aleph.http.server$eval24451$send_response__24456$f__24005__auto____24460.invoke(server.clj:124)
    at aleph.http.server$eval24451$send_response__24456$fn__24463.invoke(server.clj:117)
    at clojure.lang.AFn.run(AFn.java:22)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:339)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:356)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:742)
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
    at java.lang.Thread.run(Thread.java:745)

The reason is that there is no body conversion in PUT/POST as it is done in GetMethod/request.

Is this something yada intends to support? The HTTP spec allows for either referencing the newly created/updated entity or actually specifying it.

http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
Section 10.2.1 200 OK
POST an entity describing or containing the result of the action;

There's no mention about PUT 200 or 201 other than The newly created resource can be referenced by the URI(s) returned in the entity of the response, with the most specific URI for the resource given by a Location header field. The response SHOULD include an entity containing a list of resource characteristics and location(s) from which the user or user agent can choose the one most appropriate.

As this impacts all methods that can return a body, I'm not sure how you'd like to pull that out of the GetMethod.

Dependency on unreleased iota 0.2.3

yada's project.clj cites iota 0.2.3 (latest generally available 0.2.2) and co-dependency 0.3.0 (latest 0.2.2).

(Thanks for a fab-looking library, by the way.)

DELETE should return 404 when resource does not exist

DELETE always returns 204 regardless of the value of :exists?.

According to the spec:

A successful response SHOULD be 200 (OK) if the response includes an entity describing the status, 202 (Accepted) if the action has not yet been enacted, or 204 (No Content) if the action has been enacted but the response does not include an entity.

However, deleting a non-existent resource, I believe should not be considered 'successful'.

Headers map is not ring compliant - may contains Integers

The "content-length" element is an Integer. But it would be nice if the :headers element was compatible with the ring schema because other libraries depend on this. Quote from the ring wiki:

:headers A Clojure map of HTTP header names to header values. These values may either be strings, in which case one name/value header will be sent in the HTTP response, or a collection of strings, in which case a name/value header will be sent for each value.

Confused about compositional relationships & protocols

Very cool project!

This is a documentation issue. I've spent about a day reading all the docs, source and example code. I'm still pretty confused about the fundamentals.

The main issue is that there are a lot of protocols, and multiple layers of abstraction. Bidi, Yada, and their Modular companions. There are several things that seem to abstract Ring for example. There are multiple things that get coerced into other things. Some docs explain what can go directly into what would be helpful.

Simple example of confusion:

I see RouteProvider used a lot. I thought I could just drop something implementing RouteProvider into a bidi route tree. Doesn't seem to work.

I'm also pretty confused about all the protocols implemented by Handler, versus things like WebRequest and the various flavors of ring wrapping.

In general the execution model is unclear (when all these protocols would be invoked during a) creating the master ring handler, and b) during request processing.

OPTIONS request fails when giving a fn to yada

When you create a resource with yada and pas it a function (to only have a GET) operation

(yada/yada 
  (fn [ctx]
    (when-let [...] ...))
    {:parameters {:get {:query {:q String}}}})

when sending an OPTIONS request, it throws:

java.lang.IllegalArgumentException: No implementation of method: :interpret-options-result of protocol: #'yada.methods/OptionsResult found for class: nil

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.