Git Product home page Git Product logo

goliath's Introduction

Goliath

Goliath is an open source version of the non-blocking (asynchronous) Ruby web server framework. It is a lightweight framework designed to meet the following goals: bare metal performance, Rack API and middleware support, simple configuration, fully asynchronous processing, and readable and maintainable code (read: no callbacks).

The framework is powered by an EventMachine reactor, a high-performance HTTP parser and Ruby 1.9+ runtime. The one major advantage Goliath has over other asynchronous frameworks is the fact that by leveraging Ruby fibers introduced in Ruby 1.9+, it can untangle the complicated callback-based code into a format we are all familiar and comfortable with: linear execution, which leads to more maintainable and readable code.

Each HTTP request within Goliath is executed within its own Ruby fiber and all asynchronous I/O operations can transparently suspend and later resume the processing without requiring the developer to write any additional code. Both request processing and response processing can be done in fully asynchronous fashion: streaming uploads, firehose API's, request/response, websockets, and so on.

Installation & Prerequisites

  • Install Ruby 1.9 (via RVM or natively)
$> gem install rvm
$> rvm install 1.9.3
$> rvm use 1.9.3
  • Install Goliath:
$> gem install goliath

Getting Started: Hello World

require 'goliath'

class Hello < Goliath::API
  def response(env)
    [200, {}, "Hello World"]
  end
end

> ruby hello.rb -sv
> [97570:INFO] 2011-02-15 00:33:51 :: Starting server on 0.0.0.0:9000 in development mode. Watch out for stones.

See examples directory for more hands-on examples of building Goliath powered web-services.

Performance: MRI, JRuby, Rubinius

Goliath is not tied to a single Ruby runtime - it is able to run on MRI Ruby, JRuby and Rubinius today. Depending on which platform you are working with, you will see different performance characteristics. At the moment, we recommend MRI Ruby 1.9.3+ as the best performing VM: a roundtrip through the full Goliath stack on MRI 1.9.3 takes ~0.33ms (~3000 req/s).

Goliath has been used in production environments for 2+ years, across many different companies: PostRank (now Google), OMGPOP (now Zynga), GameSpy, and many others.

FAQ

  • How does Goliath compare to other Ruby async app-servers like Thin?

    • They are similar (both use Eventmachine reactor), but also very different. Goliath is able to run on different Ruby platforms (see above), uses a different HTTP parser, supports HTTP keepalive & pipelining, and offers a fully asynchronous API for both request and response processing.
  • How does Goliath compare to Mongrel, Passenger, Unicorn?

    • Mongrel is a threaded web-server, and both Passenger and Unicorn fork an entire VM to isolate each request from each other. By contrast, Goliath builds a single instance of the Rack app and runs all requests in parallel through a single VM, which leads to a much smaller memory footprint and less overhead.
  • How do I deploy Goliath in production?

    • We recommend deploying Goliath behind a reverse proxy such as HAProxy (sample config), Nginx or equivalent. Using one of the above, you can easily run multiple instances of the same application and load balance between them within the reverse proxy.

Guides

Hands-on applications:

If you are you new to EventMachine, or want a detailed walk-through of building a Goliath powered API? You're in luck, a super-awesome Pluralsight screencast which will teach you all you need to know:

Additionally, you can also watch this presentation from GoGaRuCo 2011, which describes the design and motivation behind Goliath:

Other resources:

Discussion and Support

License & Acknowledgments

Goliath is distributed under the MIT license, for full details please see the LICENSE file.

goliath's People

Contributors

aenain avatar cldwalker avatar dj2 avatar djones avatar duksis avatar dustalov avatar endel avatar fatum avatar gregwebs avatar igrigorik avatar janko avatar joshbuddy avatar juanmcuello avatar justinko avatar karlfreeman avatar kybishop avatar mattetti avatar mikelewis avatar moretea avatar nolman avatar radsaq avatar schmurfy avatar seenmyfate avatar sleeper avatar stevemohapibanks avatar subosito avatar timblair avatar tomykaira avatar wicz avatar xentek avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

goliath's Issues

Goliath swallows server errors as 400

I put a an exception in my response fail "WTF?"

Here is the output I see:

[2177:ERROR] 2011-05-16 14:24:50 :: WTF?
[2177:ERROR] 2011-05-16 14:24:50 :: ./epg.rb:184:in `response'
/home/gweber/.rvm/gems/ruby-1.9.2-p136/bundler/gems/goliath-dcf777db420c/lib/goliath/api.rb:155:in `block in call'
[2177:INFO] 2011-05-16 14:24:50 :: Status: 400, Content-Length: 16, Response Time: 76.87ms
[2177:INFO] 2011-05-16 14:24:50 :: Status: 400, Content-Length: 34, Response Time: 20.38ms

So there is a 400 instead of 500. This issue has also alerted me to the fact that I now see an extra 400 request tacked onto every request- not sure where I went wrong there :)

config loaded based on $0, not current directory

Let's say I have a layout like this:

goliath
|-server
|---api.rb

To run my app, I cd goliath && ruby goliath/server/api.rb. Per the Configuration documentation I would expect this to load goliath/config/api.rb automatically; however, it really loads goliath/server/config/api.rb.

I've tracked this down to the Dir.chdir call in runner.rb. I tried tracing the blame/history back on these lines, but I don't really understand what the motivation is here — something to do with Rack::Reloader it seems. Removing those two lines seems to give the expected behavior, but I'm not sure what other implications it may have.

Do people agree that loading config relative to $0 seems odd? If so, is anyone able to shed some light on the current behavior so I might be able to propose a patch?

1 Chrome request becomes 2 requests

This can be replicated with a simple hello world example.

require 'goliath'

class Minimum < Goliath::API
  def response(env)
    [200, {}, "Hello world"]
  end
end

Run that and point Chrome to localhost:9000 and two requests pop up in the output log.

[3752:INFO] 2011-03-23 17:43:52 :: Status: 200, Content-Length: 4, Response Time: 0.54ms
[3752:INFO] 2011-03-23 17:43:52 :: Status: 200, Content-Length: 4, Response Time: 0.41ms

The first one is rendered in the browser, it seems like the browser is not aware of the second one happening at all. If inspecting the params it turns out that the second request is void of params. I did not see this happening in Firefox or Safari.

undefined method get on require goliath/test_helper

require 'bundler'

Bundler.setup
Bundler.require

require 'goliath/test_helper'
rspec spec/integration/epg_spec.rb
/home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/em-synchrony-0.3.0.beta.1/lib/em-synchrony/em-http.rb:21:in `class_eval': undefined method `get' for module `EventMachine::HTTPMethods' (NameError)
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/em-synchrony-0.3.0.beta.1/lib/em-synchrony/em-http.rb:21:in `class_eval'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/em-synchrony-0.3.0.beta.1/lib/em-synchrony/em-http.rb:21:in `block in '
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/em-synchrony-0.3.0.beta.1/lib/em-synchrony/em-http.rb:9:in `each'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/em-synchrony-0.3.0.beta.1/lib/em-synchrony/em-http.rb:9:in `'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/em-synchrony-0.3.0.beta.1/lib/em-synchrony/em-http.rb:8:in `'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/em-synchrony-0.3.0.beta.1/lib/em-synchrony/em-http.rb:7:in `'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/activesupport-3.0.3/lib/active_support/dependencies.rb:239:in `require'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/activesupport-3.0.3/lib/active_support/dependencies.rb:239:in `block in require'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/activesupport-3.0.3/lib/active_support/dependencies.rb:225:in `block in load_dependency'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/activesupport-3.0.3/lib/active_support/dependencies.rb:596:in `new_constants_in'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/activesupport-3.0.3/lib/active_support/dependencies.rb:225:in `load_dependency'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/activesupport-3.0.3/lib/active_support/dependencies.rb:239:in `require'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/goliath-0.9.1/lib/goliath/test_helper.rb:2:in `'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/activesupport-3.0.3/lib/active_support/dependencies.rb:239:in `require'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/activesupport-3.0.3/lib/active_support/dependencies.rb:239:in `block in require'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/activesupport-3.0.3/lib/active_support/dependencies.rb:225:in `block in load_dependency'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/activesupport-3.0.3/lib/active_support/dependencies.rb:596:in `new_constants_in'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/activesupport-3.0.3/lib/active_support/dependencies.rb:225:in `load_dependency'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/activesupport-3.0.3/lib/active_support/dependencies.rb:239:in `require'
    from /home/gweber/proj/yap/yap-epg/spec/spec_helper.rb:6:in `'
    from /home/gweber/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
    from /home/gweber/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
    from /home/gweber/proj/yap/yap-epg/spec/integration/epg_spec.rb:2:in `'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/rspec-core-2.5.1/lib/rspec/core/configuration.rb:386:in `load'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/rspec-core-2.5.1/lib/rspec/core/configuration.rb:386:in `block in load_spec_files'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/rspec-core-2.5.1/lib/rspec/core/configuration.rb:386:in `map'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/rspec-core-2.5.1/lib/rspec/core/configuration.rb:386:in `load_spec_files'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/rspec-core-2.5.1/lib/rspec/core/command_line.rb:18:in `run'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/rspec-core-2.5.1/lib/rspec/core/runner.rb:55:in `run_in_process'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/rspec-core-2.5.1/lib/rspec/core/runner.rb:46:in `run'
    from /home/gweber/.rvm/gems/ruby-1.9.2-p0/gems/rspec-core-2.5.1/lib/rspec/core/runner.rb:10:in `block in autorun'

goliath should check body content type prior to parsing

If the client posts a non "application/x-www-form-urlencoded" body.. Goliath should not try to form decode it. Ex: application/json, or application/xml.

We could perhaps add an automatic JSON handler based on request content-type, but even that is gravy.

Rasterize example broken?

Just a little heads up:

On 1.9.2 on OSX Lion:

$ ruby rasterize.rb -sv
[12665:INFO] 2011-09-05 10:06:22 :: Starting server on 0.0.0.0:9000 in development mode. Watch out for stones.
/Users/mseeger/.rvm/gems/ruby-1.9.2-p290/gems/em-synchrony-1.0.0/lib/em-synchrony.rb:25: [BUG] Segmentation fault
ruby 1.9.2p290 (2011-07-09 revision 32553) [x86_64-darwin11.0.0]

-- control frame ----------
c:0014 p:---- s:0051 b:0051 l:000050 d:000050 CFUNC  :resume
c:0013 p:0021 s:0048 b:0048 l:001e08 d:001018 BLOCK  /Users/mseeger/.rvm/gems/ruby-1.9.2-p290/gems/em-synchrony-1.0.0/lib/em-synchrony.rb:25
c:0012 p:---- s:0046 b:0046 l:000045 d:000045 FINISH
c:0011 p:---- s:0044 b:0044 l:000043 d:000043 CFUNC  :call
c:0010 p:---- s:0042 b:0042 l:000041 d:000041 CFUNC  :run_machine
c:0009 p:0248 s:0039 b:0039 l:000038 d:000038 METHOD /Users/mseeger/.rvm/gems/ruby-1.9.2-p290/gems/eventmachine-1.0.0.beta.3/lib/eventmachine.rb:199
c:0008 p:0056 s:0032 b:0032 l:001e08 d:001e08 METHOD /Users/mseeger/.rvm/gems/ruby-1.9.2-p290/gems/em-synchrony-1.0.0/lib/em-synchrony.rb:27
c:0007 p:0017 s:0025 b:0025 l:001d30 d:001d30 METHOD /Users/mseeger/test/goliath/lib/goliath/server.rb:72
c:0006 p:0166 s:0021 b:0021 l:000020 d:000020 METHOD /Users/mseeger/test/goliath/lib/goliath/runner.rb:224
c:0005 p:0122 s:0016 b:0016 l:000015 d:000015 METHOD /Users/mseeger/test/goliath/lib/goliath/runner.rb:158
c:0004 p:0184 s:0013 b:0013 l:000012 d:000012 METHOD /Users/mseeger/test/goliath/lib/goliath/application.rb:123
c:0003 p:0048 s:0006 b:0006 l:0006c8 d:000005 BLOCK  /Users/mseeger/test/goliath/lib/goliath/application.rb:141
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
c:0001 p:0000 s:0002 b:0002 l:0015e8 d:0015e8 TOP   
---------------------------
-- Ruby level backtrace information ----------------------------------------
/Users/mseeger/test/goliath/lib/goliath/application.rb:141:in `block in <module:Goliath>'
/Users/mseeger/test/goliath/lib/goliath/application.rb:123:in `run!'
/Users/mseeger/test/goliath/lib/goliath/runner.rb:158:in `run'
/Users/mseeger/test/goliath/lib/goliath/runner.rb:224:in `run_server'
/Users/mseeger/test/goliath/lib/goliath/server.rb:72:in `start'
/Users/mseeger/.rvm/gems/ruby-1.9.2-p290/gems/em-synchrony-1.0.0/lib/em-synchrony.rb:27:in `synchrony'
/Users/mseeger/.rvm/gems/ruby-1.9.2-p290/gems/eventmachine-1.0.0.beta.3/lib/eventmachine.rb:199:in `run'
/Users/mseeger/.rvm/gems/ruby-1.9.2-p290/gems/eventmachine-1.0.0.beta.3/lib/eventmachine.rb:199:in `run_machine'
/Users/mseeger/.rvm/gems/ruby-1.9.2-p290/gems/eventmachine-1.0.0.beta.3/lib/eventmachine.rb:199:in `call'
/Users/mseeger/.rvm/gems/ruby-1.9.2-p290/gems/em-synchrony-1.0.0/lib/em-synchrony.rb:25:in `block in synchrony'
/Users/mseeger/.rvm/gems/ruby-1.9.2-p290/gems/em-synchrony-1.0.0/lib/em-synchrony.rb:25:in `resume'

-- C level backtrace information -------------------------------------------

[NOTE]
You may have encountered a bug in the Ruby interpreter or extension libraries.
Bug reports are welcome.
For details: http://www.ruby-lang.org/bugreport.html

Abort trap: 6

On 1.9.3:

ruby -v
ruby 1.9.3dev (2011-07-31 revision 32789) [x86_64-darwin11.1.0]

url = PostRank::URI.clean(params['url'])

results in : [:error, "undefined local variable or method `params' for #Rasterize:0x007feba9c2b090"]

url = PostRank::URI.clean(env['params']['url'])

Seems to work better, but I also don't get any screenshots back (probably a faulty node.js setup somewhere along the way)

Gemspec uses git ls-files

For installations where git is not available, the gemspec should not call out to git. Using Dir::[] and grep should be sufficient, or Rake's FileList.

Sending Content-Type:application/json with empty body breaks API

curl -H'Content-Type:application/json' "localhost:9000?appkey=a"

Results in an exception in params parsing code and hangs the connection.

A more sane way to handle this should be:

  1. If the body is nil, don't try and parse it, we're safe to continue...
  2. If the body is not nil, and parsing of the body fails a 500 should returned by the server

Call to env.stream_close causes server exception

Calling env_stream.close method from the API.response raises an exception which leads to an infinite recursion :

def response(env)
       env.stream_close
        [200, {}, 'Hello World']
end
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/eventmachine-1.0.0.beta.3/lib/em/deferrable.rb:47:in `call'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/eventmachine-1.0.0.beta.3/lib/em/deferrable.rb:47:in `callback'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/request.rb:162:in `post_process'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/request.rb:189:in `server_exception'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/request.rb:172:in `rescue in block in post_process'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/request.rb:163:in `block in post_process'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/eventmachine-1.0.0.beta.3/lib/em/deferrable.rb:47:in `call'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/eventmachine-1.0.0.beta.3/lib/em/deferrable.rb:47:in `callback'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/request.rb:162:in `post_process'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/request.rb:189:in `server_exception'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/request.rb:172:in `rescue in block in post_process'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/request.rb:163:in `block in post_process'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/eventmachine-1.0.0.beta.3/lib/em/deferrable.rb:47:in `call'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/eventmachine-1.0.0.beta.3/lib/em/deferrable.rb:47:in `callback'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/request.rb:162:in `post_process'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/request.rb:189:in `server_exception'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/request.rb:172:in `rescue in block in post_process'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/request.rb:163:in `block in post_process'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/eventmachine-1.0.0.beta.3/lib/em/deferrable.rb:47:in `call'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/eventmachine-1.0.0.beta.3/lib/em/deferrable.rb:47:in `callback'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/request.rb:162:in `post_process'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/request.rb:189:in `server_exception'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/request.rb:172:in `rescue in block in post_process'
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/request.rb:163:in `block in post_process'

When modifying the Custom Server example in an attempt to close the stream 1 second after the content has been provided, an exception is raised when Goliath tries to close the connection :

class HelloWorld < Goliath::API
  def response(env)
    EM.add_timer(1) do
      env.stream_close
    end
    [200, {}, "hello world!"]
  end
end
~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/connection.rb:80:in `terminate_request': undefined method `close' for nil:NilClass (NoMethodError)
    from ~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/request.rb:28:in `block (2 levels) in initialize'
    from ~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/eventmachine-1.0.0.beta.3/lib/em/deferrable.rb:47:in `call'
    from ~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/eventmachine-1.0.0.beta.3/lib/em/deferrable.rb:47:in `callback'
    from ~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/request.rb:28:in `block in initialize'
    from ~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/env.rb:67:in `call'
    from ~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/env.rb:67:in `stream_close'
    from ~/workspace/goliath-test/bin/test:23:in `block in response'
    from ~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/eventmachine-1.0.0.beta.3/lib/eventmachine.rb:199:in `call'
    from ~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/eventmachine-1.0.0.beta.3/lib/eventmachine.rb:199:in `run_machine'
    from ~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/eventmachine-1.0.0.beta.3/lib/eventmachine.rb:199:in `run'
    from ~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/em-synchrony-0.3.0.beta.1/lib/em-synchrony.rb:26:in `synchrony'
    from ~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/server.rb:72:in `start'
    from ~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/runner.rb:230:in `run_server'
    from ~/.rvm/gems/ruby-1.9.2-p180@gemset/gems/goliath-0.9.1/lib/goliath/runner.rb:164:in `run'
    from ~/workspace/goliath-test/bin/test:65:in `<main>'

In the second case, I've been able to avoid the exception with the following patch, but could not figure out what side effects I could have introduced.

    def terminate_request(keep_alive)
      if req = @pending.shift
        @current = req
        @current.succeed
      elsif !@current.nil?
        @current.close
        @current = nil
      end

What we're trying to do is to setup a piece of server software based on Goliath which could be able to serve both long-polling and regular requests, the reason why I'm trying to close the stream as soon as the content has been delivered to the client.

Thanks

Class-name to config-file-name munging is a bit broken

The current class name munging code that loads config/* files is a bit broken for some reasonable class names:

> 'TLA::MyAPI'.gsub(/(.)([A-Z])/,'\1_\2').downcase!
 => "t_la::_my_ap_i"

Something like this would be more reasonable:

> 'TLA::MyAPI'.gsub('::', '_').gsub(/([^_A-Z])([A-Z])/,'\1_\2').downcase!
 => "tla_my_api"

Stack level too deep when using map and accessing bad path

I'm using the code in master to build an app that allows you to access multiple child apps using the map '/path', Class syntax. I noticed that accessing a path/URL that isn't mapped results in a stack level too deep error. I've put together a simple example of the problem in a gist, but have had little success tracking down what's doing it.

https://gist.github.com/1064943

I might have jumped the gun here, but it took me a while to track down that this was being caused by me typing in the wrong URL!

Async processing broken when using router

https://groups.google.com/d/topic/goliath-io/-DUVYq0d09E/discussion

Repo to reproduce the problem: https://github.com/mando/Test-Async

The problem is that when the connection is established, we check for on_(header | body | close) methods on the API class. But the problem is, when using the router those methods are not defined on the class that's running, rather they are defined on the mapped endpoint.

Code: https://github.com/postrank-labs/goliath/blob/master/lib/goliath/connection.rb#L35-37

To be honest, I'm a bit hesitant of getting the router logic into connection.rb...

Goliath logging is hard-coded

Goliath's logging is hard-coded in request.rb and runner.rb

Goliath should either

  • have a configuration parameter for a flexible log formatting string (there are probably rack tools for this already) or
  • put logging in a middleware in such a way that it is easy to override

Do config/environment parsing earlier?

If you read through: #18 you'll see some things raised around the timing of config and environment parsing in Goliath.

There is currently code in place to parse out -e or --environment early in the process but this leaves everything else.

Goliath has code to allow an API to extend the options parser for custom options but this means that we can't do the full options parse until we've parsed the API class. Which makes things difficult ...

echo.rb require error

Hi guys, I'm trying to work starting from some of your examples, but right now the echo server example is giving me problems, even if it's a cut&paste.
If I start the app the response I have when connecting is error on line 1 at column 1: Document is empty
If I start using -dd this is what I get, it seems a require problem, but honestly I'm not sure:

https://gist.github.com/904842

Of course Goliath is installed:

https://gist.github.com/1c087c82a4f0d5ec010a

The logger key in the env is 'logger', not 'rack.logger'

The rack spec specifies that there should be a 'rack.logger' entry in the env which behaves like a logger object: http://rack.rubyforge.org/doc/files/SPEC.html

Goliath seems to have a similar logger object stored by the 'logger' key: https://github.com/postrank-labs/goliath/blob/master/lib/goliath/constants.rb#L13

I'm guessing that this is so the method_missing to hash lookup trick allows you to call env.logger directly. Switching the key to 'rack.logger' and adding a Env#logger would make Goliath more rack compliant & wouldn't have behavioral changes internal to Goliath.

Errors in middleware leave hanging connection in streaming

If an error is uncaught -- makes its way to Request#server_exception -- it will not properly close a streaming connection.

As an example, here I have a request to an app that uses Goliath::Rack::Params and does not use Goliath::Rack::ValidationError. I pass in some bad JSON (the trailing "," at the end); I have to ^C to kill the request.

url -H "Content-Type: application/json" --data-ascii '{ "url_base":"http://localhost:9002/?delay=", "url_vals":[1.0,2.3,1.3,4.0], }' -v 'http://localhost:9004/batch.json'
* About to connect() to localhost port 9004 (#0)
*   Trying ::1... Connection refused
*   Trying fe80::1... Connection refused
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 9004 (#0)
> POST /batch.json HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
> Host: localhost:9004
> Accept: */*
> Content-Type: application/json
> Content-Length: 129
> 
< HTTP/1.1 500 Internal Server Error
< Server: Goliath
< Date: Mon, 25 Apr 2011 04:15:00 GMT
* no chunk, no close, no size. Assume close to signal end
<
^C

ctrl-c crashes streaming API

If i telnet to a streaming server and ctrl-c the telnet session the server crashes with the following stack trace:

ruby stream.rb -sv
[4148:INFO] 2011-02-16 20:24:43 :: Starting server on 0.0.0.0:9000 in development mode. Watch out for stones.
/home/bob/goliath/goliath/lib/goliath/request.rb:92:in <<': Could not parse data entirely (HTTP::Parser::Error) from /home/bob/goliath/goliath/lib/goliath/request.rb:92:inparse'
from /home/bob/goliath/goliath/lib/goliath/connection.rb:24:in receive_data' from /home/bob/.rvm/gems/ruby-1.9.2-p136/gems/eventmachine-1.0.0.beta.2/lib/eventmachine.rb:197:inrun_machine'
from /home/bob/.rvm/gems/ruby-1.9.2-p136/gems/eventmachine-1.0.0.beta.2/lib/eventmachine.rb:197:in run' from /home/bob/.rvm/gems/ruby-1.9.2-p136/gems/em-synchrony-0.2.0/lib/em-synchrony.rb:26:insynchrony'
from /home/bob/goliath/goliath/lib/goliath/server.rb:22:in start' from /home/bob/goliath/goliath/lib/goliath/runner.rb:91:inrun_server'
from /home/hylo/goliath/goliath/lib/goliath/runner.rb:77:in run' from /home/hylo/goliath/goliath/lib/goliath/application.rb:99:inrun!'
from /home/hylo/goliath/goliath/lib/goliath/application.rb:105:in `block in module:Goliath'

rack shell environment

racksh will create a rails console like environment from a config.ru. But goliath doesn't use config.ru. Is there a way to create a console environment for goliath? Is there a way to translate between a goliath config and config.ru ?

No binary for Goliath

There is currently no binary to run Goliath like other servers. As Goliath is Rack server, we probably would like to be able to run rack files like:

goliath -R config.ru start

headers not sent in streaming responses

When returning a streaming response:

return [200, { 'Access-Control-Allow-Origin' => '*' }, Goliath::Response::STREAMING]

The headers specified are not sent back to the client. Here's verbose output from curl:

> GET /goliath-test HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
> Host: localhost:9000
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: Goliath
< Date: Sat, 16 Apr 2011 22:22:35 GMT
* no chunk, no close, no size. Assume close to signal end
< 
RESPONSE_CONTENT_HERE

But headers are returned as expected if the response is not streaming:
return [200, { 'Access-Control-Allow-Origin' => '*' }, 'RESPONSE_CONTENT_HERE']

Output:

> GET /goliath-test HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
> Host: localhost:9000
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=utf-8
< Server: PostRank Goliath API Server
< Vary: Accept
< Access-Control-Allow-Origin: *
< Content-Length: 22643
< Date: Sat, 16 Apr 2011 22:22:25 GMT
< 
RESPONSE_CONTENT_HERE

Templates are loaded from gem path by default?

class API < Goliath::API
  include Goliath::Rack::Templates

  def response(env)
     # ...
     [200, {}, erb(:hello)]
  end
end

I expected that to load the hello.erb file from views directory (same behavior as sinatra). Hitting the API with a request, shows:

{"error":"Template stream not found in /Users/igrigorik/.rvm/gems/ruby-1.9.2-p136/gems/goliath-0.9.1/lib/goliath/rack/views for erb"}%

It looks like I need to use Rack::Static before the template gets loaded from right place?

use(Rack::Static,:root => Goliath::Application.root_path("public"), :urls => ["/favicon.ico", '/stylesheets', '/javascripts', '/images'])

Seems like another gotcha, similar to ValidationError stuff which just got fixed... I would expect that the template should be loaded relative to my location by default and sourced from views.

Double Resume errors in high concurrency with EM:HttpRequest

If you throw a large (> 100) number of concurrent requests at Goliath it will start raising double resume errors after some number of them:

[61953:ERROR] 2011-05-02 21:45:14 :: double resume
[61953:ERROR] 2011-05-02 21:45:14 :: (eval):8:in `resume'
(eval):8:in `block in get'
/Users/flip/.rvm/gems/ruby-1.9.2-p136/gems/eventmachine-1.0.0.beta.3/lib/em/deferrable.rb:72:in `call'
/Users/flip/.rvm/gems/ruby-1.9.2-p136/gems/eventmachine-1.0.0.beta.3/lib/em/deferrable.rb:72:in `errback'
(eval):8:in `get'
./app/rack/simple_proxy.rb:16:in `response'
/Users/flip/ics/backend/goliath_fiddle/lib/goliath/api.rb:155:in `block in call'
[61953:INFO] 2011-05-02 21:45:14 :: Status: 400, Content-Length: 25, Response Time: 0.94ms

Now 200 concurrent requests is a little bit ridiculous but I believe it will start to fail sooner with a more complicated script.

I get the error if I run

./sleepy.rb -sv -p 9002 -e prod
./simple_proxy.rb  -sv -p 9000 -e prod
ab -c200 -n200 'http://127.0.0.1:9002'

using this script (it calls the test harness below):

#!/usr/bin/env ruby
require 'goliath'
require 'em-synchrony/em-http'

# $ ./app/rack/sleepy.rb -sv -p 9002 -e prod
# $ ./app/rack/simple_proxy.rb  -sv -p 9000 -e prod
# $ ab -c200 -n200 'http://127.0.0.1:9002'

class SimpleProxy < Goliath::API
  BASE_URL     = 'http://localhost:9002/?delay=1.0'

  def response(env)
    resp = EM::HttpRequest.new(BASE_URL).get

    [200, { }, resp.response]
  end
end
#!/usr/bin/env ruby
require 'goliath'

# Wait the amount of time given by the 'delay' parameter before responding (default 2.5, max 15.0).
class Sleepy < Goliath::API
  use Goliath::Rack::Params
  use Goliath::Rack::Validation::NumericRange, {:key => 'delay', :default => 2.5, :max => 15.0, :min => 0.0, :as => Float}

  def response(env)
    EM::Synchrony.sleep(env.params['delay'])
    [ 200,
      { 'X-Sleepy-Delay' => env.params['delay'].to_s },
      [ env[:start_time], env.params['delay'], (Time.now.to_f - env[:start_time].to_f) ].inspect
    ]
  end
end

Goliath doesn't default to an environment until too late

Unless -e is specified on the command line or the RACK_ENV environment variable is used, the Goliath code doesn't default to running in development mode until after command line parsing is done, long after application code would be loaded (thus meaning it is impossible to conditionally define things dependent on calling Goliath.env). Should the code at https://github.com/postrank-labs/goliath/blob/master/lib/goliath/application.rb#L10 have an else branch that defaults to :development?

This has the downside that the checking for -e there is already a little weak (since you could do -sve production and have that conditional fail but have the command line parsing code set production later), and thus making it less obvious if your code is going to have issues due to different environments set at load versus runtime.

"no id given" on missing var

method missing capture not doing its job right.. instead of reporting error, complains that not present in env

Params middleware doesn't parse POST body params?

I tried using the Params middleware to parse some params but could only get query strings to work.

Gist:
https://gist.github.com/883270

Am I doing it wrong?

I noticed in the code that the post body should end up in env[rack.input] but that has been empty on my attempts to POST.

Also check out the 2 second overhead when doing multipart posts. That seemed a bit crazy too.

RequiredParam results in 500

I am using RequiredParam

use Goliath::Rack::Validation::RequiredParam, key: param

When I run my specs or try a url in development mode, I get a 500 error. Internally, the middleware is raising a Goliath 400

raise Goliath::Validation::Error.new(400, "#{@type} identifier missing")

It seems like the exception is getting translated to a 500 instead of a 400. Also, a 400 is supposed to indicate bad syntax. I think a
422 Unprocessable Entity is more appropriate, which indicates:

The request was well-formed but was unable to be followed due to semantic errors

Server crash on unicode query

The following app crashes when queried with: curl "http://localhost:9000/video?test_cyr=привет"

#!/usr/bin/env ruby
# encoding: utf-8

$: << "../lib" << "./lib"
require 'goliath'
require 'yajl'

class Video < Goliath::API
  use Goliath::Rack::Params

  def response(env)
    [ 200, { 'Content-Type' => 'application/json' }, Yajl::Encoder.encode(env.params) ]
  end 
end

with_api does not parse or initialize the options parser

Trying to initialize some default values to be used in the API from within options_parse, but with_api does not call options_parser on the specified class when it runs the test, and hence I don't have access to the defaults within the test.

Dies trying production mode

I'm probably doing something silly, but when I run the app with -e prod

[39865:INFO] 2011-03-14 22:12:53 :: Starting server on 0.0.0.0:9000 in production mode. Watch out for stones.
terminate called after throwing an instance of 'std::runtime_error'
  what():  setuid_string failed: no setuid
Abort trap

Ruby version:

$ ruby -v
ruby 1.9.2p180 (2011-02-18 revision 30909) [x86_64-darwin10.6.0]

Gems

activesupport (3.0.3)
addressable (2.2.4)
async-rack (0.5.1)
builder (3.0.0)
bundler (1.0.10)
em-http-request (1.0.0.beta.3)
em-socksify (0.1.0)
em-synchrony (0.3.0.beta.1)
eventmachine (1.0.0.beta.3)
goliath (0.9.0)
http_parser.rb (0.5.1)
i18n (0.5.0)
log4r (1.1.9)
multi_json (0.0.5)
nokogiri (1.4.4)
rack (1.2.2)
rack-accept-media-types (0.9)
rack-contrib (1.1.0)
rack-respond_to (0.9.8)
rake (0.8.7)
rdoc (3.5.3)
yajl-ruby (0.8.1)

App in question: https://github.com/darkhelmet/confreaks-rss

RESTful (GET and POST) routes

Asked on the group mail list:
http://groups.google.com/group/goliath-io/browse_thread/thread/c1de2c3436cb87ef/99334bf46f515e59?lnk=gst&q=RESTful#99334bf46f515e59

One possibility is this gist for trying to integrate with a rack router: https://gist.github.com/891804
But it isn't clear how to use this when there a multiple routes, particularly when there is a need to include different middleware for different routes.

Currently I look at the REQUEST_METHOD, but I lose the ability to have different middleware configurations for different methods.

Decouple Goliath, the EM/Fiber-aware Rack add-on, and Goliath, the API building tool

I think the real underlying issue in #61 and in #18 is the tight coupling between Goliath, the Fiber-aware Rack add-on, and Goliath, the API building tool.

When I initially ran into Goliath, I thought it was some kind of drop-in add-on for Rack, that allows to use Event Machine using Fibers and continue to write code (mostly) as if everything was single threaded. I liked parts but not all of Sinatra, and strongly disliked Rails. So I figured, hey, maybe I can cook up some quick and dirty plugin-able toolkit on top of this.

The issue I'm running into, is that Goliath wants to be the toolkit itself, rather just the server. I can live with that, but configuring it to do anything more complex than a simple API service that starts on the spot feels like trying to manipulate dark magic.

Now, I'm certainly new to ruby, so I may have missed a number of things, and I probably spent more time than I should have needed to understand what was going on in there. But among the issues I've run into:

  • Whichever file gets loaded in /config gets loaded by a Goliath::Server instance, and far too late at that. I was expecting it to load for MyApp::API < Goliath::API. This should at least be made clearer in the docs, because as things stand it was immediate to me, what it was there for.
  • There seems to be no way to configure it except using command line arguments. I noted options_parser(opts, options), but that's precisely my point: it assumes ARGV gets used rather than a config file. A yaml or rb file would have been sweet.

Considering the above it's unclear to me wether or how this is even supposed to run multiple APIs together in a single app. I made good note of the RackRoutes example, but if each of the APIs requires its own configuration, then command-line arguments aren't an appropriate option; and nor is the /config file if any amount of dependency injection is needed.

Anyway, I'm beginning to realize that I spent far too much time trying to bend the app I've in mind so that it make use of Goliath::API as suggested in the docs. The closer I look, the more convinced I am that it the other classes I'm interested in, and that I should be creating some kind of Application/API classes for my own use rather than use the Goliath implementation of the latter two. The goliath code seems highly coupled, however.

So question: are there any chances of splitting the gem in two and introducing some kind of goliath-server dependency for goliath itself?

goliath should return empty body

[201, {}, nil] results in a body whose content-length is set to 4, since nil gets translated to "null" if using JSON response. nil, should instead result in an empty body.

SSL Support

Add command line option to have server run in SSL mode.

hello world is not happy

Not the nicest experience out of the gate.. we should clean that up (make it work):

https://gist.github.com/2b487b94570ea362b3dd

[86245:INFO] 2011-02-12 22:27:57 :: Starting server on 0.0.0.0:9000 in development mode. Watch out for stones.
[86245:ERROR] 2011-02-12 22:28:04 :: wrong number of arguments (3 for 1)
[86245:ERROR] 2011-02-12 22:28:04 :: /Users/igrigorik/.rvm/gems/ruby-1.9.2-p136@global/gems/async-rack-0.5.0/lib/async_rack/async_callback.rb:125:in `async_callback'

problem with your custom server example

This happens on the first visit of the browser.

→ ruby custom_server.rb -p 8081
/var/www/ruby_apps/EM/7-goliath/goliath/lib/goliath/connection.rb:36:in block (2 levels) in post_init': undefined methodset_event_handler!' for nil:NilClass (NoMethodError)
from /var/www/ruby_apps/EM/7-goliath/goliath/lib/goliath/request.rb:74:in parse_header' from /var/www/ruby_apps/EM/7-goliath/goliath/lib/goliath/connection.rb:35:inblock in post_init'
from /var/www/ruby_apps/EM/7-goliath/goliath/lib/goliath/connection.rb:66:in call' from /var/www/ruby_apps/EM/7-goliath/goliath/lib/goliath/connection.rb:66:in<<'
from /var/www/ruby_apps/EM/7-goliath/goliath/lib/goliath/connection.rb:66:in receive_data' from /home/alessio/.rvm/gems/ruby-1.9.2-p290/gems/eventmachine-1.0.0.beta.4/lib/eventmachine.rb:179:inrun_machine'
from /home/alessio/.rvm/gems/ruby-1.9.2-p290/gems/eventmachine-1.0.0.beta.4/lib/eventmachine.rb:179:in run' from /home/alessio/.rvm/gems/ruby-1.9.2-p290/gems/em-synchrony-1.0.0/lib/em-synchrony.rb:27:insynchrony'
from /var/www/ruby_apps/EM/7-goliath/goliath/lib/goliath/server.rb:73:in start' from /var/www/ruby_apps/EM/7-goliath/goliath/lib/goliath/runner.rb:235:inrun_server'
from /var/www/ruby_apps/EM/7-goliath/goliath/lib/goliath/runner.rb:169:in run' from custom_server.rb:56:in

'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.