Git Product home page Git Product logo

ngx_mruby's Introduction

ngx_mruby

ngx_mruby: A Fast and Memory-Efficient Nginx Extension Mechanism Scripting with mruby.

Documents

What's ngx_mruby

ngx_mruby is A Fast and Memory-Efficient TCP/UDP Load Balancing and Web Server Extension Mechanism Using Scripting Language mruby for nginx.

  • ngx_mruby is to provide an alternative to lua-nginx-module or mod_mruby of Apache httpd.
  • Unified Ruby Code between Apache(mod_mruby), nginx(ngx_mruby) and other Web server software(plan) for Web server extensions.
  • You can implement nginx modules by Ruby scripts on nginx!
  • You can implement some Web server software extensions by same Ruby code (as possible)
  • Supported nginx main-line and stable-line
  • Benchmark between ngx_mruby and lua-nginx-module
# location /proxy {
#   mruby_set $backend "/path/to/proxy.rb";
#   proxy_pass   http://$backend;
# }

backends = [
  "test1",
  "test2",
  "test3",
]

r = Redis.new "192.168.12.251", 6379
r.get backends[rand(backends.length)]
  • see examples
  • Sample of Unified Ruby Code between Apache(mod_mruby) and nginx(ngx_mruby) for Web server extensions
  • You can implement some Web server software extensions by same Ruby code (as possible)
# Unified Ruby Code between Apache(mod_mruby) and nginx(ngx_mruby)
# for Web server extensions.
#
# Apache httpd.conf by mod_mruby
#
# <Location /mruby>
#     mrubyHandlerMiddle "/path/to/unified_hello.rb"
# </Location>
#
# nginx nginx.conf by ngx_mruby
#
# location /mruby {
#     mruby_content_handler "/path/to/unified_hello.rb";
# }
#

Server = get_server_class

Server::rputs "Hello #{Server::module_name}/#{Server::module_version} world!"
# mod_mruby => "Hello mod_mruby/0.9.3 world!"
# ngx_mruby => "Hello ngx_mruby/0.0.1 world!"

ngx_mruby mod_mruby performance

hello world simple benchmark, see details of blog entry.

Abstract

As the increase of large-scale and complex Web services, not only the development of Web applications is required, but also the implementation of Web server extensions in many cases. Most Web server extensions are mainly implemented in the C language because of fast and memory-efficient behavior, but by writing extensions using a scripting language we can achieve better maintainability and productivity.

However, if the existing methods are primarily intended to enhance not the implementation of Web applications but the implementation of internal processing of the Web server, the problem remains in terms of speed, memory-efficiency and safety.

Therefore, we propose a fast and memory-efficient Web server extension mechanism using a scripting language. We designed an architecture where the server process creates a region in memory to save the state of the interpreter at the server process startup, and multiple scripts share this region to process the scripts quickly when new request are made.

The server process frees the global variables table, the exception flag and the byte-code which cause an increase of memory usage, in order to reduce the memory usage and extend safety by preventing interference between each script because of sharing the region. We implemented a mechanism that can extend the internal processing of nginx easily by Ruby scripts using nginx and the embeddable scripting language mruby. It's called "ngx_mruby".

Contributions

This project exists thanks to all the people who contribute. We also welcome financial contributions in full transparency on our open collective.

Backers

Thank you to all our backers! 🙏 [Become a backer]

Sponsors

Support this project by becoming a sponsor. Your logo will show up here with a link to your website. [Become a sponsor]

License

This project is under the MIT License:

ngx_mruby's People

Contributors

buty4649 avatar charlescui avatar chaspy avatar cubicdaiya avatar d6rkaiz avatar dearblue avatar hfm avatar hsbt avatar keizo042 avatar kenhys avatar kyswtnb avatar matsumotory avatar matugm avatar mihirat avatar monochromegane avatar nagachika avatar pyama86 avatar rhykw avatar riywo avatar ryotarai avatar sawanoboly avatar spin13 avatar syucream avatar takumakume avatar tk3 avatar vanhecke avatar yano3 avatar ynishi avatar yyamano avatar zogoo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ngx_mruby's Issues

Segfault appears if build nginx with customize openssl path

Platform: CENTOS 7

Build nginx 1.9.11 with following configs:

./configure 
--with-http_v2_module --with-http_ssl_module 
--prefix=/usr/local/nginx --with-http_image_filter_module 
--with-http_gzip_static_module --with-http_secure_link_module 
--with-http_stub_status_module 
--add-module="/usr/local/ruby23/lib/ruby/gems/2.3.0/gems/passenger-5.0.24/src/nginx_module" 
--with-openssl=/usr/src/openssl-1.0.2f  
--add-module=/usr/src/ngx_mruby --add-module=/usr/src/ngx_mruby/dependence/ngx_devel_kit

And after make, the segfault happens. If I remove ngx_mruby from the build config then the segfault will not appear.

I think the reason is ngx_mruby uses system's openssl which is different from Nginx uses.

Is there anyway to solve this issue?

Thanks your great work:)

[SEGV] nginx static-linked OpenSSL and ngx_mruby

Hi.

If nginx static-linked OpenSSL and ngx_mruby are built, it is possible to be caused SEGV.

How to reproduce

It is assumed to be already installed OpenSSL as the package.

$ wget https://www.openssl.org/source/openssl-1.0.2g.tar.gz
$ wget http://nginx.org/download/nginx-1.9.14.tar.gz
$ tar zxvf openssl-1.0.2g.tar.gz
$ tar zxvf nginx-1.9.14.tar.gz
$ git clone https://github.com/matsumoto-r/ngx_mruby.git
$ cd ngx_mruby
$ ./configure --with-ngx-src-root=../nginx-1.9.14
$ make build_mruby
$ make generate_gems_config
$ cd ../nginx-1.9.14
$ ./configure                     \
 --with-openssl=../openssl-1.0.2g \
 --with-http_ssl_module           \
 --add-module=../ngx_mruby
$ make
$ sudo make install
$ sudo /usr/local/nginx/sbin/nginx -g 'daemon off;'
$ echo $? # SEGV
139

Maybe the phenomenon below is similar. (Japanese only, sorry.)

https://twitter.com/d6rkaiz/status/685835026890555392

The patch below is the workaround for this problem. It works fine if it is applied.

diff --git a/build_config.rb b/build_config.rb
index 46d8941..c4d4ced 100644
--- a/build_config.rb
+++ b/build_config.rb
@@ -10,7 +10,7 @@ MRuby::Build.new do |conf|
   conf.gem :github => 'iij/mruby-io'
   conf.gem :github => 'iij/mruby-env'
   conf.gem :github => 'iij/mruby-dir'
-  conf.gem :github => 'iij/mruby-digest'
+  #conf.gem :github => 'iij/mruby-digest'
   conf.gem :github => 'iij/mruby-process'
   conf.gem :github => 'iij/mruby-pack'
   conf.gem :github => 'iij/mruby-socket'

The cause seems to be the OpenSSL dependency for mruby-digest bundled with ngx_mruby by default. mruby-digest requires OpenSSL and it is dynamic-linked when OpenSSL library already exists on system.

On the other hand, nginx provides the mechanism for linking OpenSSL with statically or dynamically.
So if nginx static-linked OpenSSL and ngx_mruby are built, two OpenSSLs seem to conflict.

Thanks.

build from source

Is there a way to build from the source without using git?

I downloaded the source https://github.com/matsumoto-r/ngx_mruby/archive/v1.7.8.tar.gz from the releases page.

It does download nginx, but fails when trying to build mruby. (obviously the submodules were not downloaded)

Any ideas how to download that or build it?

Thanks

About releasing and versioning

The version of ngx_mruby is 0.0.1 for a long time.
As now the features of ngx_mruby is increasing than it used to be,
Why don't we append a version tag and an announce of release on a routine basis or at random times?

For the matter, I want to decide the milestone. For example,

  • v0.0.1
    • current
  • v0.1.0
    • add Nginx.Regex
  • v0.2.0
    • support to access shared memory
    • support hash bindings
    • md5, sha1, crc32, base64, hmac_sha1, etc...

"Nginx::Request#headers_out" empties the assigned array, contrary to expectations

I guess it is not the intended behavior.

Reproduction code is as follows.

# nginx

server {
  listen 127.0.0.1:8888;

  location / {
    mruby_rewrite_handler_code '
      headers = %w(abc=123 foo=bar)
      Nginx::Request.new.headers_out["Set-Cookies"] = headers
      Nginx.errlogger Nginx::LOG_ERR, headers
      Nginx::return headers.empty? ? Nginx::HTTP_INTERNAL_SERVER_ERROR : Nginx::OK
    ';
  }
}

Request to the above server as below.

$ curl http://127.0.0.1:8888 -sv > /dev/null
* Rebuilt URL to: http://127.0.0.1:8888/
* Hostname was NOT found in DNS cache
*   Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.35.0
> Host: 127.0.0.1:8888
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
* Server nginx is not blacklisted
< Server: nginx
< Date: Thu, 04 Feb 2016 11:40:12 GMT
< Content-Type: text/html
< Content-Length: 186
< Connection: close
< Set-Cookies: foo=bar
< Set-Cookies: abc=123
<
{ [data not shown]
* Closing connection 0
root@node1:~#

and log

$ tail -1 /var/log/nginx/error.log
2016/02/04 11:40:12 [error] 11130#11130: *9 [], client: 127.0.0.1, server: , request: "GET / HTTP/1.1", host: "127.0.0.1:8888"

I expected status 200, and the array with two item in the log, but the results are not.

Environment

  • Ubuntu 14.04.3
  • Nginx 1.9.9
  • nginx_mruby 1.15.0

For more details of nginx please refer https://gist.github.com/lamanotrama/638e97202cf54bfd9196

Support multi path per directive

I want to support multi path per directive like the following example:

location /mruby {
  mruby_content_handler_multi /path/to/A.rb /path/to/B.rb /path/to/C.rb
}

First, ngx_mruby run A.rb. Then run B.rb, Finally run C.rb at mruby_content_handler phase.

mruby_output_filter after fastcgi_pass causes error. malloc(18446744073709551615) failed (12: Cannot allocate memory)

fastcgi_passと一緒にmruby_output_filterを使おうとすると、次のエラーでリクエストが終了してしまいました。
通常のlocationおよび、proxy_passでは正常にヘッダが追加されます。

[emerg] 8190#0: *1 malloc(18446744073709551615) failed (12: Cannot allocate memory) while sending to client, client: 127.0.0.1, server: localhost, request: "HEAD /info.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.sock", host: "localhost"
[error] 8190#0: *1 failed to read body ngx_http_mruby_body_filter_inline_handler:1763 while sending to client, client: 127.0.0.1, server: localhost, request: "HEAD /info.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.sock", host: "localhost"

Configurations for reproduce

nginx.conf

load_module modules/ngx_http_mruby_module.so;
worker_processes  2;
events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    tcp_nopush     on;
    keepalive_timeout  5;

    upstream phpfpm {
        server unix:/var/run/php-fpm.sock;
    }

    server {
        listen       8080;
        server_name  localhost;

        location / {
            root   html;
            index  index.html index.htm;
        }
    }

    server {
        listen       80;
        server_name  localhost;

        location / {
            root   html;
            index  index.html index.htm;
            proxy_pass   http://127.0.0.1:8080;
            mruby_output_filter_code '
               r = Nginx::Request.new
               r.headers_out["X-MRUBY"] = "YES"
            ';
        }

        location ~ \.php {
            root   html;
            index  index.html index.htm;
            fastcgi_split_path_info ^(.+\.php)(/.+)$;
            fastcgi_pass   phpfpm;
            fastcgi_index  index.php;
            fastcgi_param  SCRIPT_FILENAME  /usr/share/nginx/html$fastcgi_script_name;
            mruby_output_filter_code '
            r = Nginx::Request.new
            r.headers_out["X-MRUBY"] = "YES"
        ';
        }

    }
}

以下がリクエスト例です。

$ curl -I localhost/
HTTP/1.1 200 OK
Server: nginx/1.11.1
Date: Tue, 07 Jun 2016 05:04:05 GMT
Content-Type: text/html
Content-Length: 612
Connection: keep-alive
Last-Modified: Mon, 06 Jun 2016 09:51:09 GMT
ETag: "5755478d-264"
Accept-Ranges: bytes
X-MRUBY: YES

$ curl -I localhost/info.php
curl: (52) Empty reply from server ## エラーログに Cannot allocate memory

Environments

Nginx: 1.9.11
ngx_mruby: 1.17.2 (DSOビルド)

Nginxのコンフィグオプションは こちら です。

Control some C struct in nginx

We'd like to implement some methods to control C struct in nginx by same description of mod_mruby as possbile.

void ap_mruby_request_init(mrb_state *mrb, struct RClass *class_core)
{
    struct RClass *class_request;

    class_request = mrb_define_class_under(mrb, class_core, "Request", mrb->object_class);

    mrb_define_method(mrb, class_request, "run_handler", ap_mrb_run_handler, ARGS_NONE());

    //mrb_define_method(mrb, class_request, "Initialize", ap_mrb_init_request, ARGS_NONE());
    //mrb_define_method(mrb, class_request, "request_rec_json", ap_mrb_get_request_rec_json, ARGS_NONE());
    mrb_define_method(mrb, class_request, "body", ap_mrb_get_request_body, ARGS_NONE());

    mrb_define_method(mrb, class_request, "the_request=", ap_mrb_set_request_the_request, ARGS_ANY());
    mrb_define_method(mrb, class_request, "the_request", ap_mrb_get_request_the_request, ARGS_NONE());
    mrb_define_method(mrb, class_request, "protocol=", ap_mrb_set_request_protocol, ARGS_ANY());
    mrb_define_method(mrb, class_request, "protocol", ap_mrb_get_request_protocol, ARGS_NONE());
    mrb_define_method(mrb, class_request, "vlist_validator=", ap_mrb_set_request_vlist_validator, ARGS_ANY());
    mrb_define_method(mrb, class_request, "vlist_validator", ap_mrb_get_request_vlist_validator, ARGS_NONE());
    mrb_define_method(mrb, class_request, "user=", ap_mrb_set_request_user, ARGS_ANY());
    mrb_define_method(mrb, class_request, "user", ap_mrb_get_request_user, ARGS_NONE());
    mrb_define_method(mrb, class_request, "ap_auth_type=", ap_mrb_set_request_ap_auth_type, ARGS_ANY());
    mrb_define_method(mrb, class_request, "ap_auth_type", ap_mrb_get_request_ap_auth_type, ARGS_NONE());
    mrb_define_method(mrb, class_request, "unparsed_uri=", ap_mrb_set_request_unparsed_uri, ARGS_ANY());
    mrb_define_method(mrb, class_request, "unparsed_uri", ap_mrb_get_request_unparsed_uri, ARGS_NONE());
    mrb_define_method(mrb, class_request, "uri=", ap_mrb_set_request_uri, ARGS_ANY());
    mrb_define_method(mrb, class_request, "uri", ap_mrb_get_request_uri, ARGS_NONE());
    mrb_define_method(mrb, class_request, "filename=", ap_mrb_set_request_filename, ARGS_ANY());
    mrb_define_method(mrb, class_request, "filename", ap_mrb_get_request_filename, ARGS_NONE());
    mrb_define_method(mrb, class_request, "canonical_filename=", ap_mrb_set_request_canonical_filename, ARGS_ANY());
    mrb_define_method(mrb, class_request, "canonical_filename", ap_mrb_get_request_canonical_filename, ARGS_NONE());
    mrb_define_method(mrb, class_request, "path_info=", ap_mrb_set_request_path_info, ARGS_ANY());
    mrb_define_method(mrb, class_request, "path_info", ap_mrb_get_request_path_info, ARGS_NONE());
    mrb_define_method(mrb, class_request, "args=", ap_mrb_set_request_args, ARGS_ANY());
    mrb_define_method(mrb, class_request, "args", ap_mrb_get_request_args, ARGS_NONE());
    mrb_define_method(mrb, class_request, "hostname=", ap_mrb_set_request_hostname, ARGS_ANY());
    mrb_define_method(mrb, class_request, "hostname", ap_mrb_get_request_hostname, ARGS_NONE());
    mrb_define_method(mrb, class_request, "status_line=", ap_mrb_set_request_status_line, ARGS_ANY());
    mrb_define_method(mrb, class_request, "status_line", ap_mrb_get_request_status_line, ARGS_NONE());
    mrb_define_method(mrb, class_request, "method=", ap_mrb_set_request_method, ARGS_ANY());
    mrb_define_method(mrb, class_request, "method", ap_mrb_get_request_method, ARGS_NONE());
    mrb_define_method(mrb, class_request, "range=", ap_mrb_set_request_range, ARGS_ANY());
    mrb_define_method(mrb, class_request, "range", ap_mrb_get_request_range, ARGS_NONE());
    mrb_define_method(mrb, class_request, "content_type=", ap_mrb_set_request_content_type, ARGS_ANY());
    mrb_define_method(mrb, class_request, "content_type", ap_mrb_get_request_content_type, ARGS_NONE());
    mrb_define_method(mrb, class_request, "handler=", ap_mrb_set_request_handler, ARGS_ANY());
    mrb_define_method(mrb, class_request, "handler", ap_mrb_get_request_handler, ARGS_NONE());
    mrb_define_method(mrb, class_request, "content_encoding=", ap_mrb_set_request_content_encoding, ARGS_ANY());
    mrb_define_method(mrb, class_request, "content_encoding", ap_mrb_get_request_content_encoding, ARGS_NONE());
    mrb_define_method(mrb, class_request, "assbackwards", ap_mrb_get_request_assbackwards, ARGS_NONE());
    mrb_define_method(mrb, class_request, "proxyreq=", ap_mrb_set_request_proxyreq, ARGS_ANY());
    mrb_define_method(mrb, class_request, "proxyreq", ap_mrb_get_request_proxyreq, ARGS_NONE());
    mrb_define_method(mrb, class_request, "header_only", ap_mrb_get_request_header_only, ARGS_NONE());
    mrb_define_method(mrb, class_request, "proto_num", ap_mrb_get_request_proto_num, ARGS_NONE());
    mrb_define_method(mrb, class_request, "status=", ap_mrb_set_request_status, ARGS_ANY());
    mrb_define_method(mrb, class_request, "status", ap_mrb_get_request_status, ARGS_NONE());
    mrb_define_method(mrb, class_request, "method_number", ap_mrb_get_request_method_number, ARGS_NONE());
    mrb_define_method(mrb, class_request, "chunked", ap_mrb_get_request_chunked, ARGS_NONE());
    mrb_define_method(mrb, class_request, "read_body", ap_mrb_get_request_read_body, ARGS_NONE());
    mrb_define_method(mrb, class_request, "read_chunked", ap_mrb_get_request_read_chunked, ARGS_NONE());
    mrb_define_method(mrb, class_request, "used_path_info", ap_mrb_get_request_used_path_info, ARGS_NONE());
    mrb_define_method(mrb, class_request, "eos_sent", ap_mrb_get_request_eos_sent, ARGS_NONE());
    mrb_define_method(mrb, class_request, "no_cache", ap_mrb_get_request_no_cache, ARGS_NONE());
    mrb_define_method(mrb, class_request, "no_local_copy", ap_mrb_get_request_no_local_copy, ARGS_NONE());

    struct RClass *class_notes;

    class_notes = mrb_define_class_under(mrb, class_core, "Notes", mrb->object_class);
    mrb_define_method(mrb, class_notes, "[]=", ap_mrb_set_request_notes, ARGS_ANY());
    mrb_define_method(mrb, class_notes, "[]", ap_mrb_get_request_notes, ARGS_ANY());

    struct RClass *class_headers_in;

    class_headers_in = mrb_define_class_under(mrb, class_core, "Headers_in", mrb->object_class);
    mrb_define_method(mrb, class_headers_in, "[]=", ap_mrb_set_request_headers_in, ARGS_ANY());
    mrb_define_method(mrb, class_headers_in, "[]", ap_mrb_get_request_headers_in, ARGS_ANY());
    mrb_define_method(mrb, class_headers_in, "headers_in_hash", ap_mrb_get_request_headers_in_hash, ARGS_ANY());

    struct RClass *class_headers_out;

    class_headers_out = mrb_define_class_under(mrb, class_core, "Headers_out", mrb->object_class);
    mrb_define_method(mrb, class_headers_out, "headers_out=", ap_mrb_set_request_headers_out, ARGS_ANY());
    mrb_define_method(mrb, class_headers_out, "headers_out", ap_mrb_get_request_headers_out, ARGS_ANY());
    mrb_define_method(mrb, class_headers_out, "headers_out_hash", ap_mrb_get_request_headers_out_hash, ARGS_ANY());
}

How to get upstream response body?

When I using proxy_pass to a url(etc. http://github.com),

  • how could I get the response body with mruby?
  • how to send the response with new body content?

I think the code maybe look like this:

# register a callback
Nginx.response.onProxyPassResponsed{
    # replace the body with wrap.
    Nginx.response.body = "var content=#{Nginx.response.body}"
}

Some test cases fail occasionally

I found such test cases like below that occasionally fail. They don’t always fail, but do sometimes.

case1:

$ ONLY_BUILD_NGX_MRUBY=true sh test.sh
2015/08/05 14:29:11 [info] 42733#0: *27 ngx_mruby INFO ngx_mrb_run:806:
mrb_run info: rputs_chain is null and return NGX_OK, client: 127.0.0.1,
server: localhost, request: "GET /nginx_var?name=server_protocol HTTP/1.
0", host: "127.0.0.1"
2015/08/05 14:29:11 [alert] 42733#0: *27 zero size buf in writer t:1 r:0
 f:0 00007FDAB6800020 00007FDAB6800098-00007FDAB6800098 0000000000000000
 140577341440000-0, client: 127.0.0.1, server: localhost, request: "GET
/nginx_var?name=server_protocol HTTP/1.0", host: "127.0.0.1"

(snip)

Fail: ngx_mruby - Nginx::Var [location /nginx_var?name=name]
 - Assertion[2] Failed: Expected to be equal
    Expected: "HTTP/1.0"
      Actual: nil
Total: 35
   OK: 34
   KO: 1
Crash: 0
 Time: 0.023947 seconds

case2:

$ ONLY_BUILD_NGX_MRUBY=true sh test.sh
2015/08/05 15:15:00 [info] 57959#0: *18 ngx_mruby INFO ngx_mrb_run:806:
mrb_run info: rputs_chain is null and return NGX_OK, client: 127.0.0.1,
server: localhost, request: "GET /headers_out_delete HTTP/1.0", host: "1
27.0.0.1"
2015/08/05 15:15:00 [error] 57959#0: *18 ngx_mruby ERROR ngx_mrb_send_he
ader: status code is 200, but response body is empty.Return NGX_HTTP_INT
ERNAL_SERVER_ERROR, client: 127.0.0.1, server: localhost, request: "GET
/headers_out_delete HTTP/1.0", host: "127.0.0.1"

(snip)

Fail: ngx_mruby [location /headers_out_delete]
 - Assertion[50] Failed: 46
    Expected: "foo"
      Actual: nil
Fail: ngx_mruby - Nginx::Var [location /nginx_var?name=name]
 - Assertion[2] Failed: Expected to be equal
    Expected: "HTTP/1.0"
      Actual: nil
Total: 35
   OK: 33
   KO: 2
Crash: 0
 Time: 0.019233 seconds

mruby_set_code with syntax error gives segmentation fault

I get segmentation fault with the following configuration:

  location /foo {
    mruby_set_code $foo 'end';
  }
% ./build/nginx/sbin/nginx
INLINE CODE:1:3: syntax error, unexpected keyword_end
Segmentation fault

Here is a gdb backtrace. I have a patch to fix it and will send a pull request later.

% gdb ./build/nginx/sbin/nginx 
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-80.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/vagrant/ngx_mruby/build/nginx/sbin/nginx...done.
(gdb) run
Starting program: /home/vagrant/ngx_mruby/./build/nginx/sbin/nginx 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
INLINE CODE:1:3: syntax error, unexpected keyword_end

Program received signal SIGSEGV, Segmentation fault.
mrb_irep_incref (mrb=mrb@entry=0xb33830, irep=irep@entry=0x0) at /home/vagrant/ngx_mruby/mruby/src/state.c:125
125       irep->refcnt++;
Missing separate debuginfos, use: debuginfo-install glibc-2.17-106.el7_2.6.x86_64 nss-softokn-freebl-3.16.2.3-13.el7_1.x86_64 pcre-8.32-15.el7_2.1.x86_64 zlib-1.2.7-15.el7.x86_64
(gdb) bt
#0  mrb_irep_incref (mrb=mrb@entry=0xb33830, irep=irep@entry=0x0) at /home/vagrant/ngx_mruby/mruby/src/state.c:125
#1  0x0000000000505dd4 in mrb_proc_new (mrb=mrb@entry=0xb33830, irep=0x0) at /home/vagrant/ngx_mruby/mruby/src/proc.c:32
#2  0x0000000000518797 in mrb_generate_code (mrb=0xb33830, p=p@entry=0xbbc670)
    at /home/vagrant/ngx_mruby/mruby/mrbgems/mruby-compiler/core/codegen.c:2888
#3  0x00000000004ce927 in ngx_http_mruby_shared_state_compile (cf=cf@entry=0x7fffffffe020, state=0xb24628, code=0xbccb60)
    at /home/vagrant/ngx_mruby/src/http/ngx_http_mruby_module.c:923
#4  0x00000000004d0546 in ngx_http_mruby_set_inner (cf=0x7fffffffe020, cmd=<optimized out>, conf=<optimized out>, 
    type=type@entry=NGX_MRB_CODE_TYPE_STRING) at /home/vagrant/ngx_mruby/src/http/ngx_http_mruby_module.c:1688
#5  0x00000000004d0697 in ngx_http_mruby_set_inline (cf=<optimized out>, cmd=<optimized out>, conf=<optimized out>)
    at /home/vagrant/ngx_mruby/src/http/ngx_http_mruby_module.c:1719
#6  0x0000000000465b64 in ngx_conf_handler (last=0, cf=0x7fffffffe020) at src/core/ngx_conf_file.c:427
#7  ngx_conf_parse (cf=cf@entry=0x7fffffffe020, filename=filename@entry=0x0) at src/core/ngx_conf_file.c:283
#8  0x00000000004829af in ngx_http_core_location (cf=0x7fffffffe020, cmd=<optimized out>, dummy=<optimized out>)
    at src/http/ngx_http_core_module.c:3256
#9  0x0000000000465b64 in ngx_conf_handler (last=1, cf=0x7fffffffe020) at src/core/ngx_conf_file.c:427
#10 ngx_conf_parse (cf=cf@entry=0x7fffffffe020, filename=filename@entry=0x0) at src/core/ngx_conf_file.c:283
#11 0x0000000000482c1b in ngx_http_core_server (cf=0x7fffffffe020, cmd=<optimized out>, dummy=<optimized out>)
    at src/http/ngx_http_core_module.c:3028
#12 0x0000000000465b64 in ngx_conf_handler (last=1, cf=0x7fffffffe020) at src/core/ngx_conf_file.c:427
#13 ngx_conf_parse (cf=cf@entry=0x7fffffffe020, filename=filename@entry=0x0) at src/core/ngx_conf_file.c:283
#14 0x000000000047e2f9 in ngx_http_block (cf=0x7fffffffe020, cmd=<optimized out>, conf=<optimized out>)
    at src/http/ngx_http.c:237
#15 0x0000000000465b64 in ngx_conf_handler (last=1, cf=0x7fffffffe020) at src/core/ngx_conf_file.c:427
#16 ngx_conf_parse (cf=cf@entry=0x7fffffffe020, filename=filename@entry=0xb1b568) at src/core/ngx_conf_file.c:283
#17 0x00000000004635da in ngx_init_cycle (old_cycle=old_cycle@entry=0x7fffffffe0c0) at src/core/ngx_cycle.c:274
#18 0x0000000000455932 in main (argc=<optimized out>, argv=<optimized out>) at src/core/nginx.c:276
``

ngx_mruby is tripped in Valgrind check

Hi, latest ngx_mruby is tripped in Valgrind check in following condition.

How to reproduce

location /mruby {
    mruby_content_handler_code 'Nginx.rputs("bokko\n");Nginx.rputs("cubicdaiya");Nginx.return 200;'
}
$ ab -k -n 10000 -c 4 "http://localhost:${port}/mruby"

This phenomenon occurs when Nginx.rputs is called multiple count.(it does not occur when called only once)

Following is the output of Valgrind.

==24676== Syscall param writev(vector[...]) points to unaddressable byte(s)
==24676==    at 0x662460B: writev (writev.c:51)
==24676==    by 0x422CB0: ngx_linux_sendfile_chain (ngx_linux_sendfile_chain.c:294)
==24676==    by 0x432BF3: ngx_http_write_filter (ngx_http_write_filter_module.c:238)
==24676==    by 0x442530: ngx_http_chunked_body_filter (ngx_http_chunked_filter_module.c:110)
==24676==    by 0x447398: ngx_http_gzip_body_filter (ngx_http_gzip_filter_module.c:325)
==24676==    by 0x448132: ngx_http_postpone_filter (ngx_http_postpone_filter_module.c:83)
==24676==    by 0x449E0D: ngx_http_ssi_body_filter (ngx_http_ssi_filter_module.c:396)
==24676==    by 0x44CB29: ngx_http_charset_body_filter (ngx_http_charset_filter_module.c:553)
==24676==    by 0x4658DA: ngx_http_small_light_body_filter (ngx_http_small_light_module.c:221)
==24676==    by 0x409780: ngx_output_chain (ngx_output_chain.c:206)
==24676==    by 0x432FD0: ngx_http_copy_filter (ngx_http_copy_filter_module.c:143)
==24676==    by 0x442888: ngx_http_range_body_filter (ngx_http_range_filter_module.c:587)
==24676==  Address 0x9a507d0 is 0 bytes inside a block of size 55 free'd
==24676==    at 0x4C2A82E: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==24676==    by 0x46FD44: allocf (state.c:38)
==24676==    by 0x49B892: obj_free (gc.c:190)
==24676==    by 0x49B951: incremental_sweep_phase (gc.c:748)
==24676==    by 0x49C312: mrb_incremental_gc (gc.c:804)
==24676==    by 0x49C48C: mrb_obj_alloc (gc.c:334)
==24676==    by 0x4732DA: mrb_str_literal (string.c:112)
==24676==    by 0x46CC69: mrb_run (vm.c:1719)
==24676==    by 0x46948F: ngx_mrb_run (ngx_http_mruby_core.c:68)
==24676==    by 0x468779: ngx_http_mruby_content_handler (ngx_http_mruby_module.c:307)
==24676==    by 0x42A5D0: ngx_http_core_content_phase (ngx_http_core_module.c:1412)
==24676==    by 0x4259C2: ngx_http_core_run_phases (ngx_http_core_module.c:885)

Following patch tentatively fixes this problem. Though it seems like that cause is mrb_str_dup, currently I have no idea about detail.

diff --git a/ngx_http_mruby_core.c b/ngx_http_mruby_core.c
index e5dcb68..ea21c82 100644
--- a/ngx_http_mruby_core.c
+++ b/ngx_http_mruby_core.c
@@ -145,7 +145,6 @@ static mrb_value ngx_mrb_send_header(mrb_state *mrb, mrb_value self)

 static mrb_value ngx_mrb_rputs(mrb_state *mrb, mrb_value self)
 {
-    mrb_value msg;
     mrb_value argv;
     ngx_buf_t *b;
     rputs_chain_list_t *chain;
@@ -169,11 +168,9 @@ static mrb_value ngx_mrb_rputs(mrb_state *mrb, mrb_value self)

     mrb_get_args(mrb, "o", &argv);
     if (mrb_type(argv) != MRB_TT_STRING) {
-        msg = mrb_funcall(mrb, argv, "to_s", 0, NULL);
-    } else {
-        msg = mrb_str_dup(mrb, argv);
+        argv = mrb_funcall(mrb, argv, "to_s", 0, NULL);
     }
-    str                         = (u_char *)RSTRING_PTR(msg);
+    str                         = (u_char *)RSTRING_PTR(argv);
     len                         = ngx_strlen(str);
     (*chain->last)->buf->pos    = str;
     (*chain->last)->buf->last   = str + len;

Performance Comparison

Hello,

This looks great.

http://www.techempower.com/benchmarks/ is a popular framework for comparing functionality across web technologies. There is a benchmark for openresty which would probably be similar to ngx_mruby.

Have you though about implementing these benchmarks?
Does ngx_mruby have the necessary libraries to implement those benchmarks?

Thank you.

Why save many mrb_state in ngx_http_mruby_loc_conf_t?

Why save many mrb_state in ngx_http_mruby_loc_conf_t?
or not save only one mrb_state in global?

One scenario:
Some prepare jobs will run once at nginx start up, such as:$redis = Redis.new.
And in each request, mrb will use $redis.get or $redis.set.

But now, there are many different mrb state save in ngx_http_mruby_loc_conf_t, and the state can't access each other.

@matsumoto-r

ngx_mruby does not return a response when mruby sciprt returns Nginx::NGX_OK

continuation of #30 #31 #35

As #35 is closed, I will open new issue.

As I have already remarked, following mruby script does not work in ngx_mruby.

Nginx.rputs(Time.now.to_s + " hello mruby world for nginx.")
Nginx.return Nginx::NGX_OK

To be precise, this mruby script works but ngx_mruby does not return a response.

@matsumoto-r Nginx.return and Apache.return should return a value as not HTTP status code but module status.

If ngx_mruby takes on the above specification, ngx_mruby should rewrite a return code in ngx_mruby when mruby script returns NGX_OK.

install on osx

I get this error while installing on OSX

    -o objs/addon/ngx_mruby/ngx_http_mruby_module.o \
    /Users/senthil/mruby/ngx_mruby/ngx_http_mruby_module.c

In file included from /Users/senthil/mruby/ngx_mruby/ngx_http_mruby_module.h:14,
from /Users/senthil/mruby/ngx_mruby/ngx_http_mruby_module.c:49:
/Users/senthil/mruby/ngx_mruby/ngx_http_mruby_core.h:11:19: error: mruby.h: No such file or directory
In file included from /Users/senthil/mruby/ngx_mruby/ngx_http_mruby_module.h:14,
from /Users/senthil/mruby/ngx_mruby/ngx_http_mruby_module.c:49:
/Users/senthil/mruby/ngx_mruby/ngx_http_mruby_core.h:19: error: expected specifier-qualifier-list before ‘mrb_state’
/Users/senthil/mruby/ngx_mruby/ngx_http_mruby_core.h:29: error: expected ‘)’ before ‘’ token
In file included from /Users/senthil/mruby/ngx_mruby/ngx_http_mruby_module.h:15,
from /Users/senthil/mruby/ngx_mruby/ngx_http_mruby_module.c:49:
/Users/senthil/mruby/ngx_mruby/ngx_http_mruby_init.h:14: error: expected ‘)’ before ‘
’ token
In file included from /Users/senthil/mruby/ngx_mruby/ngx_http_mruby_module.c:50:
/Users/senthil/mruby/ngx_mruby/ngx_http_mruby_request.h:15: error: expected ‘)’ before ‘’ token
make[1]: *
* [objs/addon/ngx_mruby/ngx_http_mruby_module.o] Error 1
make: *** [build] Error 2

error with build ngx_mruby - "error: incompatible types in return"

I compile ngx_mruby

shell history:

525  make mruby_build
  526  cd mruby/
  527  cat > mrbgems/GEMS.active << EOF
  528  rake ENABLE_GEMS='true'
  529  ll
  530  mgem add mruby-json
  531  mgem config
  532  cat > my_config.rb
  533  export MRUBY_CONFIG=my_config.rb
  534  rake
  535  cd ..
  536  vim README.md 
  537  ./configure --enable-mrbgems --with-ngx-src-root=../nginx-1.2.7 --with-ngx-config-opt="--prefix=/usr/local/nginx.new"
  538  make

output with conpile error:

cc1: warnings being treated as errors
/Users/cuizheng/Documents/ngx_mruby/ngx_http_mruby_core.c: In function ‘ngx_mrb_get_ngx_mruby_version’:
/Users/cuizheng/Documents/ngx_mruby/ngx_http_mruby_core.c:236: warning: implicit declaration of function ‘mrb_str_new2’
/Users/cuizheng/Documents/ngx_mruby/ngx_http_mruby_core.c:236: error: incompatible types in return
/Users/cuizheng/Documents/ngx_mruby/ngx_http_mruby_core.c: In function ‘ngx_mrb_get_nginx_version’:
/Users/cuizheng/Documents/ngx_mruby/ngx_http_mruby_core.c:241: error: incompatible types in return
make[2]: *** [objs/addon/ngx_mruby/ngx_http_mruby_core.o] Error 1
make[1]: *** [build] Error 2
make: *** [ngx_mruby] Error 2

Thanks!

Mapping multiple urls to same ruby files

Hello.

I created a ruby file (World) that I want to use across multiple urls:

class World
  def db
    @db ||= Userdata.new("my_#{Process.pid}").db
  end

  def find(id)
    ret = nil
    db.execute("select * from World where id = ?", id) do |row, fields|
      ret = Hash[fields.zip(row)]
    end
    ret
  end

  def save(world)
    db.execute("update World set randomNumber = ? where id = ?", world["randomNumber"], world["randomNumber"])
  end
end

r = Nginx::Request.new

num_queries = r.var.arg_queries.to_i
num_queries = 1 if num_queries < 0
num_queries = 500 if num_queries > 500

world = World.new
ret = num_queries.times.map { world.find(rand(10000)) }
Nginx.rputs JSON::stringify(ret)

Would you suggest mapping all the urls to a single file using something like https://github.com/mattn/mruby-sinatic ?

Have you played with frameworks and ngx_mruby?
Does it decrease the performance by too much?

Sure feels like a single url would map ngx_mruby to be like typical ruby and rails/sinatra.

Just looking for your thoughts.
Thanks

Building ngx in more standard way

Hello,

I like how well ngx_mruby did in the benchmark comparisons with other servers. I also feel being the fastest ruby (by far) is very cool, too.

Unfortunately the procedure for building ngx_mruby is a little intimidating for me. ( setup )
I need to add merb for templating and possibly a database connection pool, but feel the current way I built it is a little hacky.

Do you have comments for the following improvements?

  1. I can build mruby with rvm/ruby-install/ruby-build. Can I just link that while building ngx_mruby using something like -I/home/mruby/lib/mruby.a -lmruby? Do we really need to download mruby inside of ngx_mruby source?
  2. Do I need to rebuild mruby for every gem, or can I just add the gem in a similar way mentioned above -lmruby-merb?
  3. Can I build ngx using their module syntax like cd ngx ; ./configure --add-module=$HOME/ngx_mruby See install notes

I'm very happy to do the work, just hope you would have pointers to what is possible, and what path you would like me to take.

Thank you,
Keenan

make build_mruby fails due missing hiredis

Building ngx_mruby on CentOS7 I got fatal error: hiredis/hiredis.h error

To fix that just install hiredis-devel with sudo yum install hiredis-devel

(I'm writing this just as reference for the future)

please include mymruby_with_mrbgems.sh

Hello,

There is a mruby_build target in the Makefile. It references a script that is not present.

Could you either include the script it references in the project, or give a description or blog post url for what needs to be done for this step?

(Working through the readme, trying to figure out what is missing.)

Thanks,
Keenan

compile error

Hi, latest ngx_mruby occurs the following compile error.

$ sh mymruby_with_mrbgems.sh
$ cd ../nginx-1.3.8
$ ./configure --with-pcre --add-module=../ngx_mruby
$ make
                   ・
                   ・
                   ・
../ngx_mruby/mruby/lib/libmruby.a(init.o): In function `mrb_init_core':
/home/bokko/workspace/nginx/ngx_mruby/mruby/src/init.c:70: undefined reference to `mrb_init_mrbgems'

In my environment, the following patch fixes this problem.

diff --git a/config b/config
index 2ede593..355cf8d 100644
--- a/config
+++ b/config
@@ -10,7 +10,7 @@ NGX_ADDON_SRCS="$NGX_ADDON_SRCS \
                 $ngx_addon_dir/ngx_http_mruby.c
                 "

-CORE_LIBS="$CORE_LIBS $mruby_root/mrblib/mrblib.o $mruby_root/lib/libmruby.a -lm"
+CORE_LIBS="$CORE_LIBS $mruby_root/mrblib/mrblib.o $mruby_root/lib/libmruby.a $mruby_root/mrbgems/gem_init.a $mruby_root/mrbgems/g/mruby-json/mrb-mruby-json-gem.a -lm"
 CORE_INCS="$CORE_INCS $mruby_root/src $mruby_root/include"

 source $ngx_addon_dir/mrbgems_config

But it is not the best solution. Though I think the cause is that '$ngx_addon_dir/mrbgems_config' does not exist in ngx_mruby's repository, what do you think?

Proposal for Rack based API change for error pages

I'd like to use Rack based API for an error page as below.

location /api {
  error_page 401 @error;
  mruby_access_handler /opt/nginx/lib/mruby/auth.rb;
  proxy_pass http://localhost:4567/hello;
}
location @error {
  mruby_content_handler /opt/nginx/lib/mruby/error.rb;
}
class ErrorResponse
  def call(env)
    return [401, [], ["Unauthorized"]]
  end
end
run ErrorResponse.new

It doesn't work as I expected. I get nginx's default error page.

The current implementation calls Nginx.return with a status returned by "call" method.
https://github.com/matsumoto-r/ngx_mruby/blob/master/mrbgems/rack-based-api/mrblib/rack.rb#L90
In my observation, the call causes special response handling for 401 again.

So I’d like to propose to change rack.rb as follows. With this change, 'return [nil, [], ["Unauthorized"]]' works well to me.

diff --git a/mrbgems/rack-based-api/mrblib/rack.rb b/mrbgems/rack-based-api/mrblib/rack.rb
index 3ec7c01..57498b4 100644
--- a/mrbgems/rack-based-api/mrblib/rack.rb
+++ b/mrbgems/rack-based-api/mrblib/rack.rb
@@ -75,6 +75,8 @@ module Kernel
     end

     def self.build_response r, res
+      return if res.nil?
+
       if res[1].kind_of?(Hash)
         res[1].keys.each { |k| r.headers_out[k] = res[1][k] }
       elsif res[1].kind_of?(Array)
@@ -87,7 +89,7 @@ module Kernel
       else
         raise TypeError, "response body arg type must be Array"
       end
-      Server.return res[0].to_i
+      Server.return res[0].to_i if res[0]
     end
   end

Improve performance and memory consumption

Hi, now I implemeted patch for improvement of performance(8c2147e) and memory consumption(a832892).

  • nginx.conf
        location /mruby {
            mruby_content_handler /usr/local/nginx/html/hello.mrb;
        }
  • hello.mrb
Nginx.rputs("hello mruby world for nginx.")
Nginx.return Nginx::NGX_OK

before

Performance

$ ab -c 100 -n 10000 -k http://127.0.0.1:8000/mruby

Requests per second: 1987.16 [#/sec] (mean)

Memory

  • startup
nobody     917  0.0  0.0   5200  1136 ?        S    12:52   0:00  \_ nginx: worker process
nobody     918  0.0  0.0   5200  1160 ?        S    12:52   0:00  \_ nginx: worker process
nobody     919  0.0  0.0   5200  1160 ?        S    12:52   0:00  \_ nginx: worker process
nobody     920  0.0  0.0   5200  1140 ?        S    12:52   0:00  \_ nginx: worker process
  • after above ab bench
nobody     901  6.9  6.5 547168 543596 ?       S    12:50   0:05  \_ nginx: worker process
nobody     902  2.7  2.6 219620 216124 ?       S    12:50   0:02  \_ nginx: worker process
nobody     903  2.0  1.9 162136 158572 ?       S    12:50   0:01  \_ nginx: worker process
nobody     904  1.5  1.4 124964 121464 ?       S    12:50   0:01  \_ nginx: worker process

after

applied patch of improvement of performance(8c2147e) and memory consumption(a832892)

Performance

$ ab -c 100 -n 10000 -k http://127.0.0.1:8000/mruby

Requests per second: 19386.49 [#/sec] (mean)

Memory

  • startup
nobody    4021  0.0  0.0   5200  1136 ?        S    13:01   0:00  \_ nginx: worker process
nobody    4022  0.0  0.0   5200  1136 ?        S    13:01   0:00  \_ nginx: worker process
nobody    4023  0.0  0.0   5200  1136 ?        S    13:01   0:00  \_ nginx: worker process
nobody    4024  0.0  0.0   5200  1112 ?        S    13:01   0:00  \_ nginx: worker process
  • after above ab bench
nobody    3993  0.5  0.0   5484  1904 ?        S    12:59   0:00  \_ nginx: worker process
nobody    3994  0.0  0.0   5200  1160 ?        S    12:59   0:00  \_ nginx: worker process
nobody    3995  0.2  0.0   5304  1716 ?        S    12:59   0:00  \_ nginx: worker process
nobody    3996  0.0  0.0   5200  1140 ?        S    12:59   0:00  \_ nginx: worker process

Best regards,

nginx worker thread caught Segfault at startup with openssl-1.0.1e

nginx build options

./configure \
--prefix=/usr/local/nginx-1.4.1 \
--with-openssl=/usr/local/src/openssl-1.0.1e \
--with-http_ssl_module \
--with-http_spdy_module \
--with-http_realip_module \
--with-http_gunzip_module \
--with-http_gzip_static_module \
--add-module=../ngx_pagespeed-alpha-r2748 \
--add-module=/home/matsumoto_r/DEV/ngx_mruby \
--add-module=/home/matsumoto_r/DEV/ngx_mruby/dependence/ngx_devel_kit \

nginx worker try to start as multi thread for ssl. But, nginx worker thread caught Segfault at startup.

$ sudo gdb objs/nginx
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/local/src/nginx-1.4.1/objs/nginx...done.
(gdb) r
Starting program: /usr/local/src/nginx-1.4.1/objs/nginx 
[Thread debugging using libthread_db enabled]

Program received signal SIGSEGV, Segmentation fault.
0x0000003c93727a96 in __strcmp_sse42 () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.107.el6.x86_64 hiredis-0.10.1-3.el6.x86_64 libgcc-4.4.7-3.el6.x86_64 libstdc++-4.4.7-3.el6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64 openssl-1.0.0-27.el6_4.2.x86_64 pcre-7.8-6.el6.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) where
#0  0x0000003c93727a96 in __strcmp_sse42 () from /lib64/libc.so.6
#1  0x0000003c98eb71da in ?? () from /usr/lib64/libcrypto.so.10
#2  0x0000003c98eb7526 in lh_insert () from /usr/lib64/libcrypto.so.10
#3  0x0000003c98e608fb in OBJ_NAME_add () from /usr/lib64/libcrypto.so.10
#4  0x0000000000b2d745 in SSL_library_init ()
#5  0x000000000043417e in ngx_ssl_init (log=0x10061a0) at src/event/ngx_event_openssl.c:94
#6  0x0000000000414e71 in main (argc=1, argv=0x7fffffffe738) at src/core/nginx.c:287

If removed --with-openssl option, nginx start as multi process and don't catch the segfault.

nginx does not return a resonse when mrb_run failed.

When the following location is accessed, nginx does not return a response.

location /mruby {
    mruby_content_handler_code "
#        Nginx.echo('hoge')
        Mginx.echo('hoge')
    ";
}  

result by curl

$ curl http://127.0.0.1/mruby
curl: (52) Empty reply from server

error message

mrb_run failed. error: INLINE CODE:3: uninitialized constant Mginx (NameError)

In this case, I think nginx should return a specified HTTP status(e.g. 500). What do you think?

undefined method `popen2e' for Open3:Module

getting error while build

build: [exec] git clone git://github.com/redis/hiredis.git
rake aborted!
undefined method `popen2e' for Open3:Module

(See full trace by running task with --trace)
make: *** [build_mruby] Error 1

We should implement non-blocking I/O like neverblock

We should implement non-blocking I/O like neverblock. nginx architecture has non-blocking and async I/O. So, when accessing external db like redis or mysql, this access should be non-blocking. Now, this access is blocking.

my plans are the followings:

  • use mruby-fiber
  • implement as mrbgem (mruby-neverblock?)
  • can write code like neverblock

I'ts NOT easy to implement.

ERROR 324 (net::ERR_EMPTY_RESPONSE)

nginx.conf

    server {
        listen      7788;
        location /mruby {
            mruby_content_handler /Users/cuizheng/Documents/ngx.mrb;
        }   
        access_log  /Users/cuizheng/Documents/access.log;
    }

access.log

127.0.0.1 - - [18/Mar/2013:16:21:42 +0800] "GET /mruby HTTP/1.1" 000 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.155 Safari/537.22"
127.0.0.1 - - [18/Mar/2013:16:21:52 +0800] "-" 400 0 "-" "-"

/Users/cuizheng/Documents/ngx.mrb

Nginx.rputs(Time.now.to_s + "hello mruby world for nginx.")

curl -v

saimatoMacBook-Pro:ngx_mruby cuizheng$ curl -v http://localhost:7788/mruby
* About to connect() to localhost port 7788 (#0)
*   Trying ::1... Connection refused
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 7788 (#0)
> GET /mruby HTTP/1.1
> User-Agent: curl/7.21.4 (universal-apple-darwin11.0) libcurl/7.21.4 OpenSSL/0.9.8r zlib/1.2.5
> Host: localhost:7788
> Accept: */*
> 
* Empty reply from server
* Connection #0 to host localhost left intact
curl: (52) Empty reply from server
* Closing connection #0

why respone body is zero?

When use mruby-redis in ngx_mruby, it will get blocked sometimes.

I am trying to write a little demo with ngx_mruby and redis. But i found there is a problem, I'm not sure it is my system problem or in mruby-redis.

Nginx conf


worker_processes  4;
events {
    use kqueue;
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    mruby_init_code '
      userdata = Userdata.new "redis_data_key"
      userdata.redis = Redis.new "127.0.0.1", 6379
       ';
    keepalive_timeout  65;


    server {
        listen       8080;
        server_name  localhost;


        location / {
            root   html;
            index  index.html index.htm;
        }

        location /mruby {
          mruby_content_handler_code '
            if server_name == "NGINX"
              Server = Nginx
            elsif server_name == "Apache"
              Server = Apache
            end
            userdata = Userdata.new "redis_data_key"
            redis = userdata.redis
            result = JSON::stringify({ "custom_no"=>redis.spop("custom_no")})
            #result = JSON::stringify({"custom_no"=>"20160315"})
            Server::rputs result
          ';
        }
        location /mruby2 {
          mruby_content_handler_code '
            if server_name == "NGINX"
              Server = Nginx
            elsif server_name == "Apache"
              Server = Apache
            end
            userdata = Userdata.new "redis_data_key"
            redis = userdata.redis
            #result = JSON::stringify({ "custom_no"=>redis.spop("custom_no")})
            result = JSON::stringify({"custom_no"=>"20160315"})
            Server::rputs result
          ';
        }

    }
}

# Work fine Requests per second:    19073.66 [#/sec] (mean)
ab -k -c100 -n15000 http://127.0.0.1:8080/ 

# Work fine Requests per second:    22617.13 [#/sec] (mean)
ab -k -c100 -n15000 http://127.0.0.1:8080/mruby2 

# get blocked and timeout 
# apr_pollset_poll: The timeout specified has expired (70007)
# Total of 14920 requests completed
ab -k -c100 -n15000 http://127.0.0.1:8080/mruby 

I test several times, it always get blocked, and sometimes it get blocked around 1000 requests.

And according of config i found the problem with mruby-redis spop.

But i test spop in mirb with 50000 times, it work fine.

Then i change spop to lpop, it still get blocked. Hoping to get some help.

My redis-benchmark:

====== SPOP ======
  100000 requests completed in 1.57 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

98.25% <= 1 milliseconds
99.85% <= 2 milliseconds
100.00% <= 2 milliseconds
63653.72 requests per second

My system ulimit

ulimit -n
65536

Trouble compiling nginx

I'm trying to build ngx_ruby in an exact copy of a production environment (centos7 + nginx + passenger + some external modules) but I'm getting troubles compiling it.

(all previous steps of installation went without problems)

I'm adding ngx_mruby via --add-module= and my complete command is:
sudo /usr/local/bin/passenger-install-nginx-module --auto --prefix=/opt/nginx --languages=ruby --nginx-source-dir=/tmp/nginx-1.6.1 --extra-configure-flags="--add-module=/tmp/headers-more-nginx-module --add-module=/tmp/ngx_pagespeed-1.9.32.1-beta --with-http_stub_status_module --without-http_autoindex_module --without-http_ssi_module --without-http_browser_module --without-http_empty_gif_module --without-http_fastcgi_module --without-http_limit_req_module --without-http_limit_conn_module --without-http_memcached_module --without-http_proxy_module --without-http_referer_module --without-http_scgi_module --without-http_split_clients_module --without-http_upstream_ip_hash_module --without-http_userid_module --without-http_uwsgi_module --with-http_gzip_static_module --with-http_ssl_module --with-http_spdy_module --add-module=/home/vagrant/ngx_mruby --add-module=/home/vagrant/ngx_mruby/dependence/ngx_devel_kit"

The error I'm getting is: http://pastie.org/10122799 (I'm sorry but I'm not able to understand the error)

Thank you very much

Implement proxy

the same description by mod_mruby, as possible.

backends = [
    "http://192.168.0.101:8888/",
    "http://192.168.0.102:8888/",
    "http://192.168.0.103:8888/",
    "http://192.168.0.104:8888/",
]

# この辺に色々条件を入れたり、backends配列から取り出すルールを別途定義したりするとmod_mrubyのうまみがでる?

r = Apache::Request.new()

r.handler  = "proxy-server"
r.proxyreq = Apache::PROXYREQ_REVERSE
r.filename = "proxy:" + backends[rand(backends.length)] + r.uri

Apache::return(Apache::OK)

or, lua + redis example
refs: http://d.hatena.ne.jp/hiboma/20120205/1328448746

http {

    server {
        listen       8888;
        server_name  localhost;

        location / {
            set $upstream "";
            rewrite_by_lua '
               local res = ngx.location.capture("/redis")
               if res.status == ngx.HTTP_OK then
                  ngx.var.upstream  = res.body
               else
                  ngx.exit(ngx.HTTP_FORBIDDEN)
               end
            ';
            proxy_pass http://$upstream;
        }

        # HostヘッダをキーにしてRedisに問い合わせ
        location /redis {
             internal;
             set            $redis_key $host;
             redis_pass     127.0.0.1:6379;
             default_type   text/html;
        }
   }
}

use return error in ngx_mruby code and worker process hangs up

Use return in mruby_rewrite_handler_code, first request is OK and response is correct.
But second request is not response, looks like the nginx work process hangs up, block on some code.I got nothing in GDB backtrace, GDB look ok.

        location /test/return {
            mruby_rewrite_handler_code '
                r = Nginx::Request.new()
                if r.var.arg_flag
                    Nginx.rputs "user_agent => #{r.var.arg_flag}<br>"
                    Nginx.return Nginx::NGX_HTTP_OK
                    p "Will return"
                    return
                end
                p "NO TOUCH"
                Nginx.rputs "END OF FILE"
                Nginx.return Nginx::NGX_HTTP_OK
            ';
        }

GDB Info

Starting program: /usr/local/nginx.new/sbin/nginx 
Reading symbols for shared libraries ++++++................................................ done
Reading symbols for shared libraries . done
Reading symbols for shared libraries ........................ done
"Will return"

Will ngx_mruby forbid use return in code?

Nginx::Upstream memory leak?

wikiで書かれているサンプルの設定に対しabをかけたのですが、リクエスト毎にメモリ消費量が順調に増えていっています。

[root@instance-1 nginx]# ps alx | head -1 && ps alx | grep nginx|grep work
F   UID   PID  PPID PRI  NI    VSZ   RSS WCHAN  STAT TTY        TIME COMMAND
5    99 13388 13387  20   0  62892 27568 ep_pol S    ?          0:00 nginx: worker process
[root@instance-1 nginx]# ps alx | head -1 && ps alx | grep nginx|grep work
F   UID   PID  PPID PRI  NI    VSZ   RSS WCHAN  STAT TTY        TIME COMMAND
5    99 13388 13387  20   0  68012 32768 ep_pol S    ?          0:05 nginx: worker process
[root@instance-1 nginx]# ps alx | head -1 && ps alx | grep nginx|grep work
F   UID   PID  PPID PRI  NI    VSZ   RSS WCHAN  STAT TTY        TIME COMMAND
5    99 13388 13387  20   0  73144 37948 ep_pol S    ?          0:11 nginx: worker process

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.