Git Product home page Git Product logo

lua-resty-radixtree's Introduction

Name

Radix tree implementation based on rax.

Status

Build Status License

Dependencies:

Used by:

Developed by API7.ai.

Note

API7.ai provides technical support for the software it maintains like this library and Apache APISIX. Please contact us to learn more.

Table of Contents

Synopsis

location / {
  set $arg_access 'admin';
  content_by_lua_block {
    local radix = require("resty.radixtree")
    local rx = radix.new({
        {
            paths = { "/login/*action" },
            metadata = { "metadata /login/action" },
            methods = { "GET", "POST", "PUT" },
            remote_addrs = { "127.0.0.1", "192.168.0.0/16", "::1", "fe80::/32" }
        },
        {
            paths = { "/user/:name" },
            metadata = { "metadata /user/name" },
            methods = { "GET" },
        },
        {
            paths = { "/admin/:name", "/superuser/:name" },
            metadata = { "metadata /admin/name" },
            methods = { "GET", "POST", "PUT" },
            filter_fun = function(vars, opts)
                return vars["arg_access"] == "admin"
            end
        }
    })

    local opts = {
        method = "POST",
        remote_addr = "127.0.0.1",
        matched = {}
    }

    -- matches the first route
    ngx.say(rx:match("/login/update", opts))   -- metadata /login/action
    ngx.say("action: ", opts.matched.action)   -- action: update

    ngx.say(rx:match("/login/register", opts)) -- metadata /login/action
    ngx.say("action: ", opts.matched.action)   -- action: register

    local opts = {
        method = "GET",
        matched = {}
    }

    -- matches the second route
    ngx.say(rx:match("/user/john", opts)) -- metadata /user/name
    ngx.say("name: ", opts.matched.name)  -- name: john

    local opts = {
        method = "POST",
        vars = ngx.var,
        matched = {}
    }

    -- matches the third route
    ngx.say(rx:match("/admin/jane", opts))     -- metadata /admin/name
    ngx.say("admin name: ", opts.matched.name) -- admin name: jane
    }
}

Back to TOC

Methods

new

Creates a new radix tree to store routes.

Usage

rx, err = radix.new(routes, opts)

Attributes

routes is an array ({ {...}, {...}, {...} }) where each element is a route.

Each route can have the following attributes:

Name Required? Description Example
paths Required List of request paths to match the route. By default does a full match. Adding * at the end will result in prefix match. For example, /foo* can match requests with paths /foo/bar and /foo/car/far. {"/", "/foo", "/bar/*"}
hosts Optional List of host addresses to match the route. Supports wildcards. For example *.bar.com can match foo.bar.com and car.bar.com. {"foo.com", "*.bar.com"}
remote_addrs Optional List of remote addresses (IPv4 or IPv6) to match the route. Supports CIDR format. {"127.0.0.1", "192.0.0.0/8", "::1", "fe80::/32"}
methods Optional List of HTTP methods to match the route. Valid values: "GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS", "CONNECT" and "TRACE". {"GET", "POST"}
vars Optional DSL to evaluate with the provided opts.vars or ngx.var. See: lua-resty-expr. {{"arg_name", "==", "json"}, {"arg_age", ">", 18}}
filter_fun Optional User defined filter function to match the route. Can be used for custom matching scenarios. vars and opts will be passed to the function when matching a route. function(vars) return vars["arg_name"] == "json" end
priority Optional Route priority. Defaults to 0. priority = 100
metadata Optional metadata will be returned when a route matches while using rx:match.
handler Optional handler function will be called when a route matches while using rx:dispatch.

opts is an optional configuration that controls the behavior of a match. It can have the following attribute:

Name Description Default
no_param_match Disables Parameters in path. false

Back to TOC

match

Matches client request with routes and returns metadata if successful.

Usage

metadata = rx:match(path, opts)

Attributes

path is the client request path. For example, "/foo/bar", /user/john/send.

opts is an optional attribute and a table. It can have the following attributes:

Name Required? Description
method Optional HTTP method of the client request.
host Optional Host address of the client request.
remote_addr Optional Remote address (IPv4 or IPv6) of the client. Supports CIDR format.
paths Optional A list of client request paths.
vars Optional A table to fetch variables. Defaults to ngx.var to fetch built-in Nginx variables.

Back to TOC

dispatch

Matches client requests with routes and calls the handler function if successful.

Usage

ok = rx:dispatch(path, opts, ...)

Attributes

path is the client request path. For example, "/api/metrics", /admin/john/login.

opts is an optional attribute and a table. It can have the following attributes:

Name Required? Description
method Optional HTTP method of the client request.
host Optional Host address of the client request.
remote_addr Optional Remote address (IPv4 or IPv6) of the client. Supports CIDR format.
paths Optional A list of client request paths.
vars Optional A table to fetch variables. Defaults to ngx.var to fetch built-in Nginx variables.

Back to TOC

Examples

Full Path Match

Matching full paths with multiple paths specified:

local rx = radix.new({
    {
        paths = {"/foo", "/bar/car", "/doo/soo/index.html"},
        metadata = "metadata /foo",
    },
    {
        paths = {"/example"},
        metadata = "metadata /example",
    },
    {
        paths = {"/index.html"},
        metadata = "metadata /index.html",
    },
})

Prefix Match

Matching based on prefix with multiple paths specified:

local rx = radix.new({
    {
        paths = {"/foo/*", "/bar/car/*"}, -- matches with `/foo/boo`, `/bar/car/sar/far`, etc.
        metadata = "metadata /foo",
    },
    {
        paths = {"/example/*"}, -- matches with `/example/boo`, `/example/car/sar/far`, etc.
        metadata = "metadata /example",
    },
})

Parameters in Path

You can specify parameters on a path. These can then be dynamically obtained from opts.matched.parameter-name:

local rx = radix.new({
    {
        -- matches with `/user/john` but not `/user/` or `/user`
        paths = {"/user/:user"}, -- for `/user/john`, `opts.matched.user` will be `john`
        metadata = "metadata /user",
    },
    {
        -- But this will match `/user/john/` and also `/user/john/send`
        paths = {"/user/:user/*action"}, -- for `/user/john/send`, `opts.matched.user` will be `john` and `opts.matched.action` will be `send`
        metadata = "metadata action",
    },
})

Back to TOC

Installation

From LuaRocks

luarocks install lua-resty-radixtree

From Source

make install

Back to TOC

Development

To install dependencies, run:

make deps

Back to TOC

Benchmarks

These are simple benchmarks.

Environment: MacBook Pro (16-inch, 2019), CPU 2.3 GHz Intel Core i9.

To start benchmarking, run:

make
make bench

Results:

resty -I=./lib -I=./deps/share/lua/5.1 benchmark/match-parameter.lua
matched res: 1
route count: 100000
match times: 10000000
time used  : 3.1400001049042 sec
QPS        : 3184713
each time  : 0.31400001049042 ns

resty -I=./lib -I=./deps/share/lua/5.1 benchmark/match-prefix.lua
matched res: 500
route count: 100000
match times: 1000000
time used  : 0.42700004577637 sec
QPS        : 2341920

resty -I=./lib -I=./deps/share/lua/5.1 benchmark/match-static.lua
matched res: 500
route count: 100000
match times: 10000000
time used  : 0.95000004768372 sec
QPS        : 10526315

resty -I=./lib -I=./deps/share/lua/5.1 benchmark/match-hosts.lua
matched res: 500
route count: 1000
match times: 100000
time used  : 0.60199999809265 sec
QPS        : 166112

resty -I=./lib -I=./deps/share/lua/5.1 benchmark/match-wildcard-hosts.lua
matched res: 500
route count: 1000
match times: 50000
time used  : 0.47900009155273 sec
QPS        : 104384

Back to TOC

lua-resty-radixtree's People

Contributors

boekkooi-lengoo avatar chronolaw avatar foolwc avatar gerrard-ynwa avatar hanxi avatar haoyang1994 avatar huylvt avatar johzchen avatar juzhiyuan avatar kingluo avatar lilien1010 avatar linsir avatar membphis avatar moonming avatar moonshining avatar nic-chen avatar pottekkat avatar rainingmaster avatar ranxuxin001 avatar revolyssup avatar shreemaan-abhishek avatar soulbird avatar spacewander avatar starsz avatar theweakgod avatar tzssangglass avatar u5surf avatar xuruidong avatar zaunist avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

lua-resty-radixtree's Issues

bug: got an error when match var and have no uri args

I found a bug, it got an error when match var and have no uri args.
the test case like this.

    location /t {
        content_by_lua_block {
            local radix = require("resty.radixtree")
            local rx = radix.new({
                {
                    paths = "/aa",
                    metadata = "metadata /aa",
                    vars = {
                        {"arg_k", ">", 10},
                    },
                }
            })

            ngx.say(rx:match("/aa", {vars = {}}))
        }
    }

How to delete a node of a radixtree?

Hello everyone,

I've read the lua-resty-radix source code. There's no method to delete a node in a radixtree. Could anyone help me how to delete the node, please?

release new version

the lua-resty-expr had released version v1.1.0, I think we need to release a new version of lua-resty-radixtree.

@Firstsawyou what do you think?

unexpected matching result

/user/xiangnan should route to /user/:user

    local radix = require("resty.radixtree")
    local rx = radix.new({
      {paths = "/user/:user/age/:age", metadata = "/user/:user/age/:age"},
      {paths = "/user/:user", metadata = "/user/:user"}
    })
    ngx.say(rx:match("/user/xiangnan"))
    ngx.say(rx:match("/user/xiangnan/age/26"))
    assert(rx:match("/user/xiangnan") == rx:match("/user/xiangnan/age/26"))

bug: the route doesn't match multi-level subdomains.

local radix = require("resty.radixtree")
local rx = radix.new({
   {
        paths = {"/aa/*"},
        metadata = "metadata /aa",
        hosts = {"*.bar.com"}
    },
    {
        paths = {"/aa/*"},
        metadata = "metadata /aa1",
        hosts = {"*.qq.bar.com"}
    }
})
ngx.say(rx:match("/aa/bb", {host = "1.qq.bar.com"}))

this code will output metadata /aa not metadata /aa1 as we expected

lua-resty-radixtree in docker env

dev-env: OSX
run-env: docker (which image is openresty/openresty:lastest)

when i installed lua-resty-radixtree in OSX, it would make librestyradixtree.dylib,
but the openresty/openresty:lastest image is based on debian OS, it should required librestyradixtree.so which was not make in install step

the error follow below

openresty    | tried above paths but can not load librestyradixtree.so

any idea fix this typo in friendly. just not like exec install command in docker containers.

'Unauthenticated git protocol on port 9418 is no longer supported' while installing via luarocks

I use RUN luarocks install lua-resty-radixtree 2.8.1-0 in the Dockerfile to install lua-resty-radixtree, but today the build process reports following error message and build failed.

Same issue here for the reference:
leafo/gifine#27

 > [27/28] RUN luarocks install lua-resty-radixtree 2.8.1-0:
#31 1.023 Installing https://luarocks.org/lua-resty-radixtree-2.8.1-0.src.rock
#31 1.654 Missing dependencies for lua-resty-radixtree 2.8.1-0:
#31 1.654    lua-resty-expr 1.3.0 (not installed)
#31 1.654
#31 1.654 lua-resty-radixtree 2.8.1-0 depends on lua-resty-ipmatcher (0.6-0 installed)
#31 1.654 lua-resty-radixtree 2.8.1-0 depends on lua-resty-expr 1.3.0 (not installed)
#31 1.654 Installing https://luarocks.org/lua-resty-expr-1.3.0-0.rockspec
#31 1.665 Cloning into 'lua-resty-expr'...
#31 2.254 fatal: remote error:
#31 2.254   The unauthenticated git protocol on port 9418 is no longer supported.
#31 2.254 Please see https://github.blog/2021-09-01-improving-git-protocol-security-github/ for more information.
#31 2.259
#31 2.259 Error: Failed installing dependency: https://luarocks.org/lua-resty-expr-1.3.0-0.rockspec - Failed cloning git repository.
------
failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c luarocks install lua-resty-radixtree 2.8.1-0]: exit code: 1
make: *** [local_build] Error 17

How to match host with wildcard name

I am trying to match only the host with wildcard name, no paths, methods required, how to do this with package, is there any alternative method that can be used.

build in mingw64 environment

当我在mingw环境下编译lua-resty-radixtree时,我发现只要去掉easy_rax.c文件中的以下引用就可以编译通过。

...
#include <netinet/in.h>
#include <arpa/inet.h>
...

那请问引入这两个库的用途是什么呢,除了在 lua-resty-ipmatcher 这里发现有利用ffi动态加载了这两个库的方法,还有其他地方有用到么?

bug: when set multiple route ,match the route not as expected

local radix = require("resty.radixtree")
local rx = radix.new({
   {
        paths = {"/aa/*"},
        metadata = "metadata /aa",
    },
    {
        paths = {"/aa/*"},
        metadata = "metadata /aa1",
        hosts = {"*.bar.com"}
    }
})
ngx.say(rx:match("/aa/bb", {host = "qqq.bar.com"}))

this code will output metadata /aa not metadata /aa1 as we expected

bug: record which matches the current request error

With the following two conditions met:

  • opts.method matched routr.methods
  • opts.matched = {}

the route match will fail.

case references: apache/apisix#1574

bug causes

if match_route_opts(route, opts, args) then
-- log_info("matched route: ", require("cjson").encode(route))
-- log_info("matched path: ", path)
if compare_param(path, route, opts) then

between match_route_opts and compare_param , should check whether it is resuful style url pass parameter, or prefix matching, the prefix matching does not need to be entered compare_param.

request: add doc for `var`

Now there are only test cases, no detailed instructions and usage examples for the nginx parameters supported by var

style: fix the code style

e.g:

bad styles:

if type(r_v) == "table" then
    for _, v in ipairs(r_v) do
        if v == l_v then
             return true
        end
     end
end

 return false

good styles:

if type(r_v) ~= "table" then
    return false
end

for _, v in ipairs(r_v) do
     if v == l_v then
          return true
     end
end

request help: install: 无法获取"librestyradixtree.so" 的文件状态(stat): 没有那个文件或目录

当我更新apisix-dashboard的schema.json时遇到了下面的问题,lua: ./apisix/admin/init.lua:19: module 'resty.radixtree' not found:

When I update theapisix-dashboard schema.json I met the following problems when I was working,lua: ./apisix/admin/init.lua:19: module 'resty.radixtree' not found:

image

于是我安装lua-resty-radixtree,但又遇到了下面的问题:

So I installed lua-resty-radixtree, but I encountered the following problem:

image
image
image

Reporting a vulnerability

Hello!

I hope you are doing well!

We are a security research team. Our tool automatically detected a vulnerability in this repository. We want to disclose it responsibly. GitHub has a feature called Private vulnerability reporting, which enables security research to privately disclose a vulnerability. Unfortunately, it is not enabled for this repository.

Can you enable it, so that we can report it?

Thanks in advance!

PS: you can read about how to enable private vulnerability reporting here: https://docs.github.com/en/code-security/security-advisories/repository-security-advisories/configuring-private-vulnerability-reporting-for-a-repository

run example for openresty

env:
os: CentOS Linux release 7.6.1810 (Core)
nginx version: openresty/1.15.8.1
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC)
built with OpenSSL 1.0.2k-fips 26 Jan 2017
TLS SNI support enabled

nginx.conf :

location / {
root html;
index index.html index.htm;
content_by_lua_block {
local radix = require("resty.radixtree")
local rx = radix.new({
{
path = "/aa",
metadata = "metadata /aa",
host = "foo.com",
method = {"GET", "POST"},
remote_addr = "0.0.0.0",
},
{
path = "/bb*",
metadata = "metadata /bb",
host = {"*.bar.com", "gloo.com"},
method = {"GET", "POST", "PUT"},
remote_addr = "fe80:fe80::/64",
}
})

            ngx.say(rx:match("/aa", {host = ngx.var.host,
                                     method = ngx.req.get_method,
                                     remote_addr = ngx.var.remote_addr}))
        }
    }

error.log
radixtree.lua:338: bad argument #2 to 'band' (number expected, got nil)
stack traceback:
coroutine 0:
[C]: in function 'band'
lualib/resty/radixtree.lua:338: in function 'match_route_opts'
lualib/resty/radixtree.lua:414: in function 'match_route'
lualib/resty/radixtree.lua:474: in function 'match'

test:
curl 127.0.0.1/aa

doc: update the benchmark result

# radixtree 2.2
$ make bench
resty -I=./lib -I=./deps/share/lua/5.1 benchmark/match-parameter.lua
matched res: 1
route count: 100000
match times: 1000000
time used  : 1.0579998493195 sec
QPS        : 945179
each time  : 1.0579998493195 ns

resty -I=./lib -I=./deps/share/lua/5.1 benchmark/match-prefix.lua
matched res: 500
route count: 100000
match times: 1000000
time used  : 0.55899977684021 sec
QPS        : 1788909

resty -I=./lib -I=./deps/share/lua/5.1 benchmark/match-static.lua
matched res: 500
route count: 100000
match times: 1000000
time used  : 0.10500001907349 sec
QPS        : 9523807

image

radixtree free memory fail, nginx worker exit

nginx worker exit while run radixtree test
The error message is as follows
2021/01/26 04:15:41 [alert] 10469#10469: worker process 10470 exited on signal 11

nginx.conf example :

server {
listen 127.0.0.1:80;

  location /test {
        content_by_lua_block {
                local radix = require("resty.radixtree")
                local route_count = 1000 * 100
                local match_times = 1000 * 1000 * 10
                
                local routes = {}
                for i = 1, route_count do
                    routes[i] = {paths = {"/user" .. i .. "/:name"}, metadata = i}
                end
                
                local rx = radix.new(routes)
                ngx.update_time()
                local start_time = ngx.now()
                
                local res
                local path = "/user500/gordon"
                for _ = 1, match_times do
                    res = rx:match(path)
                end
                
                ngx.update_time()
                local used_time = ngx.now() - start_time
                ngx.say("matched res: ", res)
                ngx.say("route count: ", route_count)
                ngx.say("match times: ", match_times)
                ngx.say("time used  : ", used_time, " sec")
                ngx.say("QPS        : ", math.floor(match_times / used_time))
                ngx.say("each time  : ", used_time * 1000 * 1000 / match_times, " ns")
        }
 }

}

Recurrence request :
curl http://127.0.0.1:80/test -v

more err message :
10489#10489: *4 lua entry thread aborted: runtime error: /usr/local/nginx/lua/resty/radixtree.lua:418: missing declaration for symbol 'free'

fix commit : master...jfboys:fix-free-fail

feature: need to record which matches the current request.

when there are multiple URIs, need to record which matches the current request.

we can use _uri to store the path of router.

{
                paths = {"/aa", "/bb*", "/name/:name/*other"},
                hosts = {"*.bar.com", "foo.com"},
                methods = {"GET", "POST", "PUT"},
                remote_addrs = {"127.0.0.1","192.168.0.0/16",
                                "::1", "fe80::/32"},
                vars = {
                    {"arg_name", "==", "json"},
                    {"arg_weight", ">", 10},
                },
                filter_fun = function(vars, opts)
                    return vars["arg_name"] == "json"
                end,

                metadata = "metadata /bb",
            }
ngx.say("matched uri: ", opts.matched._uri)
ngx.say("matched host: ", opts.matched._host)
ngx.say("matched method: ", opts.matched._method)

radixtree.new cost nearly 120s when uri_routes num is 10w

hi api7team,
I tested and found radixtree.new() cost nearly 120s when uri_routes num is 10w (32core, 32G, centos Physical Machine server),
I use radixtree_uri_with_parameter for http router.
do you have any optimize suggestions?

request help: Parameters in path

original issue: apache/apisix#2058 (comment)

For the "*" syntax, my understanding is that "*action" is an optional parameter. For example if we configure "uri": "/:id/foo/action/bar" in APISIX, both requests "/1/foo/get/bar" and "/2/foo/bar" should be matched by the route, while request "/3/foo/a/b/bar" should be rejected. However in APISIX the "" syntax has similar behavior with the colon syntax. The tests 7, 8 here https://github.com/api7/lua-resty-radixtree/blob/master/t/parameter.t seems broken: they are named "/name/*name/foo" but tested with "/name/:name/foo"

the end * of the path dont matchs some cases

according doc , if the end of the path is , it means that this is a prefix path. most cases do it well,but path , like //ping* , dont. it matches /xxx/ping , /xxx/ping , /xxx/pinggggg , /xxx/xxx/ping and so on. but it dont match some cases, like /xxx/pingxxx (x is not g) , /xxx/ping/xxx

release a new version

Since the lua-resty-expr lib released v1.2.0 version, any plan update to the new version? We are eager to use the new feature.

remote_addr config

local rx = radix.new({
{
path = "/aa",
metadata = "metadata /aa",
host = "foo.com",
method = {"GET", "POST"},
remote_addr = "127.0.0.1",
}
remote_addr is string type
remote_addr config like this remote_addr = {"192.168.10.0/24", "192.168.20.0/24", “192.168.50.1”,“192.168.50.2”,}
suport multiple VLAN and ip ;

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.