ostinelli / misultin Goto Github PK
View Code? Open in Web Editor NEWMisultin (pronounced mee-sool-téen) is an Erlang library for building fast lightweight HTTP(S) servers, which also supports websockets.
License: Other
Misultin (pronounced mee-sool-téen) is an Erlang library for building fast lightweight HTTP(S) servers, which also supports websockets.
License: Other
It would be nice to have an optional logger that could write response data to handle request. I believe this should be done inside the misultin_http:call_mfa/2
to handle user code crashes and still be able to log data. The format should be using a standard Apache accept log style.
Currently my implementation is similar to webmachine's logger but has to live inside my request loop because I don't want to patch misultin. Or are there other suitable solutions?
Misultin says in the README we can simply compile it with make
but it has a dependency on rebar which is not bundled with the repository. Or rebar should be bundled with the repo or we should change the README to make the dependency explicit. Thanks for the great project!
Hi,
We are trying to move from yaws to misultin on U.C.Engine (https://github.com/AF83/ucengine).
We have a streaming API (aka EventSource or Server-Sent Events). The current logic is currently inside a gen_server. It makes easier to handle all cases and the connection close (https://github.com/AF83/ucengine/blob/487cef0a3ccdbfae369938ec3ec36d296ac6c686/ucengine/src/controllers/async/uce_async_stream.erl)
With misultin I don't know how to achieve the same result. The request Pid is trashed as soon as the gen_server is started. Therefore the connection is closed.
Do you have something similar to streamcontent_from_pid from yaws?
Hello!
We use misultin-0.8 in our project and decided to generate release using rebar. We got:
=CRASH REPORT==== 7-Nov-2011::15:58:08 ===
crasher:
initial call: misultin_server:init/1
pid: <0.57.0>
registered_name: []
exception exit: {undef,[{httpd_util,rfc1123_date,[]},
{misultin_server,update_rfc_date,1},
{misultin_server,init,1},
{gen_server,init_it,6},
{proc_lib,init_p_do_apply,3}]}
in function gen_server:init_it/6
ancestors: [misultin,disp_sup,<0.49.0>]
...
Please, add inets into dependents!
I have some service that get headers from 3rd party.
Is there some automatic way of normalizing headers or I must do this on my own?
I'm talking about this fragment of code:
https://github.com/ostinelli/misultin/blob/master/src/misultin_http.erl#L579
In headers_01 example i have output like this:
HTTP/1.1 200 OK
Date: Sun, 30 Oct 2011 13:26:46 GMT
Server: misultin/0.8
Connection: Keep-Alive
Content-Length: 3
server: someserver
date: some date
ok
In headers_02 example all is OK but i need to care about proper header name formatting:
HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 3
Server: someserver
Date: some date
ok
Examples code:
-module(test).
-export([headers_01/0, headers_02/0]).
headers_01() ->
misultin:start_link([
{ port, 8080 },
{ loop, fun(Req) ->
Req:respond(200, [{'server', "someserver"}, {'date', "some date"}], <<"ok\n">>) end }
]).
headers_02() ->
misultin:start_link([
{ port, 8080 },
{ loop, fun(Req) ->
Req:respond(200, [{'Server', "someserver"}, {'Date', "some date"}], <<"ok\n">>) end }
]).
It's me again.
You've got a mistake in misultin_http and misultin_accepter.
When you replaced function such as accept/8 by **?MODULE:**accept/8.
You need to export them.
in misultin_acceptor -export([acceptor/8]).
in misultin_http -export([request/2]).
Thank you :)
Why misultin creates session cookies for subfolders every time i call session ?
Can i set global session cookie only for root folder ?
list_to_integer/2 as used in 2 files in misultin: src/misultin_http.erl and src/misultin_req.erl is no compatible in Erlang R13 since it's not autoimported.
So to stay compatible you should use erlang:list_to_integer/2
May be it will be better to create some special function to redirect response like
Req:redirect("/login"),
and not to write this code
Req:respond(302,[{'Location',"/login"}],"Redirect").
I think it will look better :)
Hi,
It would be very useful to have misultin_socket:accept/2 timeout after a few seconds if it does not accept anything, as it would allow smoother code upgrades. When it timeouts, it should re-enter the same loop with a fully qualified module name so it gets the new code. This lets us avoid killing processes to force them to get the new code and not loose any requests.
I'm testing the file upload functionality with an intentionally crippled network, and getting some timeouts after long file uploads (> 1 min). Here's the setup:
Cap network bandwidth to 600Kbit/s and add 50ms latency:
sudo ipfw pipe 1 config bw 600Kbit/s delay 50ms
sudo ipfw add 1 pipe 1 src-port 8080
sudo ipfw add 2 pipe 1 dst-port 8080
Here's the error I get after about 1.5 minutes of a 900K file:
18:46:41.773 [error] module: misultin_http
line: 392
error reading body: timeout
18:46:41.777 [error] module: misultin_http
line: 363
request timeout, sending error
Misultin version is 0.9-dev
(current master I think)
Any ideas why it's timing out? I'm tracking the file progress, and it seems like it's being uploaded...
Note: everything seems to work fine without the artificial network delay, and everything but file uploads work fine with the delay. I'm just worried this could happen when the site is live.
Firefox 7 and Aurora are using Rev8 of the draft, which does not appear to work with Misultin.
This would be really helpful.
There's one more thing, that really needed.
Ability to setup DocRoot for static files.
Becouse of that code looks not so pretty as it can be :)
Environment:
Erlang R14B02
Misultin 0.6.2
Reproduce:
After check the misultin.erl, i found in line152, it should be [start_application(crypto), start_application(public_key), start_application(ssl)].
is it right?
It would be awesome to have an ability to respond with a raw binary without any headers added by misultin.
Will the added support for these versions of the websocket protocol?
Firefox 6 will support hybi-07.
Firefox 7 and Google Chrome 14 will support hybi-10.
thanks.
This issue is derived from a bug we have in socket.io-erlang, which turns out to be a problem with misultin.
Someone reported to us that strings would fail when containing complex unicode characters and sent us some sample. The user would input a string like "m7mПривет!" and have crashes happening all the time.
Looking at the stack trace he'd send us, I saw the following data in the request body: <<"data=%7Em%7E7%7Em%7E%D0%9F%D1%80%D0%B8%D0%B2%D0%B5%D1%82%21">>
. Decoding it with misultin, it returns:
3> misultin_utility:parse_qs(S).
[{"data",
[126,109,126,55,126,109,126,208,159,209,128,208,184,208,178,208,181,209,130,33]}]
4> io:format("~ts~n",[[126,109,126,55,126,109,126,208,159,209,128,208,184,208,178,208,181,209,130,33]]).
~m~7~m~��иве�!
ok
5> io:format("~ts~n", [list_to_binary([126,109,126,55,126,109,126,208,159,209,128,208,184,208,178,208,181,209,130,33])]).
~m~7~m~Привет!
ok
The issue, as far as I can see, is that the unicode is parsed as a binary, which turns all code points in bytes (0..255). Then the binaries are blindly turned into lists, but they don't have the same unicode format in Erlang -- you actually need to convert them to codepoints greater than 255, and then output them with a ~ts
combination instead of just ~s
.
The problem is that you're likely using binary_to_list
to convert strings, but when you have unicode data with grapheme clusters, what you need is something more like this:
6> io:format("~ts~n",[unicode:characters_to_list(<<126,109,126,55,126,109,126,208,159,209,128,208,184,208,178,208,181,209,130,33>>)]).
~m~7~m~Привет!
ok
This use of unicode:characters_to_list(Str)
will work with IO lists and convert to whatever format you use. Note that the strings returned might now become invalid IO lists and should need to be converted back with unicode:characters_to_binary/1
before being pushed back in a socket.
hi,
I am trying to run misultin application in detached mode for my erlang project to handle http requests..
eg: misultin:start_link(Options).
Options consists of port,name,loop function as follows .
[{Port,port number},{name,list_to_atom("misultin_"++integer_to_list(Port))},{loop,Function}].
How to run this in detached mode?
misultin will issue a 400 when handling requests with long get parameters as the one reported at the bottom of this email
Turning on debugging this can be tracked to the warning "tcp error treating headers"
identified in the code at the following link
https://github.com/ostinelli/misultin/blob/master/src/misultin_http.erl#L191
Here's a sample request that triggers the described condition running the misultin_echo example on port 8080 on localhost specific length to trigger this condition is probably related to socket internals in our case the threshold seems to be around 1500 chars
http://127.0.0.1:8080/foo?var=foobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobar
It would be nice to have a nameless misultin start function, as start_link currently registers misultin's name which will effectively prevent us from starting multiple instances of it. Would you accept a patch that changes that?
https://github.com/ostinelli/misultin/blob/master/src/misultin.erl#L75
No issue, but a feature request. It would be fantastic if misultin supported the Socket.io client side API. Its becoming quite popular for real-time JavaScript implementations lately.
I test misultin_comet_long_polling moule with 4 clients. I want eche client take 5s. But I found the last client take 20s. Why? It is not Parallel?
Please reply me,Ths.
Hi,
in multin_req:chunk/2 can a method be added with a guard
chunk(Data, ReqT) when is_binary(Data) ->
stream([erlang:integer_to_list(erlang:iolist_size(Data), 16), "\r\n", Data, "\r\n"], ReqT);
so that binary data can be used in a chunked response, some data can cause issues with io_lib:format.
Thanks.
=ERROR REPORT==== 23-Mar-2011::17:05:53 ===
Error in process <0.90.0> with exit value: {undef,[{misultin_req,socket,[{misultin_req,{req,{sslsocket,new_ssl,<0.87.0>},ssl,{127,0,0,1},54668,undefined,close,undefined,{1,0},'GET',{abs_path,"/cometchat/"},"time=1300890927800",[{'Host',"site.dev"},{"X-Real-Ip","10.0.0.80"},{'X-Forwarded-For'...
=ERROR REPORT==== 23-Mar-2011::17:06:45 ===
Error in process <0.94.0> with exit value: {undef,[{misultin_ws,socket,[{misultin_ws,{ws,{sslsocket,new_ssl,<0.91.0>},ssl,false,{10,0,0,80},44866,undefined,{'draft-hixie',76},"https://site.dev","site.dev:8333","/manager",[{'Upgrade',"WebSocket"},{'Connection',"Upgrade"},{'Host'...
I've ported most of mochiweb to work with binaries:
https://github.com/benbro/mochiweb
The following modules are the binary version of lists:
mochiweb_binary_request.erl
mochiweb_binary_cookies.erl
mochiweb_binary_headers.erl
mochiweb_binary_util.erl
I think that the only thing missing is headers but it shouldn't be too hard.
Is there a chance this functionality will be implemented in misultin?
Thanks
i am not able to get response from the misultin server when i am trying to request through nginx. can anyone help me with this.
Thanks
When HTTP socket is closed and misultin_socket:send is closing with exit(normal),
then LoopPid, created in misultin_http:call_mfa doesn't exit because link mechanism doesn't propagate normal exit.
The only good idea to handle with it is to start both processes under same supervisor so, that he will shutdown both of them.
There is problem with header check function
it can't valid header in Firefox 7 because of this line in header.
The problem is because of multiple parameters, split by ","
{'Connection',"keep-alive, Upgrade"},
I created simple patch to solve this problem https://gist.github.com/1180310
I have two cases, that are not covered with misultin yet.
They are very, very important for me now and I think, that one of them will be important for many people.
First is big file upload. Just imagine: I want to upload BluRay disk. I can't raise memory limit, so we need just streaming input.
If I upload to Ruby on Rails application begin nginx, I would do simply thing: setup nginx to save request body to the same partition, where Rails store files, than Rails will get just local path.
But it is not what I want from misultin, because I have other case for upload. If user upload flv file, I want to be able to stream it from the very beginning. So, I need to have streaming input interface.
The same streaming input can be used for endless input streams.
I will try to make a fork and add proper patch, but if you have any ideas about best architecture, it would be great.
It seems to work fine with R14B03 on my mac, but on my centos machine:
[deklund@internal misultin]$ make
/usr/local/bin/erlc -W -I include -o ebin src/.erl
src/misultin_acceptor.erl:85: referring to built-in type tid as a remote type; please take out the module name
src/misultin_acceptor.erl:149: referring to built-in type tid as a remote type; please take out the module name
src/misultin_acceptor.erl:168: referring to built-in type tid as a remote type; please take out the module name
make: ** [all] Error 1
That's really good idea to create session server,
but can you create it for misultin_ws module, please ?
Does misultin supports file upload through HTTP?
I have a project that uses misultin, and I'm getting things set up on a new machine and I can't get misultin going.
Once I've pulled the source and run make I get these two errors:
src/../include/misultin.hrl:69: type gen_proplist() undefined
src/../include/misultin.hrl:122: type date_tuple() undefined
and looking through that file I found that if I move the lines that define those types before they are called than the compiler gets passed this and now gives the following errors:
src/misultin_acceptor.erl:85: referring to built-in type tid as a remote type; please take out the module name
src/misultin_acceptor.erl:149: referring to built-in type tid as a remote type; please take out the module name
src/misultin_acceptor.erl:168: referring to built-in type tid as a remote type; please take out the module name
So I followed the directions on this ticket: #46 and the compiler moves on to the following errors:
src/misultin_http.erl:48: type socket() undefined
src/misultin_http.erl:49: type socketmode() undefined
I ran out of ideas here. I haven't seen anyone else posting about these issues so maybe my problem lies elsewhere... Any ideas?
Two requests:
POST /chunk/a5b4608831c2e1e00c326ae3dc2873fee66cdf8f/524288
POST /chunk/a5b4608831c2e1e00c326ae3dc2873fee66cdf8f/1048576
But they just not handled. loop fun
not invoked.
I really like misultin and I'm thinking to start using it instead of mochiweb. But I really do not have enough support of PUT
, HEAD
and DELETE
methods. They are already used in client HTTP API and I can't change it.
Following the example in the wiki (https://github.com/ostinelli/misultin/wiki/ExamplesPage) entitled "Multiple Misultin servers on same node" the second server fails to start:
https://gist.github.com/1035861
It appears that the culprit was introduced through the use of named_table
ets tables in misultin_server
. I can fix the issue and submit a pull request but wanted to be sure that it wasn't already in progress.
Can anyone help me out to resolve this error
error sending data:
When giving too many requests
since commit 9ce6e54 hello world does not work on ubuntu (and debian as well).
steps to reproduce:
make;
cp examples/misultin_hello_world.erl ebin
cd ebin;
erlc misultin_hello_world.erl
shell V5.7.4 (abort with ^G)
Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:6:6] [rq:6] [async-threads:0] [hipe] [kernel-poll:false]
1> misultin_hello_world:start(8080).
** exception exit: shutdown
previous versions work fine
kubuntu 11.04
This is probably due to my lack knowledge of erlang, but
Misultin examples work fine when i am running them from erl,
but fail when i am trying to invoke erl with -noshell:
make example;
cd examples
erl
Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:6:6] [rq:6] [async-threads:0] [hipe] [kernel-poll:false]
Eshell V5.8.4 (abort with ^G)
1> misultin_hello_world:start(8000).
{ok,<0.34.0>}
Looks like there is something that erl does only when run interactively, but i have no ide what that could be.
there are no records for disabled clients in log file.
Hi!
I've clone repo, compile sources and copy misultin to my erlang library.
I've tested installation with misultin_hello_world example, and it works.
Then I tried to extend REST example with POST, and it dont work.
After that, I tried echo example, what build with POST in it, and POST don't work too.
My handle function:
handle_http(Req) ->
io:format("METHOD: ~s~n", [Req:get(method)]),
Req:ok("").
Then I start misultin server, I tried such queries with curl:
stepan@stepan ~/work $ curl -X GET localhost:8080
stepan@stepan ~/work $ curl -X POST localhost:8080
stepan@stepan ~/work $ curl -X PUT localhost:8080
stepan@stepan ~/work $ curl -X DELETE localhost:8080
And get following in erlang shell:
METHOD: GET
METHOD: DELETE
So, as I see, there is just GET and DELETE are working properly.
I create a comet application
checked the option {autoexit, false} in all browsers
it does not work in firefox. connection is not closed
This problem is a browser? (firefox 3.6.15, firefox 4.0)
Hi,
I am trying to redirect the request to a new destination URL. But it is not working.
I am using
[302,"http://github.com"]
can anyone please help me with this issue
If I respond with a 422, Misultin does not know how to throw this error back and I actually get a 200 instead.
Req:respond(422, [], "some validation error").
The response Misultin sends is:
Date: Sat, 26 Nov 2011 10:04:31 GMT
Server: misultin/0.9-dev
Connection: Close
Content-Length: 21
some validation error
=ERROR REPORT==== 9-Apr-2011::10:50:05 ===
Error in process <0.791.0> with exit value: {function_clause,[{misultin_http,convert_to_binary,[{http_request,'GET',{abs_path,"/extjs/resources/themes/images/default/form/exclamation.gif"},{1,1}}]},{misultin_http,socket_loop,3},{misultin_http,handle_get,2},{misultin_http...
Not sure what exactly is happening. Will try investigating deeper later tonight (may be it is my fault somehow?)
Hey,
I'm running this in SSL mode.
Just updated to 0.8dev and am getting an error for some requests.
=ERROR REPORT==== 30-May-2011::17:37:47 ===
Error in process <0.503.0> on node '[email protected]' with exit value: {{case_clause,undefined},[{misultin_websocket,'-check_headers/3-fun-0-',3},{lists,'-filter/2-lc$^0/1-0-',2},{lists,'-filter/2-lc$^0/1-0-',2},{misultin_websocket,check_headers,3},{misultin_websocket,check_websocket,3},...
=ERROR REPORT==== 30-May-2011::17:37:47 ===
module: misultin_server
line: 200
http process <0.503.0> has died with reason: {{case_clause,undefined},
[{misultin_websocket,
'-check_headers/3-fun-0-',3},
{lists,'-filter/2-lc$^0/1-0-',
2},
{lists,'-filter/2-lc$^0/1-0-',
2},
{misultin_websocket,
check_headers,3},
{misultin_websocket,
check_websocket,3},
{misultin_websocket,
check_websockets,3},
{misultin_http,headers,4},
{misultin_acceptor,
open_connections_switch,6}]}, removing from references of open connections and websockets
It is understandable that this fails since, on some occasions, binary_mode is undefined which is unmatched in a case:
Req is {{abs_path,"/"},'GET'}
Binary mode is false
Req is {{abs_path,"/"},'GET'}
Binary mode is undefined
I've been looking through the code but can't find what the problem is (yet).
Hi,
Thanks for your reply..
done as you told.. but still getting same problem.
when i run myapplication in erl shell as follows
application:start(my_app).
ok
myapp_server:start_misultin(7100,5).
ok
then misultin listening http_requests on ports 7100 to 7104.
but i need to run my application in detached mode(daemon)..
To perform this i wrote one program(here my_program) which open one erlang shell in detached mode
and run my application in that node. I didnt get any error..
but misultin not listening on particular ports to handle requests.
what went wrong?
what may be the problem in handing http_request when i am running my application in detached mode?
my_program()->
[|[HostName|]] = string:tokens(atom_to_list(node()),"@"),
Name = "adserver_" ++ integer_to_list(StartPort),
NodeName = list_to_atom(Name ++ "@" ++ HostName),
os:cmd("erl -noshell -env ERL_FULLSWEEP_AFTER 7 -sname " ++ Name ++ " -setcookie abc -detached"),
case catch(rpc:call(NodeName,application, start,[myapp_otp])) of
{'EXIT',} ->
io:format("Exit error ~n"),
"";
{badrpc,} ->
io:format("badrpc error ~n"),
"";
{error,Error}->
io:format("error ~p~n",[Error]),
"";
ok->
case catch( rpc:call(NodeName, myapp_server, start_misultin,[StartPort,NoOfPorts]))of
ok ->
io:format("myapp_server:misultin successfully started on node ~p~n",[NodeName]);
E ->
io:format("myapp_server:misultin not started on node with error ~p~n",[E])
end.
and my start_misultin process performs following
--------------
start_misultin()->
Ports=lists:seq(Sp,Sp+Nop-1),
lists:foreach(fun(Port)-> misultin:start_link([{port, Port}, {name, list_to_atom("misultin_"++integer_to_list(Port))}, {loop, fun(Req) -> handle_http(Req) end}]) end ,Ports).
Note: misultin is in my erlang library..my application file .app also includes all related misultin files..
I was working with cookies in misultin. I could set the cookie and see the cookies with Req:get_cookies(), but I couldn't get Req:get_cookie_value to work. Maybe I'm doing it wrong. Say I set a cookie Req:set_cookie("my_cookie", "test"). Then when I do Req:get_cookies() I would get back [{"my_cookie","test"}], but if I did Req:get_cookie_value("my_cookie","test") I get back
undefined. What is the proper way to use Req:get_cookie_value?
Content-Types by extension is hard coded in misultin_utility:get_content_type. How about customization of the function?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.