monokles / shoxy Goto Github PK

View Code? Open in Web Editor NEW

0.0 0.0 0.0 4.48 MB

D 100.00%

shoxy's People

Contributors

Watchers

shoxy's Issues

Expiration policy based on shortcode length

As mentioned in #6, there are expiration policies now based on:

Popularity of url (lasthit policy)
Constant time (simple policy)

There should be an expiration policy based on the shortcode length as well. As a basic idea it could be a linear formula of the form hours_until_expiration = (code_length * 26) - 20

IP address should be inserted into database when a new entry is made.

DB column already exists. just gotta add it to the Entry struct and related functions.

Posting a nonexistant URL results in an internal server error

Should return a 401 "Invalid URL" kinda thing.

~ ⌚ 16:08:02
$ http POST http://localhost:5050 url='http://wkekekeeiriiroieuroiekekekeke.com'
HTTP/1.1 500 Internal Server Error
Content-Length: 4984
Content-Type: text/plain; charset=UTF-8
Date: Sun, 24 Jul 2016 14:08:55 GMT
Keep-Alive: timeout=10
Server: vibe.d/0.7.28

500 - Internal Server Error

Internal Server Error

Internal error information:
object.Exception@../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/core/drivers/libevent2.d(276): Failed to lookup host 'wkekekeeiriiroieuroiekekekeke.com': nodename nor servname provided, or not known
----------------
/usr/include/dlang/dmd/std/exception.d:405 pure @safe void std.exception.bailOut!(Exception).bailOut(immutable(char)[], ulong, const(char[])) [0x763047]
/usr/include/dlang/dmd/std/exception.d:363 pure @safe bool std.exception.enforce!(Exception, bool).enforce(bool, lazy const(char)[], immutable(char)[], ulong) [0x766372]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/core/drivers/libevent2.d:276 vibe.core.net.NetworkAddress vibe.core.drivers.libevent2.Libevent2Driver.resolveHost(immutable(char)[], ushort, bool) [0x8a541b]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/core/net.d:42 vibe.core.net.NetworkAddress vibe.core.net.resolveHost(immutable(char)[], ushort, bool) [0x8c491e]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/core/net.d:37 vibe.core.net.NetworkAddress vibe.core.net.resolveHost(immutable(char)[], std.socket.AddressFamily, bool) [0x911438]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/core/net.d:93 vibe.core.net.TCPConnection vibe.core.net.connectTCP(immutable(char)[], ushort) [0x8c49f6]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/client.d:534 bool vibe.http.client.HTTPClient.doRequest(scope void delegate(vibe.http.client.HTTPClientRequest), bool*, bool, std.datetime.SysTime) [0x83dd10]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/client.d:472 bool vibe.http.client.HTTPClient.doRequestWithRetry(scope void delegate(vibe.http.client.HTTPClientRequest), bool, out bool, out std.datetime.SysTime) [0x83dac7]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/client.d:443 vibe.http.client.HTTPClientResponse vibe.http.client.HTTPClient.request(scope void delegate(vibe.http.client.HTTPClientRequest)) [0x83d8df]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/client.d:80 vibe.http.client.HTTPClientResponse vibe.http.client.requestHTTP(vibe.inet.url.URL, scope void delegate(scope vibe.http.client.HTTPClientRequest), vibe.http.client.HTTPClientSettings) [0x83c6fb]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/client.d:65 vibe.http.client.HTTPClientResponse vibe.http.client.requestHTTP(immutable(char)[], scope void delegate(scope vibe.http.client.HTTPClientRequest), vibe.http.client.HTTPClientSettings) [0x83c47e]
source/shoxyserver.d:20 bool shoxyserver.ShoxyServer.isRealUrl(immutable(char)[]) [0x782949]
source/shoxyserver.d:108 void shoxyserver.ShoxyServer.postURLRequest(vibe.http.server.HTTPServerRequest, vibe.http.server.HTTPServerResponse) [0x782d8d]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/router.d:294 _D4vibe4http6router9URLRouter13handleRequestMFC4vibe4http6server17HTTPServerRequestC4vibe4http6server18HTTPServerResponseZ21__T9__lambda4TmTAAyaZ9__lambda4MFmMAAyaZb [0x8484a7]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/router.d:677 const(bool function(immutable(char)[], scope bool delegate(ulong, scope immutable(char)[][]))) vibe.http.router.MatchTree!(vibe.http.router.Route).MatchTree.doMatch [0x8497bc]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/router.d:610 bool vibe.http.router.MatchTree!(vibe.http.router.Route).MatchTree.match(immutable(char)[], scope bool delegate(ulong, scope immutable(char)[][])) [0x849057]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/router.d:287 void vibe.http.router.URLRouter.handleRequest(vibe.http.server.HTTPServerRequest, vibe.http.server.HTTPServerResponse) [0x848161]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/server.d:1769 bool vibe.http.server.handleRequest(vibe.core.stream.Stream, vibe.core.net.TCPConnection, vibe.http.server.HTTPListenInfo, ref vibe.http.server.HTTPServerSettings, ref bool) [0x879aee]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/server.d:1543 void vibe.http.server.handleHTTPConnection(vibe.core.net.TCPConnection, vibe.http.server.HTTPListenInfo) [0x87838a]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/server.d:1424 void vibe.http.server.listenHTTPPlain(vibe.http.server.HTTPServerSettings).doListen(vibe.http.server.HTTPListenInfo, bool, bool).__lambda4(vibe.core.net.TCPConnection) [0x877d50]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/core/drivers/libevent2_tcp.d:592 void vibe.core.drivers.libevent2_tcp.ClientTask.execute() [0x90b041]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/core/core.d:486 void vibe.core.core.makeTaskFuncInfo!(void delegate()).makeTaskFuncInfo(ref void delegate()).callDelegate(vibe.core.core.TaskFuncInfo*) [0x7ee271]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/core/core.d:1084 void vibe.core.core.CoreTask.run() [0x8a18c1]
??:? void core.thread.Fiber.run() [0x99a379]
??:? fiber_entryPoint [0x99a0fb]
??:? [0xffffffff]

Make URL length configurable through a POST parameter

This allows the user to create shorter URLS if they wish to, or longer if they wish to reduce the probability of it being guessed by others.

Some things have to be taken into account though:

Users might try to create shortcodes with the same length as delete keys. This should be caught and prevented.
certain shortcodes might not be available, e.g. those with length < 4
I think a decent "collision" protocol for the above case would be to just increase the length by 1 and try again, until a free code is found.
The server should keep track of how many of a certain length are used for this to work smoothly though.

Shoxy should only proxy resources (images, pdfs, etc..), and only when enabled

Right now Shoxy just proxies the first http request for any entry.
This should be changed so that it only proxies resources (as to hide their original location), and only if this has been enabled in shoxy.json

Resources that are split over multipart body streams are not proxied correctly

Right now, we've noticed an image would be proxied partially and would seem to be randomly cut up. This is probably due to the body stream not having enough data being available for reading (yet) and hence prematurely ends writing to the response body.

create README

I guess

Posting faulty URLS should not return a 404

Whenever a faulty URL is posted, the correct error message is returned, but vibe.d also returns a 404 page because shoxy doesn't supply an alternative.

~ ⌚ 16:02:35
$ http POST http://localhost:5050 url='the cake is a lie'
HTTP/1.1 404 URL string is not allowed
Content-Length: 80
Content-Type: text/plain; charset=UTF-8
Date: Sun, 24 Jul 2016 14:04:30 GMT
Keep-Alive: timeout=10
Server: vibe.d/0.7.28

404 - Not Found

Not Found

Internal error information:
No routes match path '/'


~ ⌚ 16:04:30
$ http POST http://localhost:5050 url='HACKER\\'         
HTTP/1.1 404 URL string is not allowed
Content-Length: 80
Content-Type: text/plain; charset=UTF-8
Date: Sun, 24 Jul 2016 14:05:57 GMT
Keep-Alive: timeout=10
Server: vibe.d/0.7.28

404 - Not Found

Not Found

Internal error information:
No routes match path '/'

Index page should provide some usage info and allow for posting URLs

Ideally also some basic CSS or something to make it look a little bit better.
This will require a bit of javascript because browsers cannot (AFAIK) post forms as JSON by themselves.

See http://vibed.org/templates/diet

posting URLS without http:// or https:// results in an internal server error

Should check for this

~ ⌚ 16:06:32
$ http POST http://localhost:5050 url='google.com'                  
HTTP/1.1 500 Internal Server Error
Content-Length: 3532
Content-Type: text/plain; charset=UTF-8
Date: Sun, 24 Jul 2016 14:08:02 GMT
Keep-Alive: timeout=10
Server: vibe.d/0.7.28

500 - Internal Server Error

Internal Server Error

Internal error information:
object.Exception@../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/inet/url.d(60): No schema in URL:google.com
----------------
/usr/include/dlang/dmd/std/exception.d:405 pure @safe void std.exception.bailOut!(Exception).bailOut(immutable(char)[], ulong, const(char[])) [0x763047]
/usr/include/dlang/dmd/std/exception.d:363 pure @safe bool std.exception.enforce!(Exception, bool).enforce(bool, lazy const(char)[], immutable(char)[], ulong) [0x766372]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/inet/url.d:60 ref vibe.inet.url.URL vibe.inet.url.URL.__ctor(immutable(char)[]) [0x8e044d]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/inet/url.d:135 vibe.inet.url.URL vibe.inet.url.URL.parse(immutable(char)[]) [0x8e0d15]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/client.d:65 vibe.http.client.HTTPClientResponse vibe.http.client.requestHTTP(immutable(char)[], scope void delegate(scope vibe.http.client.HTTPClientRequest), vibe.http.client.HTTPClientSettings) [0x83c455]
source/shoxyserver.d:20 bool shoxyserver.ShoxyServer.isRealUrl(immutable(char)[]) [0x782949]
source/shoxyserver.d:108 void shoxyserver.ShoxyServer.postURLRequest(vibe.http.server.HTTPServerRequest, vibe.http.server.HTTPServerResponse) [0x782d8d]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/router.d:294 _D4vibe4http6router9URLRouter13handleRequestMFC4vibe4http6server17HTTPServerRequestC4vibe4http6server18HTTPServerResponseZ21__T9__lambda4TmTAAyaZ9__lambda4MFmMAAyaZb [0x8484a7]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/router.d:677 const(bool function(immutable(char)[], scope bool delegate(ulong, scope immutable(char)[][]))) vibe.http.router.MatchTree!(vibe.http.router.Route).MatchTree.doMatch [0x8497bc]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/router.d:610 bool vibe.http.router.MatchTree!(vibe.http.router.Route).MatchTree.match(immutable(char)[], scope bool delegate(ulong, scope immutable(char)[][])) [0x849057]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/router.d:287 void vibe.http.router.URLRouter.handleRequest(vibe.http.server.HTTPServerRequest, vibe.http.server.HTTPServerResponse) [0x848161]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/server.d:1769 bool vibe.http.server.handleRequest(vibe.core.stream.Stream, vibe.core.net.TCPConnection, vibe.http.server.HTTPListenInfo, ref vibe.http.server.HTTPServerSettings, ref bool) [0x879aee]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/server.d:1543 void vibe.http.server.handleHTTPConnection(vibe.core.net.TCPConnection, vibe.http.server.HTTPListenInfo) [0x87838a]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/http/server.d:1424 void vibe.http.server.listenHTTPPlain(vibe.http.server.HTTPServerSettings).doListen(vibe.http.server.HTTPListenInfo, bool, bool).__lambda4(vibe.core.net.TCPConnection) [0x877d50]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/core/drivers/libevent2_tcp.d:592 void vibe.core.drivers.libevent2_tcp.ClientTask.execute() [0x90b041]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/core/core.d:486 void vibe.core.core.makeTaskFuncInfo!(void delegate()).makeTaskFuncInfo(ref void delegate()).callDelegate(vibe.core.core.TaskFuncInfo*) [0x7ee271]
../../.dub/packages/vibe-d-0.7.28/vibe-d/source/vibe/core/core.d:1084 void vibe.core.core.CoreTask.run() [0x8a18c1]
??:? void core.thread.Fiber.run() [0x99a379]
??:? fiber_entryPoint [0x99a0fb]
??:? [0xffffffff]

URLS should expire based on some condition

In order to make sure the shortcode pool is not exhausted, shortcodes should expire. I'm not sure what the best option would be for the expiration condition.
Here are some I've come up with so far

after some time period (e.g. 24 hours)
after the shortcoded URL has not been accessed for X amount of time
wipe all shortcodes every X hours

I want it to be possible for some shortcodes to be permanent when a special key is passed as a POST parameter.

Reposting a URL returns just the shortcode instead of the full URL

See terminal snippet. The second call should return the full URL (but not deleteKey as it might be someone else trying to shorten the same URL)

~ ⌚ 15:58:22
$ http POST http://localhost:5050 url='http://github.com'    
HTTP/1.1 200 OK
Content-Length: 71
Content-Type: application/json; charset=UTF-8
Date: Sun, 24 Jul 2016 14:01:46 GMT
Keep-Alive: timeout=10
Server: vibe.d/0.7.28

{
    "deleteKey": *snip*,
    "url": "xn--zce.tv/txa2i"
}


~ ⌚ 16:01:47
$ http POST http://localhost:5050 url='http://github.com'
HTTP/1.1 200 OK
Content-Length: 15
Content-Type: application/json; charset=UTF-8
Date: Sun, 24 Jul 2016 14:02:34 GMT
Keep-Alive: timeout=10
Server: vibe.d/0.7.28

{
    "url": "txa2i"
}

server prints stacktrace on DELETE method without json body

how to reproduce: http DELETE <server>
(assuming httpie installed)

Options should be configurable through a config file.

This is pretty straightforward.

The following options should be configurable (incomplete)

server URL
server port
allowed shortcode length(s)
proxy settings (proxy resources y/n) (and possibly per resource type?)
DB details (host, port, user, password, schema)

server prints stacktrace on invalid HTTP methods

how to reproduce: http DEBUG <domain>
(assuming httpie is installed)

monokles / shoxy Goto Github PK

shoxy's People

Contributors

Watchers

shoxy's Issues

Recommend Projects

Recommend Topics

Recommend Org