snapframework / snap-core Goto Github PK

In http://snapframework.com/docs/quickstart it is probably worth mentioning the need to

sudo apt-get install libexpat-dev

I worked it out, but this stopped it being a 'quick start'.

There are some notes somewhere in the repos (probably TODO in snap-core) about this.

Interesting project!

Provide a do-notation-based DSL for writing routes. Inspiration:

routes :: RoutesType
routes = do
    httpget "/" index
    httppost "/echo/:stuff" echo

Model it after Rails 3's routing DSL (http://guides.rubyonrails.org/routing.html):

routes.draw do
  get "/" => "home#index"
  post "/echo/:stuff" => "echoer#echo"
end

but using a style that is beautiful in Haskell, not a style beautiful in Ruby and transported to Haskell.

Using snap 0.3, when sending HTTP/1.0 style requests (for instance with curl -10 http://localhost:8080/), the error log gets filled with
SimpleBackend.acceptThread: thread killed
errors, although the request gets properly handled;

with DEBUG=1 the following debugging output becomes visible:

[      54] writeEnd: got EOF
[      54] SimpleBackend.writeOut(8): got chunk with 124 bytes
[      54] SimpleBackend.writeOut(8): wrote 124 bytes, last 10="3.1\r\n\r\n404"
[      54] sendResponse: whenEnum: 124 bytes enumerated
[      54] sendResponse: response sent
[      54] Server.httpSession: sent 3 bytes
[      54] httpSession: Connection: Close, harikari
[      54] thread killed, closing socket
[      54] Server: SimpleBackend.acceptThread: thread killed

Many people have asked for the ability to be able to distinguish parameters passed in through POST bodies vs. through query strings. We should keep the unified parameter mapping but we should also have separate maps (and functions to access them) for query/post parameters.

If the template is:
BAR
whatever

Where hsFunc is a Splice bound to a Haskell Function.
The xml that comes through to hsFunc has
["faz","$(foo)"] instead of ["faz","BAR"] as expected

It would be useful to be able to be able to disable this behaviour or modify the timeout for operations that are expected to take a long time.

This program can be used to test the behaviour by visiting http://localhost:8000

{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE PackageImports #-}

import qualified Data.ByteString.Char8 as B
import "monads-fd" Control.Monad.Trans (liftIO)

import Control.Concurrent
import Control.Exception (SomeException)
import Control.Monad.CatchIO
import Data.Time.Clock
import Prelude hiding (catch, log)
import Snap.Http.Server
import Snap.Types

main = do
    log "Starting Snap..."
    httpServe "0.0.0.0" 8000 "hostname" Nothing Nothing (catch500 site)

site = ifTop $ do
    liftIO $ do
        log "Received request, going to sleep"
        threadDelay (60 * seconds)
        log "Finished sleeping, delivering response"
    writeBS "Success!"
  where
    seconds = 1000000

log msg = do
    time <- getCurrentTime
    putStrLn (show time ++ " - " ++ msg)

catch500 :: Snap () -> Snap ()
catch500 = (`catch` h)
  where
    h :: SomeException -> Snap ()
    h ex = do
        putResponse (setContentType "text/plain" $
                     setResponseStatus 500 "Internal Server Error" $
                     emptyResponse)
        let msg = "Error: " ++ show ex
        writeBS (B.pack msg)
        liftIO (log msg)

On my Windows box I get the following output:

2010-10-05 07:03:12.0585224 UTC - Starting Snap...
2010-10-05 07:03:16.0807525 UTC - Received request, going to sleep
2010-10-05 07:03:47.0905261 UTC - Error: thread killed

They look like this:

[10/Jul/2010:15:10:54 +0200] Server.httpServe.go: got io exception: accept: invalid argument (Bad file descriptor)
[10/Jul/2010:15:10:54 +0200] Server.httpServe.go: got io exception: user error (accept: can't perform accept on socket ((AF_INET,Stream,0)) in status Closed)

I created a template that has ' and " within <script> tags and render is converting them into ' and ", which breaks the script.

It seems 'cabal install' doesn't support underscores in the 'Name' section of '.cabal' file and 'snap init' doesn't handle the case when name of project directory contains undescores.

Reproduce (create a project directoy that contains an underscore):
maksut@rintintin ~/haskell> mkdir snap_sample
maksut@rintintin ~/haskell> cd snap_sample/
maksut@rintintin ~/h/snap_sample> snap init
maksut@rintintin ~/h/snap_sample> cabal install
cabal: snap_sample.cabal:1: Parse of field 'name' failed.

Versions:
maksut@rintintin ~/h/snap_sample> cabal --version
cabal-install version 0.8.2
using version 1.8.0.6 of the Cabal library

maksut@rintintin ~/h/snap_sample> ghc-pkg list snap-core
/usr/lib/ghc-6.12.1/package.conf.d
/home/maksut/.ghc/x86_64-linux-6.12.1/package.conf.d
snap-core-0.2.11

Cheers,
maksut

Hello,

When I use

modifyResponse $ addCookie $ cookie $ userName user
redirect "/"

the browser seems to ignore the cookie, while it works when I use

modifyResponse $ addCookie $ cookie $ userName user
modifyResponse $ setResponseStatus 302 "Temorary Moved"
    . addHeader "Location" "/"

Installation of Snap fails with the following compilation error in snap-core:

[ 3 of 12] Compiling Snap.Iteratee    ( src/Snap/Iteratee.hs, dist/build/Snap/Iteratee.o )

src/Snap/Iteratee.hs:152:27:
    Ambiguous occurrence `foldl''
    It could refer to either `Data.Enumerator.foldl'', imported from Data.Enumerator at src/Snap/Iteratee.hs:115:0-47
                          or `Data.List.foldl'', imported from Data.List at src/Snap/Iteratee.hs:117:30-35

This is probably related to the timeout table stuff -- we switched to "tibbe's algorithm" for editing a shared structure updated by many writers but read by one reader: instead of modifying the shared table in an IORef directly, you instead record a list of edits which is then applied to the table later. The rationale is to prevent "floating garbage" -- the IORef thunk is not forced until the reader wakes up, so all of the old crap which would have been deleted hangs around and gets promoted to the next GC generation, where it lingers around forever.

Switching from the MVar we were using before to an IORef & DList of edits gave me a pretty big speedup under concurrent load on my laptop; on Doug's linux box it killed performance. We need to find out why.

There are some use cases, such as large file upload or long-polling, where it doesn't make sense to have the snap server handle timeouts in its core. It would be nice to be able to disable the timeout thread through either a server configuration setting or a compile time flag.

Get test coverage as close to 100% as possible. Boring but essential work!

For some reason, when using the handleFileUploads function and specifying a folder that doesn't exist on the file system, the code throws a ShortWriteException error

To replicate:

• generate a Snap project using snap init
• put the form.html in the resources/static folder
• put the Site.hs in the src/ folder
• run the Snap app
• go to http://localhost:8000/new
• upload any file you want
• check the logs/error.log

Here is the gist with the source code to use:

https://gist.github.com/826286

Problem: when serving a large file which client reads slowly Snap uses 100% CPU.

I don't know exactly if this is reproducible in general so let me explain my setup. I've created a default project (with "snap init") and not made any changes. I then start a server and let my client request a file. The client is a UPnP device which sends a GET request with a range header (requesting all bytes). However, the device reads data very slowly effectively keeping the session going for a long time (this is a device that plays music files so "long time" means something like 4 minutes). During this time snap uses 100% of the CPU.

I'm guessing the problem can be reproduced by creating a client which requests a file without actually reading from the socket. However, before I start spending time on debugging this I thought I'd report the problem.

I'm on Mac OS X 10.6.6, GHC 6.12.3 and "snap --version" says 0.3.1.1.

I had to add gcrypt an extra library after gnutls unter "if flag(gnutls)" in snap-server.cabal to make the linking phase of the snap package work, otherwise I would get an undefined reference to gcry_control.

Using:

• the macports version of gnutls and gcrypt, had to use --extra-lib-dirs=/usr/lib --extra-lib-dirs=/opt/local/lib to make the linking work (and the +universal flags of macports, of cause).
• 32bit version of Haskell Platform, OS X Snow Leopard
• cabal-install version 0.10.2 using version 1.10.1.0 of the Cabal library
• GHC 7.0.2
  Snow Leopard

Please add sections:

~~~~~~~~ {.haskell}
...
~~~~~~~~~

When using cabal to install snap-core on Windows 7 64-bit with the Haskell Platform installed and template-haskell 2.5.0.0, cabal segfaults. Unregistering template-haskell 2.5.0.0 forces the use of 2.4.0.1 and allows for successful completion. Perhaps the build should require exactly that version.

The "cabal install" command failed for me.

Specs:

snap-server-0.2.16.2

cabal-install version 0.8.2
using version 1.8.0.2 of the Cabal library

The Glorious Glasgow Haskell Compilation System, version 6.12.1

Mac OS X 10.6.5

MacBook Pro 5,1

Error trace:

wonko:Desktop andrew$ mkdir hello-snap
wonko:Desktop andrew$ cd hello-snap/
wonko:hello-snap andrew$ snap init
wonko:hello-snap andrew$ cabal install
Resolving dependencies...
Downloading hexpat-0.19.4...
Configuring hexpat-0.19.4...
Preprocessing library hexpat-0.19.4...
Building hexpat-0.19.4...
[ 1 of 16] Compiling Text.XML.Expat.Internal.IO ( Text/XML/Expat/Internal/IO.hs, dist/build/Text/XML/Expat/Internal/IO.o )
[ 2 of 16] Compiling Text.XML.Expat.IO ( Text/XML/Expat/IO.hs, dist/build/Text/XML/Expat/IO.o )
[ 3 of 16] Compiling Text.XML.Expat.SAX ( Text/XML/Expat/SAX.hs, dist/build/Text/XML/Expat/SAX.o )

Text/XML/Expat/SAX.hs:129:18: Not in scope: `T.breakBy'
Downloading xhtml-combinators-0.2.2...
Configuring xhtml-combinators-0.2.2...
Preprocessing library xhtml-combinators-0.2.2...
Building xhtml-combinators-0.2.2...
[ 1 of 11] Compiling Text.XHtmlCombinators.Escape ( src/Text/XHtmlCombinators/Escape.hs, dist/build/Text/XHtmlCombinators/Escape.o )
[ 2 of 11] Compiling Text.XHtmlCombinators.Internal ( src/Text/XHtmlCombinators/Internal.hs, dist/build/Text/XHtmlCombinators/Internal.o )
[ 3 of 11] Compiling Text.XHtmlCombinators.Render ( src/Text/XHtmlCombinators/Render.hs, dist/build/Text/XHtmlCombinators/Render.o )
[ 4 of 11] Compiling Text.XHtmlCombinators.Attributes.Internal ( src/Text/XHtmlCombinators/Attributes/Internal.hs, dist/build/Text/XHtmlCombinators/Attributes/Internal.o )
[ 5 of 11] Compiling Text.XHtmlCombinators.Attributes ( src/Text/XHtmlCombinators/Attributes.hs, dist/build/Text/XHtmlCombinators/Attributes.o )
[ 6 of 11] Compiling Text.XHtmlCombinators.Attributes.Internal.Safe ( src/Text/XHtmlCombinators/Attributes/Internal/Safe.hs, dist/build/Text/XHtmlCombinators/Attributes/Internal/Safe.o )
[ 7 of 11] Compiling Text.XHtmlCombinators.Attributes.Safe ( src/Text/XHtmlCombinators/Attributes/Safe.hs, dist/build/Text/XHtmlCombinators/Attributes/Safe.o )
[ 8 of 11] Compiling Text.XHtmlCombinators.Combinators ( src/Text/XHtmlCombinators/Combinators.lhs, dist/build/Text/XHtmlCombinators/Combinators.o )
[ 9 of 11] Compiling Text.XHtmlCombinators.Extras.Lorem ( src/Text/XHtmlCombinators/Extras/Lorem.hs, dist/build/Text/XHtmlCombinators/Extras/Lorem.o )
[10 of 11] Compiling Text.XHtmlCombinators ( src/Text/XHtmlCombinators.hs, dist/build/Text/XHtmlCombinators.o )
[11 of 11] Compiling Text.XHtmlCombinators.Safe ( src/Text/XHtmlCombinators/Safe.hs, dist/build/Text/XHtmlCombinators/Safe.o )
Registering xhtml-combinators-0.2.2...
Installing library in
/Users/andrew/.cabal/lib/xhtml-combinators-0.2.2/ghc-6.12.1
Registering xhtml-combinators-0.2.2...
cabal: Error: some packages failed to install:
heist-0.2.4 depends on hexpat-0.19.4 which failed to install.
hello-snap-0.1 depends on hexpat-0.19.4 which failed to install.
hexpat-0.19.4 failed during the building phase. The exception was:
ExitFailure 1

You cannot currently install Snap from Hackage because snap-server depends on snap-core ==0.2.14, but the most recently released version of snap-core is 0.2.13.2.

I have found blocking behavior in the snap framework handling of requests. Take my trivial case using the following actions

blocker :: StateSnap ()
blocker = do
liftIO $ do
putStrLn "before block"
threadDelay 20000000 -- 20 seconds
putStrLn "after block"
writeBS "blocker complete"

doer :: StateSnap ()
doer = do
liftIO $ putStrLn "do it"
writeBS "done it"

The behavior that I've experienced is that all calls to blocker are well, blocked, yet any calls to doer during that time are executed immediately without any blocking behavior. The end result of this is that it's essentially impossible to write a comet style web app with snap, since waiting to read from a channel renders the action/route blocked for all other calls.

This has been tried running against the available hackage versions as well as the latest 0.3 versions I've compiled from git.snapframework.com

“Framework” is a well-known anti-pattern, because it essentially harms code re-usability. This is, because you can’t just use parts of it, and can’t easily let it interface with other things not intended by the framework.
Look at thedailywtf.com for many many examples.
It’s the reason things like Ruby on Rails are completely unusable by professionals. (I’m one, with 20 years of experience.)

Therefore, I recommend a easy solution, to give Snap all the advantages, without any of the disadvantages:
• Define all modules and aspects of Haskell. (Should result in a tree with two dimensions of branching.) → If the code is properly designed, this should already be done. :)
• Define proper interfaces between them, and document those fully. → Should also already be done, if properly designed. :)
• Modify the interfaces in a fashion, that allows completely independent usage of each module/aspect, without having to use any of the other parts of the framework.
• Stop calling it a framework. (This is, so professionals are not repulsed/disgusted by it, be it rationally or irrationally. :)

Notice, that one can still use all the parts together, like a framework, just as before. Because it’s in no way bad that you can use it like that. It’s bad that you have to. (Including having to because of a lack of documentation or a lack of interfaces designed to support using parts independently.)

Then, and only then, will Snap have a chance against the overwhelmingly more powerful system I’m developing right now. :)

Currently, POST data is only retrieved when the Content-Type header equals:

application/x-www-form-urlencoded

I would like to see support for things like:

application/x-www-form-urlencoded; charset=UTF-8

As a result of setting the header and using finishWith, one can't set cookies and redirect, or set other headers and redirect.

Because the variable capturing of route captures only url parts (e.g. parts between slashes) it makes sense to urldecode the captured values before putting them in the Params map.

E.g. when running the example web application that comes with snap init I expect that browsing to http://localhost:8000/echo/%3C produces a <, not the encoded version %3C.

I'm trying to install snap-server in OSX and I'm getting the error below:

$ cabal install snap-server
Resolving dependencies...
Configuring bytestring-mmap-0.2.1...
Preprocessing library bytestring-mmap-0.2.1...
Building bytestring-mmap-0.2.1...
[1 of 3] Compiling System.IO.Posix.MMap.Internal ( System/IO/Posix/MMap/Internal.hs, dist/build/System/IO/Posix/MMap/Internal.o )
ghc: could not execute: /Library/Frameworks/GHC.framework/Versions/612/usr/lib/ghc-6.12.3/ghc-asm
cabal: Error: some packages failed to install:
bytestring-mmap-0.2.1 failed during the building phase. The exception was:
ExitFailure 1
snap-core-0.2.16 depends on bytestring-mmap-0.2.1 which failed to install.
snap-server-0.2.16.2 depends on bytestring-mmap-0.2.1 which failed to install.

The bytestring-mmap-0.2.1 dependency fails to build during Snap installation.

Specs:

Snap 0.4.1
Cabal library 1.8.0.6
cabal-install 0.8.2
GHC 6.12.3
Mac OS X 10.6.6
MacBook Pro 5,1

Trace:

$ cabal update
$ cabal install snap

...

Registering heist-0.5.0.1...
Installing library in /Users/andrew/.cabal/lib/heist-0.5.0.1/ghc-6.12.3
Registering heist-0.5.0.1...
cabal: Error: some packages failed to install:
bytestring-mmap-0.2.1 failed during the building phase. The exception was:
ExitFailure 1
snap-0.4.1 depends on bytestring-mmap-0.2.1 which failed to install.
snap-core-0.4.1 depends on bytestring-mmap-0.2.1 which failed to install.
snap-server-0.4.1 depends on bytestring-mmap-0.2.1 which failed to install.

• the functions generated by snap init should have haddocks: this is tutorial material after all. Something really verbose and friendly, almost literate programming and written at a beginner's level
• setUTF8Locale needs an #ifdef for windows (on win32 it should be a no-op)
• none of the glue code should be in the "snap init" program at all -- we should improve what's there and lift it to the libraries.

Right now we don't do multipart POST -- we should. It shouldn't be done automagically like application/x-www-form-urlencoded; the user needs to make "policy" decisions regarding file uploads.

As discussed in http://mailman-mail5.webfaction.com/pipermail/snap/2011-March/001913.html , snap never responds when curl sends a (malformed) POST with no data. According to the HTTP spec, snap should probably be sending back a 411 response rather than hanging.

If I use curl to issue a POST against a snap-based server, it sends an "Expect: 100-continue" header in the request, and then waits for a "100 Continue" from the server before sending the body.

If the server doesn't respond with a "100 Continue" within 2 seconds, curl will send the payload anyway. Unfortunately, snap doesn't issue a "100 Continue", hence making all POSTs from curl take at least 2 seconds.

Heist's use of xhtml is very restrictive -- it would be a lot better if we wrote an html5 parsing library (or a binding to one). This one looks promising:

http://www.netsurf-browser.org/projects/hubbub/

The parsing model is very close to expat's so we should be able to adapt some of the stuff from the hexpat library.

RFC ( http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2 )
says : Multiple message-header fields with the same field-name MAY be
present in a message if and only if the entire field-value for that
header field is defined as a comma-separated list [i.e., #(values)].
It MUST be possible to combine the multiple header fields into one
"field-name: field-value" pair, without changing the semantics of the
message, by appending each subsequent field-value to the first, each
separated by a comma.

Therefore, getHeader should maybe report headers with ", " intercalated.

snap init creates a dependency on heist >=0.2.4 && <=0.3

I have installed the latest heist (0.3.0.0).

This causes cabal to complain about a missing dependency.

See http://gist.github.com/607265 for the error message.

Michael Snoyman wants to make a WAI adapter for Snap, but he expects to be able to parse his own form bodies. When auto-parsing form bodies, make a copy of the stream and provide it back again later instead of just enumerating empty string.

If "foo.html" does not exist, the code (ifTop (fileServeSingle "foo.html")) <|> writeBS "some error" will not yield "some error," but rather, will return information about the missing file exception.

I found yesterday evening that if a handler throws an exception, the error is logged, but no response is sent back to the client - the connection is simply closed. This seems wrong. That apart, I can't see how to install an exception handler that could cause a 500 to be sent back. What do I do?

The fileServe functions do not match a URI that is correctly encoded with %20 for spaces, so are unable to serve files with spaces in their names.

This is presumably related to issue #33, however I thought it was worth a separate issue since it prevents Snap from serving certain otherwise ordinary files.

Also, a much more minor related issue is that using a URI containing unencoded spaces results in an empty response from Snap after 30 seconds, presumably because the request line was not successfully parsed. By comparison, Apache succeeds in parsing such lines immediately, but appears to parse only up to the first space, which then typically results in a 404 error. I haven't checked the HTTP spec on this - it might be better to returned some sort of "malformed request" error.

While running snap 0.3.1.1 I noticed the following.

Given a simple action such as:

setCookie :: Application ()
setCookie = do
let c = Cookie (B.pack "session")
(B.pack "value")
Nothing
(Just $ B.pack "localhost")
Nothing
modifyResponse (addCookie c)

site = route [ ("/", index)
, ("/echo/:stuff", echo)
, ("/setcookie", setCookie)
]
<|> fileServe "resources/static"

My response headers do not return a "Set-Cookie" key in my header.

Curl returns:
eric@oxley:~$ curl localhost:8000/setcookie -c cookie.txt -D header.txt
Response <
headers:
==============================
fromList []
==============================
version: (1,1)
status: 200
reason: OK

It would be nice, if it was possible to optionally redirect all logging to a different logging mechanism than the built-in fastlogger, e.g. like hslogger

that way one could log to a remote syslogd or customize the timestamp format

Snap should provide functions to send cookies to the browser.

Uploading a large file (say, 100 megabytes) causes significant memory usage.

Tested with a no-op usage of handleFileUploads from snap-core-0.4.0.2.

For web app development, I'm finding the roughly 2 second delay between me hitting control-C and being able to run again quite intrusive.

I know there are plans afoot to do dynamic app reloading, but I'd love it if this could be fixed anyway, because I'm not sure I'll want to use the dynamic reloading.