Add a way to run tests in parallel about purescript-spec HOT 9 CLOSED

I think if we add parallel/sequencial we should also provide a way for users to use custom monad then just Aff so it's possible for advanced users to use some custom logging, for example if we add m param to Spec, then _run will have type like this:

_run
  :: forall m
   . Config
  -> Spec' m Unit
  -> (NonEmptyArray Name -> m ~> Aff)
  -> Producer Event Aff (Array (Group Result))

This way someone can use ReaderT (String -> Effect Unit) Aff as m and in _run use Name as prefix of all logs, or use WriterT [String] Aff as m and just log everything after each test is finished/failed.

if we can provide hoistSpec :: forall m g. (m ~> g) -> Spec' m ~> Spec' g or similar that could might also help (I was not able to write this as you can't change type of state in State)

from purescript-spec.

Hspec has a parallel combinator that I think you can wrap any Spec with: https://hspec.github.io/parallel-spec-execution.html

Maybe could borrow some design from there?

Regarding logs, are you thinking of stuff printed by the code under test, rather than purescript-spec's output?

from purescript-spec.

yes spec logs just fine if you fork tests, issue is when the code under test is logging stuff

from purescript-spec.

I like the parallel and sequential contaminators from hspec, and will try to do similar thing instead of desribePar*.

Tut the issue with logging still is there, I think instead of it :: String -> Aff Unit -> Spec Unit
we can do something like this:
it :: String -> WriterT String Aff Unit -> Spec Unit or
it :: String -> ReaderT (String -> Eff Unit) Aff Unit -> Spec Unit

so you can still log to console but it has test name as prefix, or you just get log when test finishes/fails, instead of output being polluted by random strings from all parallel tests

from purescript-spec.

When it comes to logging from the code under test, I think it's a slippery slope to provide functionality in purescript-spec, and that we shouldn't implement any special support for it. Rather, we can document parallel and note that any code that prints to the console will need to synchronize, or perhaps log to different files or with different prefixes.

from purescript-spec.

@felixmulder, any thoughts on logging?

from purescript-spec.

If ps-spec itself is not providing that then probably Spec should become be something like this:

type TestName = String
type Spec = Spec'  Aff
type Spec' m t = State (Array (Group (ReaderT TestName m Unit))) t

this way one an use define there own spec:

type SpecWithCustomLoging = Spec'  (ReaderT (String -> Effect Unit) Aff)

from purescript-spec.

from purescript-spec.

so i have added hoistSpec #82 and par&seq functions (like in hspec)

take a look at this example for how it can be used to for logging (nave version vs writer vs reader)

https://github.com/purescript-spec/purescript-spec/blob/905769151190fef6cbce5168167b5f815f38a937/test/Test/Spec/HoistSpec.purs

and here you can see travis logs https://travis-ci.org/purescript-spec/purescript-spec/builds/466439951#L1285

from purescript-spec.