purescript-spec / purescript-spec Goto Github PK
View Code? Open in Web Editor NEWTesting framework for Purescript
Home Page: https://purescript-spec.github.io/purescript-spec/
License: MIT License
Testing framework for Purescript
Home Page: https://purescript-spec.github.io/purescript-spec/
License: MIT License
I think a structure like this would provide a much better separation of concerns:
shouldEqual :: String -> String -> Either String String
shouldEqual actual expected =
if (actual /= expected)
then Left "\"" <> actual <> "\" /= \"" <> expected <> "\""
else Right ""
shouldBeOk :: forall r. Either String String -> Aff r Unit
shouldBeOk value = case value of
Left error -> fail error
Right _ -> (pure unit)
it "should do something" do
shouldBeOk do
actual <- getSomeEither
expected <- getAnotherEither
actual `shouldEqual` expected
Then all assertions would simply be of type String -> String -> Either String String
If you have some state you want to test in multiple it
blocks, instead of having to instantiate it multiple times, it would be nice to be able to do it once in the describe
block for those specs. Ie:
describe "some state" do
someRef <- readSTRef myRef
someArray <- freeze mySTArray
it "should have this property" do
...
it "should have that property" do
...
Add something like setUp and tearDown, something like hooks in hspec:
https://hspec.github.io/writing-specs.html
I've created a pull request on package-sets https://github.com/purescript/package-sets/pulls , after adding pipes and its' dependency mmorph.
I'd be happy to contribute my psc-package.json as well. psc-package only runs 'build' at the moment, there is preliminary support for using it with pulp as well. I haven't figured out how best to run tests with it, I thought at least getting 'spec' to compile would be a good start.
(added because I've been bitten by transitive dependency hell with bower, the last time bower won and this seemed like a worthwile thing to try. Also frees purescript-spec from worrying about synchronizing version numbers with other packages or not.).
For parallelism with shouldEqual/shouldNotEqual?
What do you think of the idea to have itOnly
and describeOnly
cause compile warnings similar to how traceAnyA
and friends does. I can't speak for others, but I have accidentally committed an itOnly
block far too many times, not realizing.
psc-bundle
(and pulp build
with the --to
flag) gives me the error The module could not be parsed.
whenever Test.Spec.*
modules have been built to the output
directory. When I delete those directories, I am able to bundle. Any idea what's causing this?
What should be a pretty straightforward use case becomes pretty crazy (and ugly)
main ∷ Effect Unit
main = do
specs <- discover "\\.*Spec"
runSpecT config [ consoleReporter ] specs # un Identity # launchAff_
where
config =
defaultConfig
{ slow = 5.0 # Seconds # fromDuration
, timeout = Nothing
}
How about runSpecWithConfig
in a way that the un Identity
can be avoided?
Frameworks like Mocha needs to declare it
, describe
and so on synchronously and then await the asynchronous tests. Currently the reporter type class receives all test results after the tests have run (possibly asynchronously) which makes it impossible to use with Mocha.
I can see this ���
strange symbols when travis is running tests for example here (it's not just in one place). but this is not happening in my terminal (I'm using osx). not sure what reason could be.
Hello,
Thanks for writing this library, it is very useful.
It would be very cool if it would be possible to get a diff between expected and tested values, like in PyTest (see the "Making use of context-sensitive comparisons" section)
The ReadMe links to this page: https://purescript-spec.github.io/purescript-spec/
However, the actual docs for things like Hooks (https://github.com/purescript-spec/purescript-spec/blob/master/docs/writing-specs.md) aren't included.
I was thinking on adding describePar
and describeParOnly
. so Description is marked as parallel and during execution in _run
we are run producers in parallel. it would be very useful in case ps-spec is used for running integration tests and some tests don't effect each other.
so if you have:
describe "delay" do
it "can delay for 100ms" do
delay $ Milliseconds $ 100.0 * 1.0
it "can delay for 200ms" do
delay $ Milliseconds $ 100.0 * 2.0
it "can delay for 300ms" do
delay $ Milliseconds $ 100.0 * 3.0
it "can delay for 400ms" do
delay $ Milliseconds $ 100.0 * 4.0
it "can delay for 500ms" do
delay $ Milliseconds $ 100.0 * 5.0
which takes 1500 ms, you can apply this change:
-describe "Foo" do
+describePar "Foo" do
and it will take 500ms.
Main issue would be logging from tests which are executing in parallel, as you would have no way to know from which one a logged message is coming from.
Export run''
function, please. And I think it should be run'
, since there is no run
function with single prime.
Is there something like mocha's only
modifier (i.e. it.only
) to only run a certain test?
Regardless of how the describe
s and it
s are nested this should not happen:
A » B » C
✓︎ Yay!
A » B » C
✓︎ Oh noes...
It should be:
A » B » C
✓︎ Yay!
✓︎ Oh noes...
If this is the first module you are updating, you might want to read the 0.7 migration guide.
What are thoughts on adding something like this this?
forAll :: ∀ e a f. Foldable f => (a -> String) -> String -> f a -> (a -> Aff e Unit) -> Spec e Unit
forAll itTitle title arb f = describe title do
for_ arb \a -> it (itTitle a) (f a)
...
forAll _.str "format (unformat a) = a" arb \({ str }) -> do
(format $ unformat str) `shouldEqual` (Right str)
...
I have been using for_
in it
block but if some test case fails then other items are not tested and you can't see them in log. using this function you can see each item in log and all of them will be executed.
It would be nice to have this documented on Pursuit if possible.
This library is wonderful. I'd like to start using it with purerl
. But the problem is, it has dependencies on specifics of JavaScript. There are still a couple of FFI modules here, and there's a dependency on purescript-js-timers
. Are you interested in making this library able to be used by different backends? I don't want to suggest that you should maintain different backends though. That's kind of a big ask from me.
I have some ideas how this could be separated out, but I don't want you to have to maintain multiple libraries or have to jump through hoops to get things working. Assuming you're interested in allowing different backends, do you have any thoughts on how you'd like to make that possible?
I found that having a large amount of purely synchronously running tests can cause stack overflows.
I saw that Aff
has a MonadRec
instance breaking the execution context after every 100 binds.
I am not sure how to run it using the MonadRec
instance though, as runAff
is implemented using the FFI.
I am currently recovering from this by breaking the execution context with a manual later
call.
Something like http://hspec.github.io/hspec-discover.html.
It could probably be done in runtime by searching the NODE_PATH
after *Spec modules, but that locks it down to NodeJS.
It should use the pure version to get results and translate to purescript-spec
's [Group]
which can be consumed by describe
.
https://github.com/purescript/purescript-quickcheck/blob/master/src/Test/QuickCheck.purs#L66
Should be much more beginner friendly. Start with the regular pulp test
setup, step-by-step. Then introduce details and how to build using psc
.
It would be nice to be able to write a test body with assertions for pending
tests, that isn't run, but is only there for documenting the future test. Perhaps a pending'
DSL function that just ignores the Aff
:
describe "my library" do
pending' "should do math" $
1 + 1 `shouldEqual` 2
I noticed there's some FFI modules for doing ansi color stuff. There's a pure PS implementation that only depends on purescript-strings
: https://pursuit.purescript.org/packages/purescript-ansi/4.0.0.
Are you up for a PR to use purescript-ansi
instead of writing FFI modules in this library?
While writing some tests that need to run sequentially, I tried using sequential
from 4.0.0-rc1, but it does not seem to actually run the tests in sequence. The code is at:
DotReporterConfig contains slow
prop which is not used in dotReporter
When there are failing tests suite
should process.exit(1)
.
I feel the API of this package is solid now, and not breaking often. There are some more features we'd like to have, but I think they can be added without breaking compatibility with existing users' code. Thus, I think we should go for version 1.0.0 to signal the stability.
@felixSchl thoughts?
Sorry, but you forgot to actually export run'
function. As you can see, there is only run
function, that is exported:
module Test.Spec.Runner
( RunnerEffects
, run
, runSpec
, runSpec'
, defaultConfig
, timeout
, Config
, TestEvents
, Reporter
) where
I am opening this issue as a placeholder in case anyone wants to discuss remaining tasks or attach release-related commits.
But I'm mostly just wondering whether the current "v4.0.0-rc1" tag might change at all, or if it is safe to depend on the (exact) version in a dependent project. If the latter, then this will help me close purescript-trout#12.
Thanks for your great work on this library!
Unable to find a suitable version for purescript-console, please choose one by typing one of the numbers below:
1) purescript-console#3.0.0 which resolved to 3.0.0 and is required by test
2) purescript-console#^3.0.0 which resolved to 3.0.0 and is required by purescript-psci-support#3.0.0
3) purescript-console#^4.1.0 which resolved to 4.2.0 and is required by purescript-spec#3.1.0
any option is not working.
Maybe we could provide a BaseReporter
without the CONSOLE
effect, only doing the pipes machinery, not having default implementations that print to the console. If someone would like to build a reporter for web browsers, or something like the xUnit reporter, the console effect might not be what you want.
I get the following build error with purs v0.13.0
Compiling Test.Spec.Reporter.Console
Error found:
in module Test.Spec.Reporter.Console
at bower_components/purescript-spec/src/Test/Spec/Reporter/Console.purs:49:28 - 49:39 (line 49, column 28 - line 49, column 39)
Unknown value flushCrumbs
Later is deprecated so the examples don't work anymore.
It needs to be replaced with
delay (Milliseconds 100.0)
let res = "Alligator"
I think. Maybe it should also be forked or something?
Would be nice if that worked instead 😀
After install purescript-spec
in a different repo and copy/pasting the example, I can't seem to get it to compile. I get the following error message:
Error found:
in module Main
at src/Main.purs line 13, column 8 - line 27, column 51
Could not match type
Effect
with type
Aff
while trying to match type Effect Unit
with type Aff t0
while checking that expression (run [ consoleReporter
]
)
((describe "purescript-spec") ((discard (...)) (\$__unused ->
...
)
)
)
has type Aff t0
in value declaration main
where t0 is an unknown type
See https://github.com/purescript/documentation/blob/master/errors/TypesDoNotUnify.md for more information,
or to contribute content related to this error.
The file I'm trying to run is https://github.com/purescript-spec/purescript-spec/blob/master/example/Main.purs.
Start by just outputting the results in HTML.
Maybe integrate with Mocha/Karma.
Hi, I attempted to copy and paste the example from the purescript-spec documentation and run it under PureScript 0.12.5, but the code does not compile. Here is the error:
Error found:
in module Main
at src/Main.purs:35:8 - 49:51 (line 35, column 8 - line 49, column 51)
Could not match type
Effect
with type
Aff
while trying to match type Effect Unit
with type Aff t0
while checking that expression (run [ consoleReporter
]
)
((describe "purescript-spec") ((discard (...)) (\$__unused ->
...
)
)
)
has type Aff t0
in value declaration main
where t0 is an unknown type
See https://github.com/purescript/documentation/blob/master/errors/TypesDoNotUnify.md for more information,
or to contribute content related to this error.
This is for the code here: https://purescript-spec.github.io/purescript-spec/#full-example, i.e.
module Main where
import Prelude
import Data.Time.Duration (Milliseconds(..))
import Effect (Effect)
import Effect.Aff (launchAff_, delay)
import Test.Spec (pending, describe, it)
import Test.Spec.Assertions (shouldEqual)
import Test.Spec.Reporter.Console (consoleReporter)
import Test.Spec.Runner (run)
main :: Effect Unit
main = launchAff_ $ run [consoleReporter] do
describe "purescript-spec" do
describe "Attributes" do
it "awesome" do
let isAwesome = true
isAwesome `shouldEqual` true
pending "feature complete"
describe "Features" do
it "runs in NodeJS" $ pure unit
it "runs in the browser" $ pure unit
it "supports streaming reporters" $ pure unit
it "supports async specs" do
res <- delay (Milliseconds 100.0) $> "Alligator"
res `shouldEqual` "Alligator"
it "is PureScript 0.12.x compatible" $ pure unit
Version 0.11 introduced convenient re-exports in Test.Spec.Reporter
that have consequently been removed (accidentally?). Personally, I would welcome these convenience exports as it means I can easily import bundled reporters downstream, i.e. import Test.Spec.Reporter (consoleReporter)
. I apologize for not letting you know about adding these changes when crafting #32, I simply forgot amidst the many other changes.
Good evening, greetings from Australia and thank you for your amazing work on purescript!
Occasionally I'll have a Spec
so short and to-the-point that a textual description of it serves only to duplicate the literal interpretation of the expression.
it "accepts strings that contains substrings" $
"foobar" `AS.shouldContain` "foo"
(From Test.Spec.AssertionSpec
)
Ruby uses polymorphism to accept a spec that only provides a block.
it { expect("foobar").to contain "foo" }
(From http://www.betterspecs.org/#short)
lit
creates a spec for test cases where the code reads as a literal description of its behaviour.
This requires a degree of connoisseurship as the spec reporter emits empty lines as spec descriptions:
* Build successful.
* Running tests...
Test
Spec
Runner
✓︎ collects "it" and "pending" in Describe groups
✓︎ collects "it" and "pending" with shared Describes
✓︎ filters using "only" modifier on "describe" block
✓︎ filters using "only" modifier on nested "describe" block
✓︎ filters using "only" modifier on "it" block
✓︎ supports async
Test
Spec
Assertions
String
shouldContain
✓︎ accepts strings that contains substrings
✓︎
✓︎ rejects strings that does not contain substrings
However, I would argue that for smaller projects, the cardinality of the specs is sufficient to determine success:
* Build successful.
* Running tests...
Test
Spec
Runner
✓︎
✓︎
✓︎
✓︎
✓︎
✓︎
Test
Spec
Assertions
String
shouldContain
✓︎
✓︎
✓︎
And in the case of failure, the ordinality of the failure is sufficient to locate it:
* Building project in /Users/htmldrum/code/purescript/purescript-spec
Compiling Test.Spec.AssertionSpec
Compiling Test.Main
* Build successful.
* Running tests...
Test
Spec
Runner
✓︎ collects "it" and "pending" in Describe groups
✓︎ collects "it" and "pending" with shared Describes
✓︎ filters using "only" modifier on "describe" block
✓︎ filters using "only" modifier on nested "describe" block
✓︎ filters using "only" modifier on "it" block
✓︎ supports async
Test
Spec
Assertions
String
shouldContain
✓︎ accepts strings that contains substrings
1)
✓︎ rejects strings that does not contain substrings
...
57 passing
3 pending
1 failed
1) Test Spec Assertions String shouldContain
"foo" ∉ "fozbar"
* ERROR: Subcommand terminated with exit code 1
Thank you for your time.
I'm running Pandoc 2.7.3 at the moment and finding that options --chapters
and -S
have changed since the Makefile was created. Therefore, the docs can't be compiled to HTML at the moment.
The type signature for run demands that all runners use the same config type, which is not the case. @felixSchl ideas?
The Node.Process dependency is placing a hard dependency where you have to run the tests on nodejs. The issue with that is that you can't use something like phantomjs to run tests and test libraries that interact with the DOM.
How do you feel about going back to defining a custom PROCESS effect?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.