ianb / doctestjs Goto Github PK

View Code? Open in Web Editor NEW

145.0 145.0 10.0 3.97 MB

Javascript testing made easy and explicit

Home Page: http://ianb.github.io/doctestjs/

License: MIT License

CSS 5.08% JavaScript 93.56% Shell 0.51% Python 0.85%

doctestjs's People

Contributors

Stargazers

Watchers

Forkers

oturing kevinelong davidmiller kaiquewdev mattmccray shamrin bolerio dotnetcarpenter cscott alykoshin

doctestjs's Issues

Suite runner

The tests aren't highly partitioned, but you can put multiple tests on multiple pages (especially using data-href-pattern). In that case, each page could be considered a "test", and the suite would be a list of pages.

I'd like a runner that could run and collect the output from such a list of pages. The HTML I imagine like:

<script src="doctest.js"></script>...
<body class="doctestsuite">

<a class="doctest" href="?test=test1.js">test1</a>
...

Each tagged test would be loaded in an iframe, with sufficient hooks to tell it to report (using postMessage) back to the parent frame. The parent frame would then show a summary, and allow you to look at each iframe. The iframes might run hidden, but could be expanded to inspect the test.

Tests could probably run in parallel. In cases where you might want user intervention (like if #27 is implemented) there might be the ability for an iframe to bring itself to the front (again using postMessage to request this). Or even run the test in a new window (where postMessage can still be used) instead of an iframe, for tests that don't work in iframes.

Allow testing just one test, plus

Sometimes you want to test just what's breaking. Tests often build up cross-dependencies, but at least it should be possible to test one.

Some things should still be marked as "setup" tests that always are run. Probably like it used to be, <pre class="commenttest setup">

AbortSection could also be reintroduced.

Switch to class="test" instead of class="commenttest"

class=commenttest is long and awkward. Though it should stay working, we should switch to something simpler like class=test

Loading of section-less external .js fails silently

Pull request #41 broke loading of external files without sections. E.g. examples-2.js silently fails to load in examples/examples-2.html.

How is it different from `assert.deepEqual`

You can just replace all your print statements with assert.deepEqual and it would work.

If you wanted you could even write a clever equal function to do the pattern matching you wanted.

What warrants including an entire parser and doing crazy comment parsing?

doctestjs not running on IE8

title says it all

Maybe include MockXMLHttpRequest

This library is pretty darn handy: https://github.com/philikon/MockHttpRequest

And it fits well into doctest. Maybe include it with a pattern of usage? Alongside NosyXMLHttpRequest?

examples.html SyntaxError

I downloaded the current repository and viewed the examples in my browser (Firefox).
While examples-2.html seems to work fine, all tests of examples.html seem to be broken:

Error: SyntaxError: missing } after function body
Error: ReferenceError: query is not defined
Error: ReferenceError: $ is not defined
Error: ReferenceError: $ is not defined

Create Node.js test runner

It would be nice if doctest.js included a command so you could run doctest some_test.js and it would run the test, output the results, and have an appropriate exit code. With some extra effort it could also setup PhantomJS or Zombie.js for doing full DOMish testing.

Put in hooks for integration testing systems

Systems like Jenkins (or more specific, stuff like jstestnet or whatever it is called) need to be able to tell if the test passes, without necessarily caring about the details. A reporter hook of some kind would be helpful.

This might take the form of postMessage hook, or a <script> tag that gets injected or something that defines some specific functions that get called as part of the test.

Allow Javascript to contain HTML fixture data

I imagine something like this:

/* == FIXTURE
<button id="foo">test</button>
*/

button = $('#foo');
...

And the HTML would be removed from the test (maybe with some evidence it was there) and put after the <pre> element.

Allow tests to be embedded in comments

I've kind of left out the original concept of Python's doctest, where you put tests in your inline code documentation (comments in Javascript, docstrings in Python). It would be nice to have something like that, perhaps, maybe. Mostly it would be just a fancier parser.

Why have a `print` function

Why do

print(value)
// => 42

When you could just have

// value => 42

Make Abort() failure more clear

Right now calling Abort() stops the test, but not in a way that makes it clear they were aborted.

Documentation typo

The HTML example starts with:

<DOCTYPE html>

The exclamation mark before DOCTYPE is missing which causes an invalid document.

Set favicon based on pass/fail status

I hear qunit does this.

Use stacktracejs

This library could be helpful: http://stacktracejs.com/

Uncaught TypeError: Illegal invocation on Chromium Version 21.0.1180.89 (154005)

This page gives uncaught TypeError on Chromium after pressing 'Test' (it works on Firefox):

<html>
    <head>
        <title>cutplayer testing page</title>
        <script type="text/javascript" src="doctest.js"/></script>
        <script type="text/javascript" src="../static/js/jquery-1.8.1.min.js"></script>
        <script type="text/javascript" src="../static/js/jquery.jplayer.min.js"/></script>
        <script type="text/javascript">
var JPLAYER_SWF_PATH = '../static/swf';


function AudioSrc(media, offset, length) {
    this.media = media;
    this.offset = offset;
    this.length = length;
};


function get_audio_for(station, seconds) {
    var MEDIA_ROOT, CLIP_DURATION, audio_start, radio_id, base_url;

    MEDIA_ROOT = '../../media/';
    CLIP_DURATION = 10;

    radio_id = {'Sos Rara': 'rara',
                'Bufalo': 'bufalo',
                'Hells Bells': 'hells_bells'}[station];

    audio_start = Math.floor(seconds / CLIP_DURATION) * CLIP_DURATION;

    base_url = MEDIA_ROOT + radio_id + '_' + audio_start.toString();
    return new AudioSrc({mp3: base_url + '.mp3'}, audio_start, CLIP_DURATION);
};

        </script>
    </head>
    <body>
        <h1>cutplayer tests</h1>

        <div>
            <button onclick="doctest()">test all</button>
            <button onclick="doctest.reload(this)">reload</button>
            <br>
            <pre id="doctestOutput" class="output"></pre>
        </div>
        <hr>
        <hr>
        <div>
            <h2><code>get_audio_for(station, seconds)</code></h2>
            <button onclick="doctest(0, 'test_getaudio_for', 'output_getaudio_for')">test</button>
            <button onclick="doctest.reload(this)">reload</button>
            <br>
            <pre id="output_getaudio_for" class="output"></pre>
<pre id="test_getaudio_for" class="doctest">
$ get_audio_for('Sos Rara', 1348168009).media.mp3;
"../../media/rara_1348168000.mp3"
</pre>
        </div>
    </body>
</html>

Documentation broken link

At the bottom of http://doctestjs.org/ you say:

If you've used Python's doctest and found it annoying or not widely useful for doctest: doctest.js fixes all those problems: see this post for more.
But the link to "this post" is broken! It is:
http://blog.ianbicking.org/2012/10/02/why-doctest-js-is-better-than-pythons-doctest/
...but should be:
http://www.ianbicking.org/blog/2012/10/why-doctestjs-is-better-than-pythons-doctest.html

get yer variables outta my window! (please)

just had an unexpected collision between my variable 'output' and doctestjs. I'd suggest tucking absolutely everything under a single global.

Helper for long-wait interactive prompts

Sometimes a test really needs user intervention, like they'll need to accept a permission or something we can't do in content. Practically it's not so bad to throw up a dialog and wait indefinitely for the user to actually do this. A little helper would be nice, e.g.:

doctest.Dialog("You have to accept the <i>geo</i> permission, get ready!").wait()

We could also include a question in this, like:

doctest.Dialog.confirm("Did the video start playing?");
// => Did the video start playing? Yes

What's the license for doctestjs?

Refine Node.js support

Technically doctest can be used with Node.js, using ConsoleReporter. But it's pretty scrappy. A lot of the best features are really dependent on the rich interface of the HTML report. Run it as a web server? Generate an HTML report on disk?

Catch and format parse errors

Since we already have two parsers built into doctest.js we should be able to catch syntax errors and display them nicely (highlighting the line).

Bonus points if we could check <script> files (especially if we detect some error that indicates a missing/failed script)

Markup for known-failing test

There are often tests that are broken, but you know are broken. Basically they represent pending work, not a regression. A bit of markup for these would be nice. Off the top of my head:

thisDoesNotWork();
/* = FAILING
=> what it should do
*/

Not sure if I like or dislike using caps to signify special control structures in the comments. @ is also often used, e.g., @failing or @when The = is intended to go with the core markup, =>.

array pretty printing is not

ewww:

14:26 < lth> Actual output:
14:26 < lth> {
14:26 < lth>       apps: [
14:26 < lth>   "are",
14:26 < lth>   "super",
14:26 < lth>   "awesome"
14:26 < lth>       ],
14:26 < lth>       open: "web"
14:26 < lth>     }

docs: html must be left justified

I tried building simple example, but with html indented at each div.

Tests wouldn't run.

Left-justified HTML, and now tests ran. This should be documented, it's definitely unexpected behavior.

syntax errors in tests do not manifest well

consider this example code:

https://gist.github.com/729297

upon clicking the button there's no visibile output on the page, and I have to go into my debugger to figure out what's happened. It turns out the exception parsing logic in doctest.js doesn't handle this exception well:

https://github.com/ianb/doctestjs/blob/4bced7339ff97834a7561d0e6626deed70018de7/doctest.js#L387-404

This makes debugging tests more difficult than necessary.

(issue experience primarily on chromium/linux, though I can imagine it'd be much different in other environments)

Fix up NosyXMLHttpRequest invocation/installation

The invocation/installation of NosyXMLHttpRequest is kind of awkward. Also it should have some options to show the result.

Create Node.js nodeunit wrapper

It would be nice if you could make nodeunit-compatible tests from doctests. This might look like:

doctest = require('doctest');
exports.testDoctests = doctest.UnitTest('some_file.js');

This might be very simplistic, like just create a function that raises an exception when it fails. But it could also do something like explode tests into a suite.

http://doctestjs.org/ is a spam domain

Could you put the docs up somewhere else, or stop redirecting to this site?

Test loading failure

The <pre href="URL.js"> style doesn't seem to work locally.

My pre tag:

<pre class="commenttest" href="./test_basic.js"></pre>

Safari, Chrome and Firefox all return:

// Error fetching ./test_basic.js?nocache=1350190004281 status: 0

Are local (file:///) doctests not supported?

Support QUnit Composite

It seems like QUnit Composite should be a reasonable container for multiple doctest tests, so long as we "act" like a QUnit test.

Tutorial for wait/async only has an incorrect example

The tutorial for using wait() ONLY has an incorrect example, it doesn't actually show you who to use it correctly! Took me quite a bit of time digging in the code an rereading the docs several times to understand that the trivial example is:

var done = false;
window.setTimeout(function() {
done = true;
}, 10);
wait(function(){return done;});
print(done);
// => false

print(done);
// => true

I suggest adding this trivial example in the tutorial and deleting the existing example. It's shorter AND shows what will actually work.

This all assuming my example above actually IS the correct way of doing it, it might be totally wrong :P

ISC, not GPL

license.html is ISC.

index.html says GPL (and href's non-existent license.txt).

Let // => have continuations

Right now you can't do:

example()
// => line 1
// line 2

Instead you must do:

example()
/* => line 1
line 2 */

// is sometimes nicer, especially if you have to continue the line or something. Maybe detect lines with // that are adjacent?

Conditional output expectations

Some output differs based on environment. For instance:

x = null;
x.y;

That says something like TypeError: x is null in Firefox, and TypeError: Cannot read property 'y' of null in Chrome. Neither one is "right" per se. You could just use ... to ignore the content entirely, but that's not super.

Instead, something like:

/* == when userAgent('Chrome')
=> Error: TypeError: Cannot read property 'y' of null
== when userAgent('Firefox')
=> Error: TypeError: x is null
*/

The code after when would be an expression, with some helpers available like a function for simple user-agent selection.

... not always behaving as expected

Here's output of a test failure where the expected output has an elipse. Shouldn't this test be passing?

Failed example:
    chan.call({method: "error", params: "smush_object", error: ef, success: sf}); wait(); // thrown objects get smushed.
Expected:
    output("runtime_error", "{\"dont\":\"smush\",\"me\":\"no!!\"...)
Got:
    output("runtime_error", "{\"dont\":\"smush\",\"me\":\"no!!\",\"line\":21,\"expressionBeginOffset\":553,\"expressionEndOffset\":583,\"sourceId\":2126914640,\"sourceURL\":\"file:///C:/Users/lth/dev/jschannel/test/index_child.html\"}")

New Spy pattern for success/failure callbacks

There's many APIs that work like func(onsuccess, onfailure) where one of those two functions will be called. You can make two spies, but it's hard to wait on either of two spies, and the naming is tedious.

I'm not sure what this should look like, maybe:

func(Spy('func.success'), Spy('func.error'));
Spy.waitEither('func.success', 'func.error');

That's kind of tedious. Maybe:

funcSpy = Spy.successFailure('func');
func(funcSpy.success, funcSpy.failure);
funcSpy.wait();

Or:

func(Spy('func'), Spy('func').error());
Spy('func').wait();

And .error() would be a special method that returns a spy that basically works as the wait callback for the main spy, but with its own name ("func.error").