Git Product home page Git Product logo

acai's Issues

Fail on misuse of TestScoped bindings

A user reported an issue where they were binding an instance in test scope and then injecting that into a singleton scoped object. Apparently this does not produce an error from Acai but leads to incorrect behavior in the tests.

If possible it would be nice to fail fast in such a scenario.

TestScoped doesn't work in non-test thread.

When start a thread from the test, it can't inject @TestScoped instance to the new thread.
It throws com.google.inject.OutOfScopeException: Attempt to inject @TestScoped binding outside test:

Support running TestingServices in parallel

Tests that start multiple servers using @BeforeSuite in multiple testing services may be slower than they need to be as these @BeforeSuite methods are run sequentially.

It might be nice to support running these methods in parallel. I think if we do this it should probably be the only mode of operation to avoid adding too many options to the API. This may require updates to existing clients to add missing @DependsOn annotations.

Alternatively we could support an async version of @BeforeSuite etc that perhaps returns a future which we wait on before starting the tests. That way we don't have an on/off control in the API but clients don't start getting their code unexpectedly running in parallel automatically.

Test scope left open if test injection fails

We enter and exit test scope in TestEnvironment#beforeTest and TestEnvironment#afterTest respectively. The code in Acai which calls these methods looks like:

TestEnvironment testEnvironment = getOrCreateTestEnvironment(module);
testEnvironment.beforeSuiteIfNotAlreadyRun();
testEnvironment.beforeTest();
testEnvironment.inject(target);
try {
  statement.evaluate();
} finally {
  testEnvironment.afterTest();
}

If testEnvironment.inject(target) throws an exception we leave the scope open and all future tests fail with the exception:

java.lang.IllegalStateException: TestScope is already in progress.
    at com.google.common.base.Preconditions.checkState(Preconditions.java:177)
    at com.google.acai.TestScope.enter(TestScope.java:37)
    at com.google.acai.Acai$TestEnvironment.beforeTest(Acai.java:160)
    at com.google.acai.Acai$1.evaluate(Acai.java:78)

This also means, of course, that we fail to do any per-test tear-down as well in this case.

In fixing this issue another case to be aware of is that if a user defined @BeforeTest method fails currently this would also cause the TestScope to not be exited so simply moving testEnvironment.inject(target) into the try block will not make the code completely robust. We probably need to decouple the enter/exit scope from running @BeforeTest and @AfterTest methods.

Provide a clean way to stop services

Currently TestingService classes can provide a @BeforeSuite method but there is no corresponding @AfterSuite. This means if you start a bunch of services in the @BeforeSuite method there is no easy way to stop them when you are done testing.

One work around is to attach a JVM shutdown hook in your @BeforeSuite method but this is a bit ugly.

RunListener#testRunFinished in JUnit is basically what we need but a RunListener can only be attached via JUnitCore which would mean forcing users to implement their own main method or use one we provide. That seems much too intrusive.

The proposal in junit-team/junit4#874 may be useful in implementing this but we would need something that could attach the listener while still only requiring the user to add one thing to their tests. Telling people to add the Acai @Rule and also to register a listener for things to work correctly would be too onerous. Probably Acai could be implemented as a listener rather than a rule.

Support injecting methods of testing services

Currently each TestingService is instantiated once per environment. This means if you wish to do something with an object bound with a @TestScoped binding you need to use a Provider.

A common example might be a service that quits the webdriver instance started for an individual test:

class WebDriverQuitter implements TestingService {
  @Inject Provider<WebDriver> webDriver;

  @AfterTest void quitWebDriver() throws Exception {
    // Calling get on the Provider here returns the instance
    // for the test case which we are currently tearing down.
    webDriver.get().quit();
  }
}

If we supported injecting each @BeforeTest, @AfterTest method then this could become:

class WebDriverQuitter implements TestingService {
  @AfterTest void quitWebDriver(Webdriver webDriver) throws Exception {
    webDriver.quit();
  }
}

injecting the correct instance for the current test scope.

While cute this adds a bit of complexity so we should consider if it's worth bloating the API for something that's already achievable anyway. Also the scope will not be test scope when running @BeforeSuite methods so we should be careful that doesn't cause confusion.

Also would users then want to inject test methods too? That doesn't add as much value since the scope is the same as when injecting the test object. It would mean objects used in one test case only could be injected there rather than being fields but in such cases maybe the user should be splitting into multiple test classes anyway. Worth considering if this inconsistency between test methods and test services would be confusing though if we only supported injecting the service methods.

Use standard JUnit4 annotations.

Consider if we can kill the custom annotations in favour of using @Before, @After and @BeforeClass from JUnit4. Would this make sense and would the semantics be clear?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.