Git Product home page Git Product logo

testparameterinjector's Introduction

TestParameterInjector

Link to Javadoc.

Introduction

TestParameterInjector is a JUnit4 and JUnit5 test runner that runs its test methods for different combinations of field/parameter values.

Parameterized tests are a great way to avoid code duplication between tests and promote high test coverage for data-driven tests.

There are a lot of alternative parameterized test frameworks, such as junit.runners.Parameterized and JUnitParams. We believe TestParameterInjector is an improvement of those because it is more powerful and simpler to use.

This blogpost goes into a bit more detail about how TestParameterInjector compares to other frameworks used at Google.

Getting started

JUnit4

To start using TestParameterInjector right away, copy the following snippet:

import com.google.testing.junit.testparameterinjector.TestParameterInjector;
import com.google.testing.junit.testparameterinjector.TestParameter;

@RunWith(TestParameterInjector.class)
public class MyTest {

  @TestParameter boolean isDryRun;

  @Test public void test1(@TestParameter boolean enableFlag) {
    // ...
  }

  @Test public void test2(@TestParameter MyEnum myEnum) {
    // ...
  }

  enum MyEnum { VALUE_A, VALUE_B, VALUE_C }
}

And add the following dependency to your .pom file:

<dependency>
  <groupId>com.google.testparameterinjector</groupId>
  <artifactId>test-parameter-injector</artifactId>
  <version>1.15</version>
  <scope>test</scope>
</dependency>

or see this maven.org page for instructions for other build tools.

JUnit5 (Jupiter)

Click to expand

To start using TestParameterInjector right away, copy the following snippet:

import com.google.testing.junit.testparameterinjector.junit5.TestParameterInjectorTest;
import com.google.testing.junit.testparameterinjector.junit5.TestParameter;

class MyTest {

  @TestParameter boolean isDryRun;

  @TestParameterInjectorTest
  void test1(@TestParameter boolean enableFlag) {
    // ...
  }

  @TestParameterInjectorTest
  void test2(@TestParameter MyEnum myEnum) {
    // ...
  }

  enum MyEnum { VALUE_A, VALUE_B, VALUE_C }
}

And add the following dependency to your .pom file:

<dependency>
  <groupId>com.google.testparameterinjector</groupId>
  <artifactId>test-parameter-injector-junit5</artifactId>
  <version>1.15</version>
  <scope>test</scope>
</dependency>

or see this maven.org page for instructions for other build tools.

Basics

Note about JUnit4 vs JUnit5:
The code below assumes you're using JUnit4. For JUnit5 users, simply remove the @RunWith annotation and replace @Test by @TestParameterInjectorTest.

@TestParameter for testing all combinations

Parameterizing a single test method

The simplest way to use this library is to use @TestParameter. For example:

@RunWith(TestParameterInjector.class)
public class MyTest {

  @Test
  public void test(@TestParameter boolean isOwner) {...}
}

In this example, two tests will be automatically generated by the test framework:

  • One with isOwner set to true
  • One with isOwner set to false

When running the tests, the result will show the following test names:

MyTest#test[isOwner=true]
MyTest#test[isOwner=false]

Parameterizing the whole class

@TestParameter can also annotate a field:

@RunWith(TestParameterInjector.class)
public class MyTest {

  @TestParameter private boolean isOwner;

  @Test public void test1() {...}
  @Test public void test2() {...}
}

In this example, both test1 and test2 will be run twice (once for each parameter value).

The test runner will set these fields before calling any methods, so it is safe to use such @TestParameter-annotated fields for setting up other test values and behavior in @Before methods.

Supported types

The following examples show most of the supported types. See the @TestParameter javadoc for more details.

// Enums
@TestParameter AnimalEnum a; // Implies all possible values of AnimalEnum
@TestParameter({"CAT", "DOG"}) AnimalEnum a; // Implies AnimalEnum.CAT and AnimalEnum.DOG.

// Strings
@TestParameter({"cat", "dog"}) String animalName;

// Java primitives
@TestParameter boolean b; // Implies {true, false}
@TestParameter({"1", "2", "3"}) int i;
@TestParameter({"1", "1.5", "2"}) double d;

// Bytes
@TestParameter({"!!binary 'ZGF0YQ=='", "some_string"}) byte[] bytes;

For non-primitive types (e.g. String, enums, bytes), "null" is always parsed as the null reference.

Multiple parameters: All combinations are run

If there are multiple @TestParameter-annotated values applicable to one test method, the test is run for all possible combinations of those values. Example:

@RunWith(TestParameterInjector.class)
public class MyTest {

  @TestParameter private boolean a;

  @Test public void test1(@TestParameter boolean b, @TestParameter boolean c) {
    // Run for these combinations:
    //   (a=false, b=false, c=false)
    //   (a=false, b=false, c=true )
    //   (a=false, b=true,  c=false)
    //   (a=false, b=true,  c=true )
    //   (a=true,  b=false, c=false)
    //   (a=true,  b=false, c=true )
    //   (a=true,  b=true,  c=false)
    //   (a=true,  b=true,  c=true )
  }
}

If you want to explicitly define which combinations are run, see the next sections.

Use a test enum for enumerating more complex parameter combinations

Use this strategy if you want to:

  • Explicitly specify the combination of parameters
  • or your parameters are too large to be encoded in a String in a readable way

Example:

@RunWith(TestParameterInjector.class)
class MyTest {

  enum FruitVolumeTestCase {
    APPLE(Fruit.newBuilder().setName("Apple").setShape(SPHERE).build(), /* expectedVolume= */ 3.1),
    BANANA(Fruit.newBuilder().setName("Banana").setShape(CURVED).build(), /* expectedVolume= */ 2.1),
    MELON(Fruit.newBuilder().setName("Melon").setShape(SPHERE).build(), /* expectedVolume= */ 6);

    final Fruit fruit;
    final double expectedVolume;

    FruitVolumeTestCase(Fruit fruit, double expectedVolume) { ... }
  }

  @Test
  public void calculateVolume_success(@TestParameter FruitVolumeTestCase fruitVolumeTestCase) {
    assertThat(calculateVolume(fruitVolumeTestCase.fruit))
        .isEqualTo(fruitVolumeTestCase.expectedVolume);
  }
}

The enum constant name has the added benefit of making for sensible test names:

MyTest#calculateVolume_success[APPLE]
MyTest#calculateVolume_success[BANANA]
MyTest#calculateVolume_success[MELON]

@TestParameters for defining sets of parameters

You can also explicitly enumerate the sets of test parameters via a list of YAML mappings:

@Test
@TestParameters("{age: 17, expectIsAdult: false}")
@TestParameters("{age: 22, expectIsAdult: true}")
public void personIsAdult(int age, boolean expectIsAdult) { ... }

which would generate the following tests:

MyTest#personIsAdult[{age: 17, expectIsAdult: false}]
MyTest#personIsAdult[{age: 22, expectIsAdult: true}]

The string format supports the same types as @TestParameter (e.g. enums). See the @TestParameters javadoc for more info.

@TestParameters works in the same way on the constructor, in which case all tests will be run for the given parameter sets.

Tip: Consider setting a custom name if the YAML string is large:

@Test
@TestParameters(customName = "teenager", value = "{age: 17, expectIsAdult: false}")
@TestParameters(customName = "young adult", value = "{age: 22, expectIsAdult: true}")
public void personIsAdult(int age, boolean expectIsAdult) { ... }

This will generate the following test names:

MyTest#personIsAdult[teenager]
MyTest#personIsAdult[young adult]

Filtering unwanted parameters

Sometimes, you want to exclude a parameter or a combination of parameters. We recommend doing this via JUnit assumptions which is also supported by Truth:

import static com.google.common.truth.TruthJUnit.assume;

@Test
public void myTest(@TestParameter Fruit fruit) {
  assume().that(fruit).isNotEqualTo(Fruit.BANANA);

  // At this point, the test will only run for APPLE and CHERRY.
  // The BANANA case will silently be ignored.
}

enum Fruit { APPLE, BANANA, CHERRY }

Note that the above works regardless of what parameterization framework you choose.

Advanced usage

Note about JUnit4 vs JUnit5:
The code below assumes you're using JUnit4. For JUnit5 users, simply remove the @RunWith annotation and replace @Test by @TestParameterInjectorTest.

Dynamic parameter generation for @TestParameter

Instead of providing a list of parsable strings, you can implement your own TestParameterValuesProvider as follows:

import com.google.testing.junit.testparameterinjector.TestParameterValuesProvider;

@Test
public void matchesAllOf_throwsOnNull(
    @TestParameter(valuesProvider = CharMatcherProvider.class) CharMatcher charMatcher) {
  assertThrows(NullPointerException.class, () -> charMatcher.matchesAllOf(null));
}

private static final class CharMatcherProvider extends TestParameterValuesProvider {
  @Override
  public List<CharMatcher> provideValues(Context context) {
    return ImmutableList.of(CharMatcher.any(), CharMatcher.ascii(), CharMatcher.whitespace());
  }
}

Notes:

  • The provideValues() method can dynamically construct the returned list, e.g. by reading a file.

  • There are no restrictions on the object types returned.

  • The provideValues() method is called before @BeforeClass, so don't rely on any static state initialized in there.

  • The returned objects' toString() will be used for the test names. If you want to customize the value names, you can do that as follows:

    private static final class FruitProvider extends TestParameterValuesProvider {
      @Override
      public List<?> provideValues(Context context) {
        return ImmutableList.of(
            value(new Apple()).withName("apple"),
            value(new Banana()).withName("banana"));
      }
    }
    
  • The given Context contains the test class and other annotations on the @TestParameter-annotated parameter/field. This allows more generic providers that take into account custom annotations with extra data, or the implementation of abstract methods on a base test class.

Dynamic parameter generation for @TestParameters

Instead of providing a YAML mapping of parameters, you can implement your own TestParametersValuesProvider as follows:

@Test
@TestParameters(valuesProvider = IsAdultValueProvider.class)
public void personIsAdult(int age, boolean expectIsAdult) { ... }

static final class IsAdultValueProvider implements TestParametersValuesProvider {
  @Override public ImmutableList<TestParametersValues> provideValues() {
    return ImmutableList.of(
      TestParametersValues.builder()
        .name("teenager")
        .addParameter("age", 17)
        .addParameter("expectIsAdult", false)
        .build(),
      TestParametersValues.builder()
        .name("young adult")
        .addParameter("age", 22)
        .addParameter("expectIsAdult", true)
        .build()
    );
  }
}

testparameterinjector's People

Contributors

alexjski avatar arnassmicius avatar bjchris32 avatar dependabot[bot] avatar eikemeier avatar gediminaszukas avatar jakewharton avatar nymanjens avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

testparameterinjector's Issues

JUnit 5: Support `@Nested` classes

If one runs an @TestParameterInjectorTest test method inside an inner class annotated with JUnit 5's @Nested annotation, it throws the following stack trace.

Example

import com.google.testing.junit.testparameterinjector.junit5.TestParameterInjectorTest;
import com.google.testing.junit.testparameterinjector.junit5.TestParameters;
import org.junit.jupiter.api.Nested;

import static org.junit.jupiter.api.Assertions.assertEquals;

class OuterTests {

  @Nested
  class NestedTests {

    @TestParameterInjectorTest
    @TestParameters("{input: foo, output: bar}")
    void test(String input, String output) {
      assertEquals(output.length(), input.length());
    }
  }
}
Stack trace
java.lang.IllegalStateException: No matching test parameter annotation found for parameter of type OuterTests in method/constructor OuterTests$NestedTests(OuterTests)
java.lang.AssertionError: java.lang.IllegalStateException: No matching test parameter annotation found for parameter of type OuterTests in method/constructor OuterTests$NestedTests(OuterTests)
	at com.google.testing.junit.testparameterinjector.junit5.ExecutableValidationResult.assertValid(ExecutableValidationResult.java:66)
	at com.google.testing.junit.testparameterinjector.junit5.TestParameterInjectorExtension.validateTestMethodAndConstructor(TestParameterInjectorExtension.java:67)
	at com.google.testing.junit.testparameterinjector.junit5.TestParameterInjectorExtension.provideTestTemplateInvocationContexts(TestParameterInjectorExtension.java:49)
	at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.lambda$execute$0(TestTemplateTestDescriptor.java:106)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:110)
	at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:44)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
	at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
	at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:62)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at jdk.proxy1/jdk.proxy1.$Proxy2.stop(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
Caused by: java.lang.IllegalStateException: No matching test parameter annotation found for parameter of type OuterTests in method/constructor OuterTests$NestedTests(OuterTests)
	at com.google.testing.junit.testparameterinjector.junit5.TestParameterAnnotationMethodProcessor.validateMethodOrConstructorParameters(TestParameterAnnotationMethodProcessor.java:619)
	at com.google.testing.junit.testparameterinjector.junit5.TestParameterAnnotationMethodProcessor.validateConstructor(TestParameterAnnotationMethodProcessor.java:518)
	at com.google.testing.junit.testparameterinjector.junit5.TestMethodProcessorList.lambda$validateConstructor$4(TestMethodProcessorList.java:134)
	at com.google.common.collect.Iterators$6.transform(Iterators.java:829)
	at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:52)
	at com.google.common.collect.Iterators.tryFind(Iterators.java:780)
	at com.google.common.collect.Iterables.tryFind(Iterables.java:708)
	at com.google.common.collect.FluentIterable.firstMatch(FluentIterable.java:476)
	at com.google.testing.junit.testparameterinjector.junit5.TestMethodProcessorList.validateConstructor(TestMethodProcessorList.java:135)
	... 85 more

Consider keeping a changelog?

I noticed the project doesn't publish changelog notes in releases or keep a changelog.md file. I'd recommend keeping one, as it's hard to see what's in the last two releases (1.3 and 1.2) unless we grep commit history instead

Run in Firebase Test Lab

Looks like the TestParameterInjector is unsupported by the Firebase Test Lab, at least all tests, that use the injector don't run in my case.
Isn't it?

Support for Android pre-API 26

I'm trying to migrate our codebases from Burst in an effort to deprecate Burst, but this trips us up pretty hard on Android.

I found this comment:

// Parameter is not available on old Android SDKs, and isn't desugared. Many (most?) Android tests
// will run against a more recent Java SDK, so this will work fine. If it proves problematic for
// users trying to run, say, emulator tests, it would be possible to just not provide parameter
// names on Android.

We could probably get this method and associated interface into D8/R8 desugaring since most (all?) of Parameter can be emulated by wrapping the originating Executable and delegating to its methods. Have you spoken to the R8 team at all about this?

Or would you be open to taking that approach for now while separately we try and convince the R8 team? Since AnnotationWithMetadata is already an abstraction the only change would seem to be to duplicate the getAnnotationWithMetadataListWithType and have it accept the Executable subtypes (Constructor, Method, etc.) to pull out the needed components.

Build fails with firebase dependency

This library is using protobuf-java dependency which clashes with Firebase library which contains protobuf-javalite.
I'm not sure if there is a way to resolve this dependency issue.

My android instrumentation builds fails with multiple errors like this:

Duplicate class com.google.protobuf.AbstractMessageLite found in modules jetified-protobuf-java-3.14.0 (com.google.protobuf:protobuf-java:3.14.0) and jetified-protobuf-lite-3.0.1 (com.google.protobuf:protobuf-lite:3.0.1)

How to be inject parameter into the test class constructor

This is my code, and I want to inject the class in the test constructor, but idk how to achieve that with the TestParameterInjector runner.

@RunWith(TestParameterInjector::class)
class SampleScreenshotTest(private val config: ScreenshotHelper.Companion.Config) {
  
  @get:Rule
  val paparazzi = Paparazzi(
    deviceConfig = config.deviceConfig,
    theme = "ThemeNoActionBar",
    maxPercentDifference = 0.01,
    imageSize = ImageSize.FullBleed,
    renderingMode = SessionParams.RenderingMode.SHRINK,
    shrinkDirection = ShrinkDirection.VERTICAL
  )

  @Test
  fun test1(
    @TestParameter canLearnMore: Boolean,
  ) = paparazzi.lazyViewItemSnapshot(
    viewItem = // view,
    isDarkTheme = config.deviceConfig.nightMode == NightMode.NIGHT
  )
}

Looking forward for the response
Thanks

Non-static method support for dynamic parameter generation

Hello,
We have some dynamic parameter generation logic (its a bit complex) that has a few non-static calls. Both the options we have here - enum or TestParameterValuesProvider require static methods only. Are there other options we could use here so as to keep our current parameter generation logic?

Thanks!

Wrong transitive dependency protobuf

👋 Hi,
In the 1.4 release, you mentionned

Replaced dependency on protobuf-java by a dependency on protobuf-javalite

But in fact, this is not what has been done in acb188b#diff-9c5fb3d1b7e3b0f54bc5c4182965c4fe1f9023d449017cece3005d3f90e8e4d8L137-R138

-      <artifactId>protobuf-java</artifactId>
-      <version>3.14.0</version>
+      <artifactId>protobuf-lite</artifactId>
+      <version>3.0.1</version>

The switch is from protobuf-java to protobuf-lite.
And this is still causing errors when using with firebase-perfs (where GeneratedMessageLite is coming from your transitive dependency instead of firebase's):

java.lang.NoSuchMethodError: 'void com.google.protobuf.GeneratedMessageLite.registerDefaultInstance(java.lang.Class, com.google.protobuf.GeneratedMessageLite)'	
at com.google.firebase.perf.v1.TraceMetric.<clinit>(TraceMetric.java:1945)	
at com.google.firebase.perf.metrics.TraceMetricBuilder.build(TraceMetricBuilder.java:39)	
at com.google.firebase.perf.metrics.Trace.stop(Trace.java:258)

It's only "recently" that the correct dependency has been restored:
4d0fe88

-      <artifactId>protobuf-lite</artifactId>
-      <version>3.0.1</version>
+      <artifactId>protobuf-javalite</artifactId>
+      <version>3.20.0</version>

Do you have any plan to release this fix?
Regards

test execution time is not accurate

Hi,
I am running java/maven/JUnit4 on multiple test classes from testparameterinjector 1.14 , I do find the first method of first test class execution is very slow , it takes more than 3-4 mins compared with Junit Parameterized , any suggestion on this ?

I can not find this delay on sequently method/class .

Support for null strings

Any tips/suggestions for getting null as a value? I know JSR won't allow null to be passed in because it's not a constant but is there any clean way of doing it besides generating my own TestParameterValuesProvider for every parameter that needs to support null?

// Doesn't work
@Test
fun someTest(@TestParameter(value = [null, "", "foo"]) param: String) {
    ...
}

// Ugly bc you'd have to define the provider for every String argument in every test method
@Test
fun someTest(@TestParameter(valuesProvider = FooProvider::class) param: String) {
    ...
}

class FooProvider : TestParameter.TestParameterValuesProvider {
    override fun provideValues(): MutableList<String?> {
        return mutableListOf(null, "", "foo")
    }
}

Support tests written in kotlin with parameter types that are `@JvmInline` value classes

Kotlin has a feature called "value classes" which currently desugar into primitive types. E.g. the following defines a class that will compile down to just being an float anywhere it's used:

@JvmInline value class Temp(val value: Float)

When such a class is used as the type of a constructor parameter, Kotlin generates an additional constructor with an extra parameter of type DefaultConstructorMarker. E.g. given the above class, this test class will actually have two constructors:

@RunWith(TestParameterInjector::class)
class TestClass(temp: Temp)

This will generate constructors (as described by IntelliJ's "Show Kotlin bytecode" view):

  • private <init>(F)V
  • public synthetic <init>(FLkotlin/jvm/internal/DefaultConstructorMarker;)V

When ran as a test, this library will see two constructors and give the following error message: "Expected exactly one constructor, but got {list of constructors}".

Use case

There are a number of reasons kotlin tests might need to be parameterized on value classes. E.g., the Jetpack Compose project frequently uses value classes to replicate enums but get around the various exhaustiveness checks that Kotlin performs for backwards compatibility. So we'd like to be able to parameterize tests on those types. We still need to explicitly list all the possible values, which is fine, but right now the runner will just crash.

Can't use more than one `TestParameter` annotation with Android Instrumentation tests.

When I use only one property with TestParameter annotation tests works well.
As soon as I add a 2nd property with TestParameter annotation then all tests fail.

After some investigation I've found that androidx.test.runner.AndroidJUnitRunner.runnerArgs.tests contains incorrect list of test classes:

  • my.test.MyTestClass#testCheckSmth[boolParam1=false
  • boolParam2=false]

And as soon as TestLoader.doCreateRunner() tries to create test class instance from string boolParam2=false] using reflection it fails later.

It is possible that input parameters incorrectly parsed from test args bundle splitting my.test.MyTestClass#testCheckSmth[boolParam1=false,boolParam2=false] by coma.

Test class sample:

// This is instrumentation test, should run on device or emulator
@RunWith(TestParameterInjector::class)
class MyTestClass {

	@TestParameter
	var boolParam1 = false

	@TestParameter
	var boolParam2 = false

	@Test
	fun testCheckSmth() {
	  // test body...
	}
}

When I use the same sample with standard JUnit tests it works well.

Test output:

Starting 4 tests on Android_TV_720p_API_Q(AVD) - Q

boolParam2=false] > [Android_TV_720p_API_Q(AVD) - Q] FAILED 
        java.lang.ClassNotFoundException: Invalid name: boolParam2=false]
        at java.lang.Class.classForName(Native Method)

boolParam2=true] > [Android_TV_720p_API_Q(AVD) - Q] FAILED 
        java.lang.ClassNotFoundException: Invalid name: boolParam2=true]
        at java.lang.Class.classForName(Native Method)

boolParam2=false] > [Android_TV_720p_API_Q(AVD) - Q] FAILED 
        java.lang.ClassNotFoundException: Invalid name: boolParam2=false]
        at java.lang.Class.classForName(Native Method)

boolParam2=true] > [Android_TV_720p_API_Q(AVD) - Q] FAILED 
        java.lang.ClassNotFoundException: Invalid name: boolParam2=true]
        at java.lang.Class.classForName(Native Method)

> Task :app:connectedDebugAndroidTest FAILED

Integrate with RobolectricTestRunner

If we test an Android component(Activity, Fragment, Service...) with Robolectric, we will use RobolectricTestRunner as the test runner.

And a test class only have a test runner.
I also find a way to run a test class within multiple test runners, but I have no idea if it works with TestParameterInjector.
https://bekce.github.io/junit-multiple-runwith-dependent-tests/

Could you have a look about this topic?
Is possible that using TestParameterInjector as a Rule?

Does this work with Powermock libraries?

Hi,
I am having issues when using TestParameterInjector with powermock libraries. To be specific, when I use PowerMockRunnerDelegate, I get the following error:

@RunWith(PowerMockRunner.class)
@PowerMockRunnerDelegate(TestParameterInjector.class)
java.lang.IllegalStateException: Expected exactly one constructor, but got [protected SomeClass(org.powermock.core.IndicateReloadClass), public SomeClass()]

Run a single test in IntelliJ

Hi,
Thank you for this nice runner,
I wanted to know if it is possible to run a test unitary without running whole test class each time ?
Because if I run only 1 test, i encounter the following error :
java.lang.Exception: No tests found matching Method...

Thank you for your reply,
Feel free to close the issue if it will never be possible

Feature request: shade protobuf dependency

We've had some issues with conflicting protobuf dependencies (javalite vs protobuf-lite) due to this dependency and others. Would you be open to shading the protobuf dependency instead?

Running a single test using maven verify

Problem
I am unable to run a single test method using mvn verify.

Scenario
My test class has a combination of parameterized tests (using @TestParameters) and a bunch of other tests that aren't parameterized.
When executing tests for the entire class using -Dit.test=MyClass, all tests (including the parameterized ones execute). But when I run a specific test method using -Dit.test=MyClass#MyParameterizedTestMethod, no tests get executed.

Issue exists when using the @TestParameter annotation as well.

Feature request: Expose `PluggableTestRunner`

We are currently using TestParameterInjector for our snapshot tests. There are some UI configurations that we care for every snapshot tests such as

  • light/dark mode
  • screen sizes
    etc.

It would be awesome if we could implement our own PluggableTestRunner to automatically inject those UI configs for those tests.

Right now, we use @TestParameter/ @TestParameters for light/dark mode, but it is repetitive. Are there any concerns for exposing the class so we could implement our own test runner while preserving the TestParameterInjector behavior?

Test naming changed from 1.8 to 1.9 for `float`s

I had this test:

@RunWith(TestParameterInjector.class)
...

@Test
public void testLightStateOnDevice(
	@TestParameter({"45", "0", "-3", "-6", "-9", "-12", "-15", "-18", "-21"}) float angle,
	@TestParameter Preset preset
) {

I'm using it for screenshot test recording, so the test names between previous and next versions has to match (otherwise hard to compare screenshots :). When updating from TPI 1.8 to 1.9 my PR failed:

java.lang.AssertionError: File src/test/snapshots/images/net.twisterrob.sun.android.logic_SunAngleWidgetViewScreenshotTest_testLightStateOnDevice[45.0,Nice_Preview].png does not exist

when inspecting the golden values on master (1.8) I found:

net.twisterrob.sun.android.logic_SunAngleWidgetViewScreenshotTest_testLightStateOnDevice[45,Nice_Preview].png

Since the only thing changed in the PR was the TPI version number, it's very likely that this had an unintended behavior change:
e390bab#diff-f0f3c84fc90c4da3bd5e1267e7b74d5774029389e86dfe1bba895740b8b886d0R142

How to set custom name for `@TestParameter`

TestParameterInjector will pick the test name from the parameter values using toString().

For my test case

@Test
  fun test1(
    @TestParameter("10% off discounts", "You’re off to see the world!") title: String,
    @TestParameter(
      "You unlocked 10% off your next X bookings. Your discount will be automatically " +
      "applied at checkout.",
      "Get 10% off your next booking."
    ) description: String,
    @TestParameter("Use by Dec 31, 2022", "null") expirationDate: String?,
    @TestParameter("Active", "null") status: String?,
    @TestParameter("Learn more", "null") ctaText: String?,
    @TestParameter canLearnMore: Boolean,
  ) = paparazzi.lazyViewItemSnapshot(
    Item(
      title = UIString(title),
      description = UIString(description),
      stamps = listOf(R.drawable.ic_checkmark, R.drawable.ic_confetti, R.drawable.ic_confetti),
      expirationDate = expirationDate?.let { UIString(it) },
      status = status?.let { UIString(it) },
      canLearnMore = canLearnMore,
      ctaText = ctaText?.let { UIString(it) }
    ),
    isDarkTheme = config.deviceConfig.nightMode == NightMode.NIGHT
  )

So, for above test case, test name generating is to way too long i.e screenshots_SampleScreenshotTest_test1[NEXUS_5_DAY,You’re off to see the world!,You unlocked 10% off your next X bookings. Your discount will be automatically applied at checkout.,Use by Dec 31, 2022,null,Learn more,false].

Im writing a snapshot tests using paparazzi and it uses the test name to generate the file name, due to long file name which is taken from the test name Im getting this error File name too long, and in order to solve this Ive to shorten the test name.

So, here if I want to generate a custom name for @TestParameter annotation how can I do that? In doc I found custom name is available for @TestParameters but not for TestParameter

Please suggestion some solution
Thanks

Support JUnit 5

I quite like the look of this API (well done, @nymanjens!). But I'm a JUnit 5 user, so if I wanted to use this library, I'd have to port it to a JUnit 5 extension, or use a potentially more verbose alternative.

Is this something that's been considered already, by any chance?

Support non-number floating point values

For negative scenario testing and edge cases this would be really useful. Right now to test these we have to write a custom provider.

@TestParameter({"NaN", "-Infinity", "+Infinity", "Infinity"}) float myFloat;
@TestParameter({"NaN", "-Infinity", "+Infinity", "Infinity"}) double myDouble;
Could not map YAML value NaN (class = class java.lang.String) to java class float
java.lang.IllegalArgumentException: Could not map YAML value NaN (class = class java.lang.String) to java class float
	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:453)
	at com.google.testing.junit.testparameterinjector.ParameterValueParsing$YamlValueTransformer.transformedJavaValue(ParameterValueParsing.java:187)
	at com.google.testing.junit.testparameterinjector.ParameterValueParsing.parseYamlObjectToJavaType(ParameterValueParsing.java:146)
	at com.google.testing.junit.testparameterinjector.ParameterValueParsing.parseYamlStringToJavaType(ParameterValueParsing.java:59)
	at com.google.testing.junit.testparameterinjector.TestParameter$InternalImplementationOfThisParameter.parseStringValue(TestParameter.java:194)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.