osgi / osgi-test Goto Github PK

View Code? Open in Web Editor NEW

32.0 11.0 17.0 2.11 MB

Testing support for OSGi. Includes JUnit 4 and JUnit 5 support and AssertJ support.

Home Page: https://osgi.github.io/osgi-test/

License: Apache License 2.0

Java 100.00%

osgi testing test junit junit4 junit5 assertj osgi-testing assertj-support

osgi-test's Introduction

OSGi Testing Support

Testing support for OSGi

Purpose

This project provides a set of bundles which contain useful and helpful classes for testing of OSGi API.

org.osgi.test.common

This artifact includes common utility classes which are useful in all testing scenarios.

org.osgi.test.assertj.*

These artifacts provides support classes for OSGi testing with AssertJ including custom assertions. Currently there are artifacts for org.osgi.framework, org.osgi.util.promise, and org.osgi.service.log.

org.osgi.test.junit4

This artifact provides support classes for OSGi testing with JUnit 4 including JUnit 4 Rules.

org.osgi.test.junit5

This artifact provides support classes for OSGi testing with JUnit 5 including JUnit 5 Extensions.

org.osgi.test.junit5.cm

This artifact provides support classes for OSGi testing with ConfigurationAdmin and JUnit 5 including JUnit 5 Extensions.

org.osgi.test.junit5.listeners.log.osgi

This artifact provides a JUnit 5 TestExecutionListener that logs test output via the OSGi Log Service.

Building

We use Maven to build and the repo includes mvnw. You can use your system mvn but we require a recent version.

./mvnw clean install - Assembles and tests the project

Code Quality

We use CodeQL for continuous security analysis.

Repository

Release versions of osgi-test bundles are available on Maven Central under the org.osgi group ID. You can find them here.

For those who want the bleeding edge, snapshot artifacts are published to the Sonatype OSS repository:

https://oss.sonatype.org/content/repositories/snapshots/

Future work

See the open issues for the list of outstanding TODOs.

License

This program and the accompanying materials are made available under the terms of the Apache License, Version 2.0 which is available at https://www.apache.org/licenses/LICENSE-2.0.

Contributing

Want to hack? There are instructions to get you started.

They are probably not perfect, please let us know if anything feels wrong or incomplete.

Acknowledgments

This project uses the Bnd Maven Plugins to build.

osgi-test's People

Contributors

Stargazers

Watchers

Forkers

rotty3000 kriegfrj stbischof 595385335 erwindl0 datainmotion laeubi timothyjward mause stjordanis kchobantonov bjhargrave isabella232 juergen-albert mypublic-teamwork step-security-bot maho7791

osgi-test's Issues

[assertj] Add assertions for legacy types

OSGi is so old that it has a number of legacy type usages baked deep in its core. Use of the Dictionary abstract base class for properties is a classic example.

AssertJ does not have any direct support for some of these legacy classes, as they are not commonly used in modern code that might be using AssertJ.

We already have utility methods like Dictionaries.asMap() that allow you to build a "shim" wrapper around the dictionary and view it as a Map:

osgi-test/org.osgi.test.common/src/main/java/org/osgi/test/common/dictionary/Dictionaries.java

Lines 44 to 54 in 4bdb06c

 /** 

  * Return a Map wrapper around a Dictionary. 

  * 

  * @param <K> The type of the key. 

  * @param <V> The type of the value. 

  * @param dictionary The dictionary to wrap. 

  * @return A Map object which wraps the specified dictionary. If the 

  * specified dictionary can be cast to a Map, then the specified 

  * dictionary is returned. 

  */ 

 public static <K, V> Map<K, V> asMap(Dictionary<? extends K, ? extends V> dictionary) {

As @bjhargrave noted (https://github.com/osgi/osgi-test/pull/165/files/8f6ce896a19ca13d2aa1125c7f640ebd77206877#r467947029) one of the reasons he created this was so that the rich set of AssertJ Map assertions would be available for testing dictionaries. This allows:

assertThat(Dictionaries.asMap(dictionary)).containsKeys("key1", "key2");

But we could go a step further and create an assertion entry point that can be invoked directly, which would allow this:

assertThat(dictionary).containsKeys("key1", "key2")

Provide way to handle configAdmin configutations via Annotations/Extension

The ConfigurationExtension should help to config the Config-Admin using annotations. I also should inject Fields and Parameters of the org.osgi.service.cm.Configuration Type if they are annotated.

Could you have a look at this Example and give me some feedback

/*
 * Configure the Config-Admin using the annotation `@Configuration`
 * Get or Create an new Configuration Object.
 * Inject the `org.osgi.service.cm.Configuration` (if Parameter or Field) for further updates and delete
 *
 */
@Testable
@ExtendWith(ConfigurationExtension.class)
public class ConfigAnnotationTest2
{

    /*
     *
     * Create or Get a Configuration and us it in a Test
     *
     */

    @Configuration(pid = "my.pid", factory = true, properties = {
            @ConfigEntry(key = "foo", value = "bar") })
    static org.osgi.service.cm.Configuration configuration;

    @Test
    public void testUse() throws IOException
    {
        //TODO: some Tests

        //change Config
        configuration.update(Dictionaries.dictionaryOf("foo", "barbar"));

        //TODO: some Tests

        //delete Config
        configuration.delete();

        //TODO: some Tests
    }
    /*
     *
     * Create or Get @Configuration on for the Test and use the Configuration in the Test
     *
     */
    @Test
    public void updateConfigInTest(
        @Configuration(pid = "my.pid") org.osgi.service.cm.Configuration configuration)
            throws IOException
    {
        //TODO: some Tests
        //change Config
        configuration.update(Dictionaries.dictionaryOf("foo", "barbar"));
        //TODO: some Tests
    }

    /*
     *
     * Configure a service just for this Test
     *
     */
    private final String PID_NAME = "my.pid";

    @Test
    @Configuration(pid = PID_NAME, factory = true, properties = {
            @ConfigEntry(key = "key1", value = "value") })
    public void configMethodInjectService(
        @InjectService(filter = "(%s=%s)", filterArguments = { "service.pid",
                PID_NAME }) Object service)
    {
        service.toString();
    }

[commons] CloseableServiceObjects needs to keep track of use count

Was implementing #116 and wondering why my test kept failing - resources not getting cleaned up when expected. I think it's because I uncovered a real bug.

CloseableServiceObjects keeps track of services returned by getService() in an IdentityHashMap-backed set called instances. Then on close it iterates through them all and calls ungetService() to release them.

However, I discovered using the debugger that getService() internally increments the use count every time you call it, even if it returns the exact same service instance as a previous call. So really, you need to make sure that every getService() is paired with an ungetService(), and not simply call it once per service object.

Should be an easy fix (on its way).

Service testing sometimes need to test what happens when a service is released

In trying this project in an OSGi compliance test, I came across a test case which uses a service to get an object and then needs to release the service to confirm some behavior is performed on the object (closing the object in this case).

But the ServiceExtension support does not provide anyway to release a service. You can inject it as a ServiceAware, but ServiceAware does not provide access to the underlying close method on ServiceConfiguration (which implements ServiceAware).

Perhaps ServiceAware can add a close method to enable tests to release the service early to confirm certain behavior?

take the executorRule from the test harness and make it an official rule

Although it perhaps goes beyond the scope of osgi-test, there is a lot about proper osgi testing that involves concurrency. I think we should consider making these executor-related extensions/rules part of the public API. There are also some concurrency-related utilities I wrote as part of the bnd tester testing that have generic application (eg, waiting for another thread to enter the wait state) that could form part of a concurrency testing library. That can be a topic for another issue/PR though; happy for it to be test-only scope for now.

Originally posted by @kriegfrj in #34

ServiceAware should provide acces to ServiceTrackerCustomizer

Suppose I inject a ServiceAware at the beginning of my test case, and I expect cardinality of 1.

Then, at some point during the test, I create a configuration, which should cause another service of the same type to get activated. How can I set the ServiceAware instance to make it wait till also the second service arrives?
I tried with waitForService, but it returns immediately because one service is already there... (because it is the first one)

It would be great to have a way to wait for a defined count of services. Without loop over getServices and Thread.sleep
List<T> waitForServices(long timeout, int cardinality) throws InterruptedException;

FrameworkExtension to start a new framework on each test

The most Test-Cases expects, that they are running inside an running OSGi-Framework, right?
Is that the only goal of this Project?

PR #64 tries to show a way to run the test also as unit-tests using an framework-extension. This takes the existing Framework(integration-test) or launches a new (unit test).

Is it useful to try out more in this way?

Test BundleActivators by shuffeling the start order of a Bundle

@kriegfrj
sometimes I test (mostly foreign bundles) if the bundle-order matters and the activation/deactivation works fine.
I just shuffle the bundles and restart them several times.
Would it be something we should add to osgi-test?
If Yes, do you have any idea how we can do this?

Just Util
ParameterizedTest that injects a RuntimeDTO on every shuffle Execution
separate Test-Annotation
...?

public class BundleUtil {
	private static List<String> testFrameworkBundle = Arrays.asList("assertj-core", //
		"biz\\.aQute\\.tester.*", //
		"junit-jupiter-.*", //
		"junit-platform-.*", //
		"org\\.mockito\\.mockito-core", //
		"org\\.opentest4j", //
		"org\\.osgi\\.test.*");
	private static Predicate<? super Bundle> frameworkBundle() {
		return bundle -> {
			return bundle.getBundleId() != 0l;
		};
	}
	private static Predicate<? super Bundle> testCaseBundle() {
		return bundle -> {
			return !Dictionaries.asMap(bundle.getHeaders())
				.containsKey("Test-Cases");
		};
	}
	private static Predicate<? super Bundle> testFrameworkBundle() {
		return bundle -> {
			final String bsn = bundle.getSymbolicName();
			for (final String string : testFrameworkBundle) {
				if (bsn.matches(string)) {
					return false;
				}
			}
			return true;
		};
	}
	/**
	 * Shuffles the order of the bundles in the BundleContext, stops and starts
	 * the bundle again.
	 *
	 * @param bundleContext
	 * @param count - shuffle and restart n times
	 * @throws BundleException
	 */
	public static void shuffle(final BundleContext bundleContext, final int count) throws BundleException {
		final List<Bundle> allBundles = Stream.of(bundleContext.getBundles())
			.collect(Collectors.toList());
		shuffle(allBundles, count);
	}
	/**
	 * Shuffles the order of the bundles, stops and starts the bundle again.
	 *
	 * @param bundles - Bundles to be shuffled and restarted
	 * @param count - shuffle and restart n times
	 * @throws BundleException
	 */
	public static void shuffle(final List<Bundle> bundles, final int count) throws BundleException {
		final List<Bundle> shuffleableBundles = bundles.stream()
			.filter(frameworkBundle())
			.filter(testCaseBundle())
			.filter(testFrameworkBundle())
			.collect(Collectors.toList());
		for (int i = 0; i < count; i++) {
			System.out.println(i);
			shuffleFiltered(shuffleableBundles);
		}
	}
	private static void shuffleFiltered(final List<Bundle> bundles) throws BundleException {
		Collections.shuffle(bundles);
		for (final Bundle bundle : bundles) {
			System.out.println(bundle.getSymbolicName());
			bundle.stop();
		}
		Collections.shuffle(bundles);
		for (final Bundle bundle : bundles) {
			bundle.start();
		}
	}

Thank you!

Consider renaming `master` branch to `main` for inclusivity

In comport with being a more inclusive world, we should consider renaming the "main" branch from master to main.

[junit4] ServiceUseRule race condition

The tests for ServiceUseRule are intermittently failing.

After a detailed look at this, I think I have nailed down the cause:

ServiceUse relies on the ServiceTracker's addingService() event to keep track of when services are added.
However, the addingService() event is fired before the service is actually added to the tracker's map:

https://github.com/eclipse/rt.equinox.framework/blob/1f5b39b749d49214e2383c1333cf8603a1da604e/bundles/org.eclipse.osgi/osgi/src/org/osgi/util/tracker/AbstractTracked.java#L255-L269

So when ServiceUses' addingService() method returns, it hasn't actually appeared in the map yet, and so won't be visible in the list of tracked bundles.

The flow of addingService() makes sense - the purpose of a customizer is to have a chance to customize which services are added to the tracker. However, it means it is not that useful for our intended purpose.

Unfortunately the ServiceTracker doesn't seem to have any kind of addedService() event (called after the put() statement), and the object that it is waiting on is private. The only solution I can think of is to poll until tracker.count() == cardinality, but I'm open to other suggestions.

Inject ServiceProperties

Would it be useful for osgi-test to inject the ServiceProperties of an @InjectService annotated Service into an MethodParameter?

Here some Examples:

@ExtendWith(ServiceExtension.class)
public class ServicePropertyInjectionTest {

	@BeforeAll
	public static void beforeAll(@InjectBundleContext BundleContext bundleContext) throws Exception {
		bundleContext.registerService(Foo.class, new Foo() {}, Dictionaries.dictionaryOf("a", "b"));
		bundleContext.registerService(Bar.class, new Bar() {}, Dictionaries.dictionaryOf("c", "d"));
	}

	@Test
	public void testWithServiceAwareWildcardParameter(@InjectService Foo foo, Map<String, Object> fooProperties)
		throws Exception {
		assertThat(foo).isNotNull();
		assertThat(fooProperties).isNotNull()
			.containsEntry("a", "b");
	}

	@Test
	public void testWithServiceAwareWildcardParameter(@InjectService Foo foo, Map<String, Object> fooProperties,
		@InjectService Bar bar, Map<String, Object> barProperties) throws Exception {
		assertThat(foo).isNotNull();
		assertThat(fooProperties).isNotNull()
			.containsEntry("a", "b");

		assertThat(bar).isNotNull();
		assertThat(barProperties).isNotNull()
			.containsEntry("a", "b");
	}

	@Test
	public void testWithServiceAwareWildcardParameter(@InjectService Foo foo, @InjectService Bar bar,
		Map<String, Object> barProperties) throws Exception {
		assertThat(foo).isNotNull();

		assertThat(bar).isNotNull();
		assertThat(barProperties).isNotNull()
			.containsEntry("a", "b");
	}

}

docs: Build a doc website for osgi-test

I think we have enough content now that we need to make a github pages website in docs/ (like we have in Bnd). We don't need a custom domain name (at least not yet).

[junit5] Better PER_CLASS lifecycle support for Jupiter extensions

Following up from #181 (comment):

In Jupiter, there are two lifecycle modes for test classes - one new instance is created per test method (PER_METHOD), or one is created and re-used for all test methods (PER_CLASS). PER_METHOD is the default and historically what we're all used to from JUnit 4. But there are occasions where PER_CLASS is useful. In particular, PER_CLASS allows you to use non-static @BeforeAll and @AfterAll methods, which are useful in situations where you can't use static methods (eg in Kotlin, or for non-static nested test classes).

We have not paid close attention to the implementation of PER_CLASS test lifecycle support in our Jupiter extensions, and in particular to the combination of PER_CLASS lifecycle with CONCURRENT execution (which requires special care). For my part, this has mainly been due to a lack of understanding of PER_CLASS - what it does, and what it's for. But I've just now been through the exercise of implementing field injection for AssertJ's SoftAssertionsExtension (see assertj/assertj#1984) and I think I've hit upon a pattern of implementation for injector-type extensions that we should consider replicating in the osgi-test extensions.

As noted, one of the main concerns with PER_CLASS semantics is the state is shared across the test instance. For the most part in our extensions this is ok as we set/reset the state on every test. The main problem is if you're trying to run PER_CLASS and CONCURRENT, where the tests may be trying to access the same state concurrently.

The way around this is to use ThreadLocal storage for the injected fields, so that each thread accesses its own copy. This fixes the concurrency problem, but it requires a little more work on the part of the extension developer (and in some cases the test developer/extension user) to support it. In SoftAssertionsExtension, I figured out that when you're injecting a type that you've got some control over, it can be possible to inject a delegating implementation in this case, which hides the thread-localisation from the user. In this pattern, you initialise the field with a thread-local-aware delegator when the instance is created, and then when the test is actually run (in beforeEach()) you configure the delegator with the per-test instance.

Of the types that we're currently injecting, we have:

BundleContextExtension: BundleContext, InstallBundle
ServiceExtension: ServiceAware, List<?>, and arbitrary service types.
ConfigurationExtension: Configuration

My assessment of the above is that potentially we could use the thread-local-delegator pattern with nearly all of them, as they are interfaces and we can easily inject a different implementation.

The only difficulty I can see is for for injecting arbitrary service types in ServiceExtension. There are a few possibilities that I can see:

Require injecting into a ThreadLocal<?> instead of the plain service type, and throw an ExtensionConfigurationException if this has not been specified.
Use Mockito, which will enable the creation of a thread-local-delegator for most service types.
A combination of 1 & 2 the above - to avoid a hard dependency on Mockito, use the second approach if Mockito is available and the first if it is not.
A progressive rollout of the above that starts with 1 and then adds 3 later.

I also know that we're looking at a 0.10.0 release soon - I'm not sure if we want to try and squeeze any of this into it or defer it for a later release.

Thoughts?

[common] CloseableBundleContext to handle superclass methods

Was running some tests using the BundleContextRule and found that it blew up when I tried to call toString() on the proxied bundle context.

Looking at the code, it seems that invoke() will only handle method calls for methods declared in the AutoCloseable and BundleContext interfaces. For any other method call, it will throw IllegalArgumentException. This includes method calls for inherited methods, such as those inherited from Object and BundleReference. By a happy coincidence, the only method declared in BundleReference (getBundle()) is also declared in BundleContext (though without the @Override tag - should it have it?), so it still works. However, the methods of Object code would rightly expect to be available too.

For CloseableBundleContext to do the best possible job of masquerading as a true BundleContext, it needs to ensure that it handles all method calls.

Fix to come.

Define OSGi Runtime rules/extensions

There are several OSGi Runtime services ; DS, CDI, Http Whiteboard, JAXRS Whiteboard, etc. that provide features such as DTOs and service.changecount to make it possible to provide a reactive style interaction between a service provider and the extender, whiteboard.

However using these features involves a lot of boilerplate and is sometimes conceptually hard to understand. Providing rules that simplify the task will greatly reduce the time it takes to build good tests.

[junit5] Extensions to handle multiple layers of scope

It would be nice if our Jupiter extensions (eg) BundleContextExtension handled multiple layers of scope in a sensible way.

In a test class, for example, you might have a certain dummy service you wish to install for the life of the class, so that it is available for all tests in that class, and for all nested test classes in that class. Then give each individual method/nested class the ability to install services for the life of that test method/class, etc.

I know that Jupiter's extension model has been designed with this sort of hierarchy in mind. With a few tweaks we could probably leverage it in our extensions.

Release 1.0 wishlist

The idea of this issue is to track discussion of what features should make it into release 1.0 of osgi-test.

From where I'm sitting, I think the biggest area to work on before release 1.0 is additional AssertJ assertion classes for the core OSGi types. Having a quick look through the OSGi Javadoc for R7, I think these are the ones that we should be aiming for (the ones that actually store retrievable state):

In org.osgi.framework:
- Bundle (see #22)
- BundleContext
- BundleReference
- Filter ?
- ServiceReference
- ServiceRegistration
- BundleEvent
- FrameworkEvent
- ServiceEvent
- Version
- VersionRange
All classes in org.osgi.framework.dto.
In org.osgi.framework.launch:
- Framework (a lot of overlap here with Bundle)
In org.osgi.framework.startlevel:
- BundleStartLevel
- FrameworkStartLevel
All in org.osgi.framework.startlevel.dto.
In org.osgi.framework.wiring:
- BundleCapability
- BundleRequirement
- BundleRevision
- BundleRevisions
- BundleWire
- BundleWiriing
- FrameworkWiring
All in org.osgi.framework.wiring.dto.
All the interfaces in org.osgi.resource.
All in org.osgi.resource.dto.
All of org.osgi.service.permissionadmin.
In org.osgi.service.resolver:
- HostedCapability
- ResolveContext
In org.osgi.util.tracker:
- BundleTracker
- ServiceTracker

Whether or not we target all of these or only some for release 1.0 is something up for discussion. I think probably at a minimum for 1.0 we should get org.osgi.framework, org.osgi.util.tracker and org.osgi.framework.launch implemented. Finishing #22 will be a large chunk of this.

assertThat() ambiguity for Hashtables

osgi-test/org.osgi.test.assertj.framework/src/main/java/org/osgi/test/assertj/dictionary/DictionaryAssert.java

Line 44 in 7efdc56

 public static <K, V> DictionaryAssert<K, V> assertThat(Dictionary<K, V> actual) { 

osgi-test/org.osgi.test.assertj.framework/src/main/java/org/osgi/test/assertj/dictionary/DictionarySoftAssertionsProvider.java

Line 30 in 7efdc56

 default <K, V> ProxyableDictionaryAssert<K, V> assertThat(Dictionary<K, V> actual) { 

We have a small problem here: the issue is that the most common concrete type of Dictionary (Hashtable) also implements Map. Because there is an assertThat(Map<K, V>) method in the standard set of assertions, simply calling assertThat(hashtable) is ambiguous for the compiler and you need to cast it to either Map or Dictionary to make it compile.

Possible solutions:

Rename this to assertThatDictionary() to avoid this name collision
Add an assertThat(Hashtable) to DictionaryAssert that can resolve the ambiguity for the compiler

Advantages/disadvantages: the second preserves the assertThat() convention and avoids the user having to remember the special case for Dictionaries. However, the second will not solve the problem for other implementations that inherit Dictionary and implement Map.

Given that there are no public classes other than Hashtable that ~~extend~~ directly extend Dictionary in either the JDK or in OSGi, and it also implements Map, it is the only class in either the JDK or OSGi that will suffer from this issue. I consider this disadvantage of the second approach to be small, so I think I will go with that.

Edit: clarified "extend"

compile warning in org/osgi/test/common/context/CloseableBundleContextTest.java

The CI build shows the following warning:

[WARNING] /home/runner/work/osgi-test/osgi-test/org.osgi.test.common/src/test/java/org/osgi/test/common/context/CloseableBundleContextTest.java:[159,60] unchecked conversion
  required: org.osgi.framework.ServiceReference<java.lang.Object>
  found:    org.osgi.framework.ServiceReference

"TimeWarp" extension

This is a bit of a pie-in-the-sky idea, but I think it could turn into something useful. Just throwing this out there for now in case there are any other ideas about potential design.

I hit a strange case with my test today: I had some code with a bug in it - it was parsing times using "hh:mm:ss" (12h) format when it should have been using "HH:mm:ss" (24h) format. But the parser was forgiving and still accepted hours >=13. So most of the time it worked - the only issue was times between [12:00:00,13:00:00) - they were interpreted as 12am rather than 12pm, which produced the incorrect result. I only discovered the bug by accident when I happened to be running the relevant test around noon.

The main problem with the test design is that it was dependent on an external, uncontrolled factor - the system time. But this was unavoidable because the code-under-test is an expiry algorithm and so it is dependent on system time too. Tests should always try and control the external factors to produce repeatable results.

Completely coincidentally, I got another notification today which reminded me of a feature in Robolectric - its code-weaving classloader modifies code-under-test so that all calls to System.currentTimeMillis() would be routed through itself. This gave it the opportunity to return a different value if desired. It wasn't perfect, but it enabled you to do some simple tests changing the system time on-the-fly.

I believe something similar could be done with OSGi-test using code-weaving hooks. Or possibly using an agent approach to get lower-level access to System.currentTimeMillis().

Issues that would need to be worked out:

Code libraries that are used on both sides of the "fence" that separates test code from code-under-test. I think this could be handled by instrumenting them, and then when the call to System.currentTimeMillis() is intercepted the library could examine the stack trace to see which side of the "fence" the call originates from.
Code in system libraries that cannot be instrumented by OSGi code-weaving hooks - eg, core Java system libraries, OSGi framework. Perhaps a Java agent could be used here instead, I'm not sure how feasible this is.
Interaction with other time-dependent libaries and methods in the JDK (eg, Thread.sleep() and various other functions that have a timeout). A simple implementation of the timewarp extension could focus on System.currentTimeMillis() which would facilitate simpler tests, and a more complex implementation could look at a more comprehensive integration with other system calls later.

I anticipate two components:

The core (in org.osgi.test.common) which provides the time-warping functionality.
A JUnit 5 extension (and if there is sufficient demand, a JUnit 4 rule counterpart) that takes care of installing a time warp at the beginning of each test and removing it at the end.

[junit5] BundleContextExtension using the wrong classloader

@chrisr3 contacted us with a NoClassDefFoundError he was getting using the ServiceExtension in one of his own tests:

java.lang.NoClassDefFoundError: org.osgi.framework.ServiceListener not found by impl-tests [5]
        at com.sun.proxy.$Proxy14.<clinit>(Unknown Source)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:739)
        at org.osgi.test.common.context.CloseableBundleContext.proxy(CloseableBundleContext.java:84)
        at org.osgi.test.junit5.context.BundleContextExtension$CloseableResourceBundleContext.<init>(BundleContextExtension.java:192)

It was a very simple test, and that was actually what exposed the problem - the test doesn't actually import org.osgi.framework - it is simple enough that it doesn't need to. However, when the BundleContextExtension instantiates the CloseableBundleContext, it passes a class from the test bundle to the constructor, which the CloseableBundleContext then uses to get a ClassLoader for building the dynamic-proxied BundleContext. So if the host test bundle doesn't import classes that the proxy class implementation requires, then the dynamic proxy class will fail to load with NoClassDefFoundErrors.

I suspect that we've been getting away with it in our own tests because our own tests are directly importing org.osgi.framework for their own purposes anyway. But @chrisr3's example highlighted a legitimate use case where the test class/bundle may have no direct need to link to org.osgi.framework. I managed to set up a workaround for him simply by creating a reference to the missing class (and hence forcing Bnd to generate an import for it for his own test class).

I think that the solution here is not to use the test bundle's class loader, but perhaps the CloseableBundleContext's own class loader. I know that AssertJ's soft proxying does this, but that's because it must be able to generate dynamic proxies of proxying arbitrary user classes defined in an arbitrary classloaders with arbitrary dependencies. In the case of CloseableBundleContext, all the dependencies that its generated proxies will ever need are known ahead of time, and they should all be available to CloseableBundleContext's class loader.

It will still be important to use the "host" class at the end of the BundleContext delegation chain, so that when the test makes changes to the bundle context it does so to its own bundle context.

Coming up with a regression test for this could be tricky though...

junit4: Intermitted test failure in CI build ServiceUseRuleTest.successWhenServiceWithTimeout

I have seen this failure on several occasions in the CI build. Restarting the build can the succeed. So perhaps there is some timing error here.

[INFO] --- bnd-testing-maven-plugin:4.3.1:testing (testing) @ org.osgi.test.junit4 ---

[INFO] Matching glob *

[INFO] Bnd inferred -runee: JavaSE-1.8

required wires for org.eclipse.osgi_3.15.100.v20191114-1701 [0] []

required wires for org.apache.servicemix.bundles.junit_4.12.0.1 [1] []

required wires for org.assertj.core_3.13.2 [2] []

required wires for org.osgi.test.common_1.0.0.201912190658 [3] []

required wires for org.osgi.test.junit4-test_1.0.0.201912190658 [4] [osgi.wiring.host; filter:="(osgi.wiring.host=org.osgi.test.junit4)" -> osgi.wiring.host; bundle-version:Version="1.0.0.201912190658"; osgi.wiring.host="org.osgi.test.junit4"]

required wires for org.osgi.test.junit4_1.0.0.201912190658 [5] []

required wires for biz.aQute.tester_4.3.1.201911131708 [6] []

required wires for org.osgi.test.junit4-test_1.0.0.201912190658 [4] [osgi.wiring.host; filter:="(osgi.wiring.host=org.osgi.test.junit4)" -> osgi.wiring.host; bundle-version:Version="1.0.0.201912190658"; osgi.wiring.host="org.osgi.test.junit4"]

required wires for org.osgi.test.junit4.tb1_1.0.0.201912190658 [10] []

required wires for org.osgi.test.junit4.tb1_1.0.0.201912190658 [10] []

required wires for org.osgi.test.junit4.tb1_1.0.0.201912190658 [11] []

required wires for org.osgi.test.junit4.tb1_1.0.0.201912190658 [11] []

TEST successWhenServiceWithTimeout(org.osgi.test.junit4.service.ServiceUseRuleTest) <<< ERROR: expected:<null> but was:<[{org.osgi.test.junit4.types.Foo}={service.id=35, service.bundleid=5, service.scope=singleton, foo=bar}]>

org.junit.ComparisonFailure: expected:<null> but was:<[{org.osgi.test.junit4.types.Foo}={service.id=35, service.bundleid=5, service.scope=singleton, foo=bar}]>

	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

	at org.osgi.test.junit4.service.ServiceUseRuleTest.successWhenServiceWithTimeout(ServiceUseRuleTest.java:156)

	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

	at java.base/java.lang.reflect.Method.invoke(Method.java:566)

	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)

	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)

	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)

	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)

	at org.osgi.test.junit4.ExecutorRule$1.evaluate(ExecutorRule.java:28)

	at org.junit.rules.RunRules.evaluate(RunRules.java:20)

	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)

	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)

	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)

	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)

	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)

	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)

	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)

	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)

	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)

	at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:38)

	at junit.framework.TestSuite.runTest(TestSuite.java:252)

	at junit.framework.TestSuite.run(TestSuite.java:247)

	at aQute.junit.Activator.test(Activator.java:326)

	at aQute.junit.Activator.automatic(Activator.java:220)

	at aQute.junit.Activator.run(Activator.java:176)

	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)

	at aQute.launcher.Launcher.launch(Launcher.java:385)

	at aQute.launcher.Launcher.run(Launcher.java:143)

	at aQute.launcher.Launcher.main(Launcher.java:121)

	at aQute.launcher.pre.EmbeddedLauncher.executeWithRunPath(EmbeddedLauncher.java:154)

	at aQute.launcher.pre.EmbeddedLauncher.findAndExecute(EmbeddedLauncher.java:119)

	at aQute.launcher.pre.EmbeddedLauncher.main(EmbeddedLauncher.java:51)

Tests run  : 19

Passed     : 18

Errors     : 1

Failures   : 0

uncaught exceptions in junit5 tests

2020-06-11T17:23:53.1081906Z 15044 [pool-4-thread-1] ERROR o.j.j.e.e.JupiterEngineExecutionContext:148 - Caught exception while closing extension context: org.junit.jupiter.engine.descriptor.MethodExtensionContext@4daf8903
2020-06-11T17:23:53.1083395Z java.lang.IllegalArgumentException: timeout must be zero or greater
2020-06-11T17:23:53.1084815Z 	at org.osgi.test.common.service.ServiceConfiguration.<init>(ServiceConfiguration.java:63)
2020-06-11T17:23:53.1085938Z 	at org.osgi.test.junit5.service.ServiceExtension.lambda$getServiceUseConfiguration$1(ServiceExtension.java:146)
2020-06-11T17:23:53.1087435Z 	at org.junit.jupiter.engine.execution.ExtensionValuesStore.lambda$getOrComputeIfAbsent$0(ExtensionValuesStore.java:81)
2020-06-11T17:23:53.1088786Z 	at org.junit.jupiter.engine.execution.ExtensionValuesStore$MemoizingSupplier.get(ExtensionValuesStore.java:182)
2020-06-11T17:23:53.1090130Z 	at org.junit.jupiter.engine.execution.ExtensionValuesStore.closeAllStoredCloseableValues(ExtensionValuesStore.java:58)
2020-06-11T17:23:53.1091344Z 	at org.junit.jupiter.engine.descriptor.AbstractExtensionContext.close(AbstractExtensionContext.java:73)
2020-06-11T17:23:53.1092549Z 	at org.junit.jupiter.engine.execution.JupiterEngineExecutionContext.close(JupiterEngineExecutionContext.java:53)
2020-06-11T17:23:53.1093709Z 	at org.junit.jupiter.engine.descriptor.JupiterTestDescriptor.cleanUp(JupiterTestDescriptor.java:222)
2020-06-11T17:23:53.1095029Z 	at org.junit.jupiter.engine.descriptor.JupiterTestDescriptor.cleanUp(JupiterTestDescriptor.java:57)
2020-06-11T17:23:53.1096165Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$cleanUp$9(NodeTestTask.java:151)
2020-06-11T17:23:53.1097328Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
2020-06-11T17:23:53.1098440Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.cleanUp(NodeTestTask.java:151)
2020-06-11T17:23:53.1099558Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:83)
2020-06-11T17:23:53.1100747Z 	at java.util.ArrayList.forEach(ArrayList.java:1257)
2020-06-11T17:23:53.1103464Z 	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
2020-06-11T17:23:53.1104532Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
2020-06-11T17:23:53.1105816Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
2020-06-11T17:23:53.1111818Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
2020-06-11T17:23:53.1118520Z 	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
2020-06-11T17:23:53.1127813Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
2020-06-11T17:23:53.1135743Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
2020-06-11T17:23:53.1144203Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
2020-06-11T17:23:53.1151598Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
2020-06-11T17:23:53.1155854Z 	at java.util.ArrayList.forEach(ArrayList.java:1257)
2020-06-11T17:23:53.1167340Z 	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
2020-06-11T17:23:53.1189751Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
2020-06-11T17:23:53.1198800Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
2020-06-11T17:23:53.1208018Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
2020-06-11T17:23:53.1214900Z 	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
2020-06-11T17:23:53.1245641Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
2020-06-11T17:23:53.1255434Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
2020-06-11T17:23:53.1263975Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
2020-06-11T17:23:53.1271678Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
2020-06-11T17:23:53.1283106Z 	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
2020-06-11T17:23:53.1293874Z 	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
2020-06-11T17:23:53.1302234Z 	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
2020-06-11T17:23:53.1309781Z 	at org.junit.platform.testkit.engine.EngineTestKit.execute(EngineTestKit.java:189)
2020-06-11T17:23:53.1316787Z 	at org.junit.platform.testkit.engine.EngineTestKit.access$100(EngineTestKit.java:54)
2020-06-11T17:23:53.1325837Z 	at org.junit.platform.testkit.engine.EngineTestKit$Builder.execute(EngineTestKit.java:343)
2020-06-11T17:23:53.1335766Z 	at org.osgi.test.junit5.service.AbstractServiceExtensionTest.assertThatTest(AbstractServiceExtensionTest.java:126)
2020-06-11T17:23:53.1347349Z 	at org.osgi.test.junit5.service.AbstractServiceExtensionTest.lambda$futureAssertThatTest$0(AbstractServiceExtensionTest.java:113)
2020-06-11T17:23:53.1353191Z 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
2020-06-11T17:23:53.1362322Z 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
2020-06-11T17:23:53.1372502Z 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
2020-06-11T17:23:53.1380238Z 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
2020-06-11T17:23:53.1387473Z 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
2020-06-11T17:23:53.1391941Z 	at java.lang.Thread.run(Thread.java:748)
2020-06-11T17:23:53.1809645Z 15121 [pool-7-thread-1] ERROR o.j.j.e.e.JupiterEngineExecutionContext:148 - Caught exception while closing extension context: org.junit.jupiter.engine.descriptor.MethodExtensionContext@1df1ebcc
2020-06-11T17:23:53.1811365Z java.lang.IllegalArgumentException: cardinality must be zero or greater
2020-06-11T17:23:53.1812232Z 	at org.osgi.test.common.service.ServiceConfiguration.<init>(ServiceConfiguration.java:58)
2020-06-11T17:23:53.1814089Z 	at org.osgi.test.junit5.service.ServiceExtension.lambda$getServiceUseConfiguration$1(ServiceExtension.java:146)
2020-06-11T17:23:53.1815141Z 	at org.junit.jupiter.engine.execution.ExtensionValuesStore.lambda$getOrComputeIfAbsent$0(ExtensionValuesStore.java:81)
2020-06-11T17:23:53.1816131Z 	at org.junit.jupiter.engine.execution.ExtensionValuesStore$MemoizingSupplier.get(ExtensionValuesStore.java:182)
2020-06-11T17:23:53.1817124Z 	at org.junit.jupiter.engine.execution.ExtensionValuesStore.closeAllStoredCloseableValues(ExtensionValuesStore.java:58)
2020-06-11T17:23:53.1818093Z 	at org.junit.jupiter.engine.descriptor.AbstractExtensionContext.close(AbstractExtensionContext.java:73)
2020-06-11T17:23:53.1819375Z 	at org.junit.jupiter.engine.execution.JupiterEngineExecutionContext.close(JupiterEngineExecutionContext.java:53)
2020-06-11T17:23:53.1820249Z 	at org.junit.jupiter.engine.descriptor.JupiterTestDescriptor.cleanUp(JupiterTestDescriptor.java:222)
2020-06-11T17:23:53.1821131Z 	at org.junit.jupiter.engine.descriptor.JupiterTestDescriptor.cleanUp(JupiterTestDescriptor.java:57)
2020-06-11T17:23:53.1822009Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$cleanUp$9(NodeTestTask.java:151)
2020-06-11T17:23:53.1823249Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
2020-06-11T17:23:53.1824170Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.cleanUp(NodeTestTask.java:151)
2020-06-11T17:23:53.1825251Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:83)
2020-06-11T17:23:53.1826044Z 	at java.util.ArrayList.forEach(ArrayList.java:1257)
2020-06-11T17:23:53.1827005Z 	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
2020-06-11T17:23:53.1828039Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
2020-06-11T17:23:53.1829011Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
2020-06-11T17:23:53.1829959Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
2020-06-11T17:23:53.1830833Z 	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
2020-06-11T17:23:53.1831769Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
2020-06-11T17:23:53.1832719Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
2020-06-11T17:23:53.1833830Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
2020-06-11T17:23:53.1834714Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
2020-06-11T17:23:53.1835486Z 	at java.util.ArrayList.forEach(ArrayList.java:1257)
2020-06-11T17:23:53.1836386Z 	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
2020-06-11T17:23:53.1837396Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
2020-06-11T17:23:53.1838467Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
2020-06-11T17:23:53.1839608Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
2020-06-11T17:23:53.1840460Z 	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
2020-06-11T17:23:53.1841346Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
2020-06-11T17:23:53.1842536Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
2020-06-11T17:23:53.1843405Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
2020-06-11T17:23:53.1844269Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
2020-06-11T17:23:53.1845262Z 	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
2020-06-11T17:23:53.1846237Z 	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
2020-06-11T17:23:53.1920709Z 	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
2020-06-11T17:23:53.1921689Z 	at org.junit.platform.testkit.engine.EngineTestKit.execute(EngineTestKit.java:189)
2020-06-11T17:23:53.1922629Z 	at org.junit.platform.testkit.engine.EngineTestKit.access$100(EngineTestKit.java:54)
2020-06-11T17:23:53.1923821Z 	at org.junit.platform.testkit.engine.EngineTestKit$Builder.execute(EngineTestKit.java:343)
2020-06-11T17:23:53.1924729Z 	at org.osgi.test.junit5.service.AbstractServiceExtensionTest.assertThatTest(AbstractServiceExtensionTest.java:126)
2020-06-11T17:23:53.1925878Z 	at org.osgi.test.junit5.service.AbstractServiceExtensionTest.lambda$futureAssertThatTest$0(AbstractServiceExtensionTest.java:113)
2020-06-11T17:23:53.1946280Z 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
2020-06-11T17:23:53.1947273Z 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
2020-06-11T17:23:53.1948272Z 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
2020-06-11T17:23:53.1949132Z 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
2020-06-11T17:23:53.1949960Z 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
2020-06-11T17:23:53.1950679Z 	at java.lang.Thread.run(Thread.java:748)
2020-06-11T17:23:53.6420972Z 15587 [ool-11-thread-1] ERROR o.j.j.e.e.JupiterEngineExecutionContext:148 - Caught exception while closing extension context: org.junit.jupiter.engine.descriptor.MethodExtensionContext@7100b8f1
2020-06-11T17:23:53.6422731Z org.osgi.framework.InvalidSyntaxException: Filter ended abruptly: (&(objectClass=org.osgi.test.junit5.types.Foo)(foo=baz)
2020-06-11T17:23:53.6423893Z 	at org.eclipse.osgi.internal.framework.FilterImpl$Parser.parse(FilterImpl.java:1378)
2020-06-11T17:23:53.6424933Z 	at org.eclipse.osgi.internal.framework.FilterImpl.newInstance(FilterImpl.java:180)
2020-06-11T17:23:53.6425937Z 	at org.eclipse.osgi.internal.framework.FilterImpl.newInstance(FilterImpl.java:176)
2020-06-11T17:23:53.6427372Z 	at org.osgi.framework.FrameworkUtil.createFilter(FrameworkUtil.java:72)
2020-06-11T17:23:53.6428342Z 	at org.osgi.test.common.exceptions.FunctionWithException.lambda$orElseThrow$0(FunctionWithException.java:21)
2020-06-11T17:23:53.6429221Z 	at org.osgi.test.common.filter.Filters.format(Filters.java:43)
2020-06-11T17:23:53.6430093Z 	at org.osgi.test.common.service.ServiceConfiguration.<init>(ServiceConfiguration.java:53)
2020-06-11T17:23:53.6431460Z 	at org.osgi.test.junit5.service.ServiceExtension.lambda$getServiceUseConfiguration$1(ServiceExtension.java:146)
2020-06-11T17:23:53.6432777Z 	at org.junit.jupiter.engine.execution.ExtensionValuesStore.lambda$getOrComputeIfAbsent$0(ExtensionValuesStore.java:81)
2020-06-11T17:23:53.6433911Z 	at org.junit.jupiter.engine.execution.ExtensionValuesStore$MemoizingSupplier.get(ExtensionValuesStore.java:182)
2020-06-11T17:23:53.6435472Z 	at org.junit.jupiter.engine.execution.ExtensionValuesStore.closeAllStoredCloseableValues(ExtensionValuesStore.java:58)
2020-06-11T17:23:53.6436612Z 	at org.junit.jupiter.engine.descriptor.AbstractExtensionContext.close(AbstractExtensionContext.java:73)
2020-06-11T17:23:53.6437918Z 	at org.junit.jupiter.engine.execution.JupiterEngineExecutionContext.close(JupiterEngineExecutionContext.java:53)
2020-06-11T17:23:53.6439091Z 	at org.junit.jupiter.engine.descriptor.JupiterTestDescriptor.cleanUp(JupiterTestDescriptor.java:222)
2020-06-11T17:23:53.6440262Z 	at org.junit.jupiter.engine.descriptor.JupiterTestDescriptor.cleanUp(JupiterTestDescriptor.java:57)
2020-06-11T17:23:53.6441534Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$cleanUp$9(NodeTestTask.java:151)
2020-06-11T17:23:53.6442832Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
2020-06-11T17:23:53.6443891Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.cleanUp(NodeTestTask.java:151)
2020-06-11T17:23:53.6445548Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:83)
2020-06-11T17:23:53.6446707Z 	at java.util.ArrayList.forEach(ArrayList.java:1257)
2020-06-11T17:23:53.6451087Z 	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
2020-06-11T17:23:53.6454086Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
2020-06-11T17:23:53.6456329Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
2020-06-11T17:23:53.6460064Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
2020-06-11T17:23:53.6461396Z 	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
2020-06-11T17:23:53.6464189Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
2020-06-11T17:23:53.6466732Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
2020-06-11T17:23:53.6469656Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
2020-06-11T17:23:53.6472310Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
2020-06-11T17:23:53.6473972Z 	at java.util.ArrayList.forEach(ArrayList.java:1257)
2020-06-11T17:23:53.6478322Z 	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
2020-06-11T17:23:53.6481642Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
2020-06-11T17:23:53.6484175Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
2020-06-11T17:23:53.6487086Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
2020-06-11T17:23:53.6489080Z 	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
2020-06-11T17:23:53.6493654Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
2020-06-11T17:23:53.6527240Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
2020-06-11T17:23:53.6529803Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
2020-06-11T17:23:53.6532661Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
2020-06-11T17:23:53.6536534Z 	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
2020-06-11T17:23:53.6539352Z 	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
2020-06-11T17:23:53.6542320Z 	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
2020-06-11T17:23:53.6544245Z 	at org.junit.platform.testkit.engine.EngineTestKit.execute(EngineTestKit.java:189)
2020-06-11T17:23:53.6546908Z 	at org.junit.platform.testkit.engine.EngineTestKit.access$100(EngineTestKit.java:54)
2020-06-11T17:23:53.6550043Z 	at org.junit.platform.testkit.engine.EngineTestKit$Builder.execute(EngineTestKit.java:343)
2020-06-11T17:23:53.6552658Z 	at org.osgi.test.junit5.service.AbstractServiceExtensionTest.assertThatTest(AbstractServiceExtensionTest.java:126)
2020-06-11T17:23:53.6556662Z 	at org.osgi.test.junit5.service.AbstractServiceExtensionTest.lambda$futureAssertThatTest$0(AbstractServiceExtensionTest.java:113)
2020-06-11T17:23:53.6558905Z 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
2020-06-11T17:23:53.6561702Z 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
2020-06-11T17:23:53.6564965Z 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
2020-06-11T17:23:53.6567344Z 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
2020-06-11T17:23:53.6569511Z 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
2020-06-11T17:23:53.6571205Z 	at java.lang.Thread.run(Thread.java:748)
2020-06-11T17:23:53.6573020Z Caused by: java.lang.ArrayIndexOutOfBoundsException: 55
2020-06-11T17:23:53.6576495Z 	at org.eclipse.osgi.internal.framework.FilterImpl$Parser.parse_and(FilterImpl.java:1445)
2020-06-11T17:23:53.6579165Z 	at org.eclipse.osgi.internal.framework.FilterImpl$Parser.parse_filtercomp(FilterImpl.java:1420)
2020-06-11T17:23:53.6581901Z 	at org.eclipse.osgi.internal.framework.FilterImpl$Parser.parse_filter(FilterImpl.java:1397)
2020-06-11T17:23:53.6584567Z 	at org.eclipse.osgi.internal.framework.FilterImpl$Parser.parse(FilterImpl.java:1376)
2020-06-11T17:23:53.6585347Z 	... 52 common frames omitted

I am not sure if these are expected or an error. If expected, we should try to get them not to appear in the build output.

Refactor Configuration reset behavior

As discussed today with @stbischof:

The Test tries to reset Configurations to a state before the test started. This currently happens by a best guess approach. A better solution could look as follows:

Take a snapshot of the Configurations the ConfigurationAdmin knows about before a test starts.
Record all the ConfigurationsEvents that appear during a test runs
Try to reverse them in the order received.

While writing this down and thinking about it, I came to the point that this is error prone as well, because this would catch all configuration changes that happen during a test and not only the changes that someone directly does in the a test. Instead of magically trying to catch a ConfigurationEvents and reverting them, I believe we should only revert changes a test causes directly by interacting with the ConfigurationAdmin.
An Example why: We have the real case, where we have a configurable Component that in turn creates/updates a couple of other configurations if it receives a configuration. It also takes care of cleanup and changes. If this would happen during a test, the reset mechanism could in theory mess up the build in mechanism to handle configurations handled by this component. (Yes, one could argue that it should be robust enough to handle external manipulation of configurations, but no).

In our tests, we have one central point we use to handle configurations, so we can clean up what we have done after the test has finished. Maybe here something different is possible, by providing a change aware ConfigurationAdmin. This could be a complete Wrapper around the actual Service that records what the user does and that can be reverted afterwards.

javadoc warnings in maven build

The CI build shows the following javadoc warnings:

[WARNING] Javadoc Warnings
[WARNING] /home/runner/work/osgi-test/osgi-test/org.osgi.test.common/src/main/java/org/osgi/test/common/annotation/InjectService.java:76: warning: no @return
[WARNING] int cardinality() default 1;
[WARNING] ^
[WARNING] /home/runner/work/osgi-test/osgi-test/org.osgi.test.common/src/main/java/org/osgi/test/common/annotation/InjectService.java:65: warning: no @return
[WARNING] String filter() default "";
[WARNING] ^
[WARNING] /home/runner/work/osgi-test/osgi-test/org.osgi.test.common/src/main/java/org/osgi/test/common/annotation/InjectService.java:70: warning: no @return
[WARNING] String[] filterArguments() default {};
[WARNING] ^
[WARNING] /home/runner/work/osgi-test/osgi-test/org.osgi.test.common/src/main/java/org/osgi/test/common/annotation/InjectService.java:81: warning: no @return
[WARNING] long timeout() default DEFAULT_TIMEOUT;
[WARNING] ^
[WARNING] /home/runner/work/osgi-test/osgi-test/org.osgi.test.common/src/main/java/org/osgi/test/common/install/InstallBundle.java:47: warning: no description for @param
[WARNING] * @param pathToEmbeddedJar
[WARNING] ^
[WARNING] /home/runner/work/osgi-test/osgi-test/org.osgi.test.common/src/main/java/org/osgi/test/common/install/InstallBundle.java:66: warning: no description for @param
[WARNING] * @param pathToEmbeddedJar
[WARNING] ^

and

[WARNING] Javadoc Warnings
[WARNING] /home/runner/work/osgi-test/osgi-test/org.osgi.test.assertj/src/main/java/org/osgi/test/assertj/promise/PromiseAssert.java:66: warning: no @param for <ACTUAL>
[WARNING] public static <ACTUAL extends Promise<? extends RESULT>, RESULT> InstanceOfAssertFactory<ACTUAL, PromiseAssert<RESULT>> promise(
[WARNING] ^

assertj assertions for org.osgi.framework.Bundle

I can see this pattern occurring frequently in your tests. I've used similar patterns a lot in my own tests.

If we had org.osgi.test.assertj up-to-date, then the above could simplify to (eg) assertThat(installedBundle).isInState(Bundle.UNINSTALLED). I can see this pattern getting used a lot so perhaps we should look at an AssertJ assertion class for Bundle sooner rather than later.

Originally posted by @kriegfrj in https://github.com/osgi/osgi-test/diffs

[assertj] Soft assertions proxy generating references to non-imported packages

I started trying to use the newly-added soft proxies implementation of the Bundle, BundleEvent and BundleContext assertions. However, I found that the dynamically-generated soft assertions proxy classes have references to AssertJ core classes that are not statically referenced in the class files themselves. This means that bnd doesn't include them in Import-Package and the soft assertion class fails to load at runtime with NoClassDefFoundError. In my case, it was org.assertj.core.api.recursive.comparison.RecursiveComparisonConfiguration.

Possible solutions that I see:

Manually add Import-Package filter to cover all of the core assertj packages (possibly as optional imports).
Ditto but use DynamicImport-Package
A Bnd plugin that can analyze the class files to find all instances of proxy generation invocation, run them, and examine the dependencies of the generated classes, and include those in the import-package analysis (Rolls-Royce!)

I tried the first two but only succeeded in getting rid of the default Import-Package statement. I tried using the syntax Import-Package: org.assertj.core.api.recursive.comparison, * only succeeded in ending up with an Import-Package statement that included only org.assertj.core.api.recursive.comparison. Ditto - when I tried DynamicImport-Package: *, the Import-Package statement disappeared completely.

bom is broken

The bom project just added seems rather broken.

When I build from the command line

[INFO] ---------------------< org.osgi:org.osgi.test.bom >---------------------
[INFO] Building OSGi-Test Bill of Materials ${revision}                   [8/8]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ org.osgi.test.bom ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] org.osgi:org.osgi.test.parent 0.11.0-SNAPSHOT ...... SUCCESS [  1.627 s]
[INFO] org.osgi:org.osgi.test.common 0.11.0-SNAPSHOT ...... SUCCESS [ 11.846 s]
[INFO] org.osgi:org.osgi.test.assertj.framework 0.11.0-SNAPSHOT SUCCESS [  7.091 s]
[INFO] org.osgi:org.osgi.test.assertj.promise 0.11.0-SNAPSHOT SUCCESS [  7.939 s]
[INFO] org.osgi:org.osgi.test.junit4 0.11.0-SNAPSHOT ...... SUCCESS [  4.440 s]
[INFO] org.osgi:org.osgi.test.junit5 0.11.0-SNAPSHOT ...... SUCCESS [ 12.422 s]
[INFO] org.osgi:org.osgi.test.junit5.cm 0.11.0-SNAPSHOT ... SUCCESS [ 13.901 s]
[INFO] OSGi-Test Bill of Materials ${revision} ............ SUCCESS [  0.001 s]

The ${revision} in the output seems like something bad. The bom does not reference the parent project and so does not know what any of the properties like revision are. Also, don't we need some tests?

It seems like this was merged prematurely.

[junit5] org.osgi.test.junit5.bundle is not exported

This was a bit of an oversight on our part. We have useful utilities in here that can't be used because we didn't actually export the package.

This went undetected because our test bundle is implemented as a fragment and had special access rights to that package which our client bundles will not have. One of the reasons to prefer implementing tests in standalone bundles rather than as fragments.

@InjectBundleContext lifecycle problems

Background

The "normal" way to use the bnd-testing-maven-plugin is to package up all the unit tests and OSGi tests in a single test jar which is a fragment of the bundle being tested. This allows the unit tests to be in the same package(s) as the classes under test and therefore default visibility methods can be tested. The consequence of this is that in the OSGi tests the BundleContext for the tester bundle is the same as for the bundle being tested.

The Problem

In some of my OSGi tests I need to ensure a "clean" start of the bundle under test to avoid residual state (e.g. historical monitoring data). The simplest way to do this is to restart the bundle under test. This invalidates the BundleContext injected into my test and creates a new one. To ensure that I don't use a stale BundleContext I inject a new one into my test class using an @BeforeEach method, unfortunately the object my test is injected with appears to be the old invalid BundleContext and throws Exceptions when I try to use it in the next test.

Proposed solution

Each time the BundleContext is injected into a lifecycle it should be obtained from the tester bundle, and not cached and reused.

[junit4] Define an OSGi service rule

Depending on OSGi services in tests can be tricky, with a nice JUnit4 rule/JUnit5 extension that eliminates the boiler plate and helps assert the invariants the problem can be greatly simplified.

[assertj] Integration testing

Maybe we should also be doing full in-framework integration testing of our AssertJ bundle to catch these.

That is probably a necessary thing since the purpose of these is to use for in-framework integration testing. We should be confident that it works well.

Originally posted by @bjhargrave in #118 (comment)

I just got bitten by this again when I added a package to commons (org.osgi.test.common.bitmaps) that I then referenced in assertj - the unit test compiled and ran happily, but then when the junit5 integration test attempted to resolve the assertj bundle it failed because new package wasn't exported. Given that our full integration tests are still pretty fast and the IDE integration is still good, I see no harm with changing the unit tests to run as integration tests which will help to catch issues like these.

Split org.osgi.test.assertj into org.osgi.test.assertj.framework and org.osgi.test.assertj.promise

Having PromiseAssert in the bundle means all users must provide the promise/function packages. This is unfortunate in the OSGi compliance tests for those test which are not otherwise touching promise.

Since we may make more custom assertj asserts for other packages, we should probably have a unique artifacts for each package.

-

wrong project

Register Jupiter extensions globally (discussion)

I've been thinking a little about the usage of our extensions.

All of our extensions are quite non-invasive. They won't do a lot if you haven't explicitly annotated your tests with the relevant @Injectxxx annotation(s).

Perhaps we should consider registering the Jupiter extensions globally? This should be possible simply by registering them as OSGi services with the Extension component type. In this way, you could dispense with the @ExtendWith annotation - it will be sufficient to use the @Injectxxx annotations.

The only downside to this (that I can see) is the extra execution overhead - each extension needs to run a pass over the fields in the test class to see if any have fields that it is interested in. If your class doesn't use the extension then that is extra overhead you could otherwise avoid.

Thoughts?

BundleContextExtension does not check for null

If the bundles are not started that are used to acquire the bundle-context then null is used internally leading to NPE in BundleContextExtension.

ServiceUseExtensionTest suffers from intermittent failure

CI builds regularly fail due to some timing bug in ServiceUseExtensionTest or the ServiceUseExtension the test is testing.

We need to solve this before any release.

2020-04-03T07:00:17.5195056Z [INFO] --- bnd-testing-maven-plugin:5.0.1:testing (testing) @ org.osgi.test.junit5 ---
2020-04-03T07:00:17.5253098Z [INFO] Matching glob *
2020-04-03T07:00:17.5343099Z [INFO] Bnd inferred -runee: JavaSE-1.8
2020-04-03T07:00:22.5700497Z TEST org.osgi.test.junit5.service.ServiceUseExtensionTest#successWhenServiceWithTimeout() <<< ERROR: 
2020-04-03T07:00:22.5701228Z The following assertion failed:
2020-04-03T07:00:22.5704550Z 1) [Services [{org.osgi.test.junit5.types.Foo}={service.id=36, service.bundleid=14, service.scope=singleton, case=successWhenServiceWithTimeout}]] 
2020-04-03T07:00:22.5704896Z Expecting:
2020-04-03T07:00:22.5705161Z  <2>
2020-04-03T07:00:22.5705450Z to be equal to:
2020-04-03T07:00:22.5705706Z  <1>
2020-04-03T07:00:22.5705946Z but was not.
2020-04-03T07:00:22.5707339Z at ServiceUseExtensionTest.successWhenServiceWithTimeout(ServiceUseExtensionTest.java:211)
2020-04-03T07:00:22.5760010Z 
2020-04-03T07:00:22.5763277Z org.assertj.core.api.SoftAssertionError: 
2020-04-03T07:00:22.5763884Z The following assertion failed:
2020-04-03T07:00:22.5766862Z 1) [Services [{org.osgi.test.junit5.types.Foo}={service.id=36, service.bundleid=14, service.scope=singleton, case=successWhenServiceWithTimeout}]] 
2020-04-03T07:00:22.5767226Z Expecting:
2020-04-03T07:00:22.5767526Z  <2>
2020-04-03T07:00:22.5767790Z to be equal to:
2020-04-03T07:00:22.5768083Z  <1>
2020-04-03T07:00:22.5768345Z but was not.
2020-04-03T07:00:22.5768657Z at ServiceUseExtensionTest.successWhenServiceWithTimeout(ServiceUseExtensionTest.java:211)
2020-04-03T07:00:22.5773562Z 
2020-04-03T07:00:22.5803562Z 	at org.assertj.core.error.AssertionErrorCreator.multipleSoftAssertionsError(AssertionErrorCreator.java:75)
2020-04-03T07:00:22.5804029Z 	at org.assertj.core.api.SoftAssertions.assertAll(SoftAssertions.java:166)
2020-04-03T07:00:22.5810325Z 	at org.osgi.test.junit5.service.ServiceUseExtensionTest.successWhenServiceWithTimeout(ServiceUseExtensionTest.java:225)
2020-04-03T07:00:22.5811224Z 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2020-04-03T07:00:22.5811733Z 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2020-04-03T07:00:22.5812114Z 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2020-04-03T07:00:22.5812599Z 	at java.lang.reflect.Method.invoke(Method.java:498)
2020-04-03T07:00:22.5824576Z 	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:515)
2020-04-03T07:00:22.5825198Z 	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:115)
2020-04-03T07:00:22.5825709Z 	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:171)
2020-04-03T07:00:22.5826032Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72)
2020-04-03T07:00:22.5826646Z 	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:167)
2020-04-03T07:00:22.5827017Z 	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:114)
2020-04-03T07:00:22.5828240Z 	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:59)
2020-04-03T07:00:22.5832055Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:105)
2020-04-03T07:00:22.5868638Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72)
2020-04-03T07:00:22.5870120Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:95)
2020-04-03T07:00:22.5872199Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:71)
2020-04-03T07:00:22.5873141Z 	at java.util.ArrayList.forEach(ArrayList.java:1257)
2020-04-03T07:00:22.5876731Z 	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
2020-04-03T07:00:22.5878984Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:110)
2020-04-03T07:00:22.5881491Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72)
2020-04-03T07:00:22.5885064Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:95)
2020-04-03T07:00:22.5886878Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:71)
2020-04-03T07:00:22.5889406Z 	at java.util.ArrayList.forEach(ArrayList.java:1257)
2020-04-03T07:00:22.5893088Z 	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
2020-04-03T07:00:22.5895251Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:110)
2020-04-03T07:00:22.5897357Z 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72)
2020-04-03T07:00:22.5899267Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:95)
2020-04-03T07:00:22.5901145Z 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:71)
2020-04-03T07:00:22.5905044Z 	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
2020-04-03T07:00:22.5907136Z 	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
2020-04-03T07:00:22.5909367Z 	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
2020-04-03T07:00:22.5911781Z 	at aQute.tester.bundle.engine.BundleDescriptor.executeChild(BundleDescriptor.java:49)
2020-04-03T07:00:22.5914208Z 	at aQute.tester.bundle.engine.BundleEngine.lambda$executeBundle$7(BundleEngine.java:120)
2020-04-03T07:00:22.5916212Z 	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
2020-04-03T07:00:22.5918063Z 	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
2020-04-03T07:00:22.5919119Z 	at java.util.Iterator.forEachRemaining(Iterator.java:116)
2020-04-03T07:00:22.5921145Z 	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
2020-04-03T07:00:22.5922953Z 	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
2020-04-03T07:00:22.5924919Z 	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
2020-04-03T07:00:22.5926422Z 	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
2020-04-03T07:00:22.5928071Z 	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
2020-04-03T07:00:22.5929383Z 	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
2020-04-03T07:00:22.5931674Z 	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
2020-04-03T07:00:22.5933340Z 	at aQute.tester.bundle.engine.BundleEngine.executeBundle(BundleEngine.java:120)
2020-04-03T07:00:22.5935934Z 	at aQute.tester.bundle.engine.BundleEngine.lambda$executeBundle$8(BundleEngine.java:133)
2020-04-03T07:00:22.5937185Z 	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
2020-04-03T07:00:22.5938664Z 	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
2020-04-03T07:00:22.5940049Z 	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
2020-04-03T07:00:22.5941739Z 	at java.util.Iterator.forEachRemaining(Iterator.java:116)
2020-04-03T07:00:22.5943505Z 	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
2020-04-03T07:00:22.5945079Z 	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
2020-04-03T07:00:22.5946541Z 	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
2020-04-03T07:00:22.6039062Z 	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
2020-04-03T07:00:22.6041598Z 	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
2020-04-03T07:00:22.6043013Z 	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
2020-04-03T07:00:22.6044491Z 	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
2020-04-03T07:00:22.6046289Z 	at aQute.tester.bundle.engine.BundleEngine.executeBundle(BundleEngine.java:133)
2020-04-03T07:00:22.6047895Z 	at aQute.tester.bundle.engine.BundleEngine.lambda$execute$5(BundleEngine.java:100)
2020-04-03T07:00:22.6049353Z 	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
2020-04-03T07:00:22.6050892Z 	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
2020-04-03T07:00:22.6052377Z 	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
2020-04-03T07:00:22.6053511Z 	at java.util.Iterator.forEachRemaining(Iterator.java:116)
2020-04-03T07:00:22.6055386Z 	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
2020-04-03T07:00:22.6056797Z 	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
2020-04-03T07:00:22.6058233Z 	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
2020-04-03T07:00:22.6059786Z 	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
2020-04-03T07:00:22.6061395Z 	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
2020-04-03T07:00:22.6062758Z 	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
2020-04-03T07:00:22.6064214Z 	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
2020-04-03T07:00:22.6065472Z 	at aQute.tester.bundle.engine.BundleEngine.execute(BundleEngine.java:100)
2020-04-03T07:00:22.6067141Z 	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:220)
2020-04-03T07:00:22.6069140Z 	at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:188)
2020-04-03T07:00:22.6071025Z 	at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:202)
2020-04-03T07:00:22.6072723Z 	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:181)
2020-04-03T07:00:22.6074566Z 	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
2020-04-03T07:00:22.6075684Z 	at aQute.tester.junit.platform.Activator.test(Activator.java:352)
2020-04-03T07:00:22.6076902Z 	at aQute.tester.junit.platform.Activator.automatic(Activator.java:317)
2020-04-03T07:00:22.6078044Z 	at aQute.tester.junit.platform.Activator.run(Activator.java:217)
2020-04-03T07:00:22.6079457Z 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
2020-04-03T07:00:22.6080589Z 	at aQute.launcher.Launcher.launch(Launcher.java:412)
2020-04-03T07:00:22.6081548Z 	at aQute.launcher.Launcher.run(Launcher.java:170)
2020-04-03T07:00:22.6082712Z 	at aQute.launcher.Launcher.main(Launcher.java:148)
2020-04-03T07:00:22.6085020Z 	at aQute.launcher.pre.EmbeddedLauncher.executeWithRunPath(EmbeddedLauncher.java:154)
2020-04-03T07:00:22.6190393Z 	at aQute.launcher.pre.EmbeddedLauncher.findAndExecute(EmbeddedLauncher.java:119)
2020-04-03T07:00:22.6191016Z 	at aQute.launcher.pre.EmbeddedLauncher.main(EmbeddedLauncher.java:48)
2020-04-03T07:00:22.8481726Z 
2020-04-03T07:00:22.8482490Z Test run finished after 2742 ms
2020-04-03T07:00:22.8483390Z [         9 containers found      ]
2020-04-03T07:00:22.8484056Z [         0 containers skipped    ]
2020-04-03T07:00:22.8484506Z [         9 containers started    ]
2020-04-03T07:00:22.8484795Z [         0 containers aborted    ]
2020-04-03T07:00:22.8485248Z [         9 containers successful ]
2020-04-03T07:00:22.8485740Z [         0 containers failed     ]
2020-04-03T07:00:22.8486189Z [        26 tests found           ]
2020-04-03T07:00:22.8486640Z [         0 tests skipped         ]
2020-04-03T07:00:22.8487019Z [        26 tests started         ]
2020-04-03T07:00:22.8487493Z [         0 tests aborted         ]
2020-04-03T07:00:22.8487913Z [        25 tests successful      ]
2020-04-03T07:00:22.8488376Z [         1 tests failed          ]
2020-04-03T07:00:22.8488676Z 
2020-04-03T07:00:22.9000070Z [INFO] 1 Error(s)
2020-04-03T07:00:22.9018694Z [ERROR] Error   : Exit code remote process 1: /opt/hostedtoolcache/jdk/8.0.242/x64/bin/java -cp /home/runner/work/osgi-test/osgi-test/org.osgi.test.junit5/target/test/tmp/testing/test/cnf/cache/5.0.1/bnd-cache/biz.aQute.launcher/biz.aQute.launcher.pre.jar -Dlauncher.properties=/home/runner/work/osgi-test/osgi-test/org.osgi.test.junit5/target/test/tmp/testing/test/generated/launch2633557528947728349.properties -ea -Dlauncher.runpath=/home/runner/.m2/repository/ch/qos/logback/logback-classic/1.2.0/logback-classic-1.2.0.jar,/home/runner/.m2/repository/ch/qos/logback/logback-core/1.2.0/logback-core-1.2.0.jar,/home/runner/.m2/repository/org/apache/felix/org.apache.felix.logback/1.0.2/org.apache.felix.logback-1.0.2.jar,/home/runner/.m2/repository/org/slf4j/slf4j-api/1.7.22/slf4j-api-1.7.22.jar,/home/runner/.m2/repository/org/eclipse/platform/org.eclipse.osgi/3.15.100/org.eclipse.osgi-3.15.100.jar,/home/runner/work/osgi-test/osgi-test/org.osgi.test.junit5/target/test/tmp/testing/test/cnf/cache/5.0.1/bnd-cache/biz.aQute.launcher/biz.aQute.launcher-5.0.1.jar aQute.launcher.pre.EmbeddedLauncher

[assertj] Global assert entry-points

Creating a new issue to capture discussion that began in a pull request here: #110 (comment) Thought that the discussion was important enough that it be captured in a top-level issue for posterity rather than buried somewhere as an appendage to another PR.

In summary, we think it might be a good idea on balance to have an "uber soft assertion provider" interface which amalgamates all of the individual soft assertion providers into a single interface. Probably, the same arguments apply (to a lesser degree) to have an Assertions class that contains entry points for the hard assertions.

One question still to be resolved is where to put these uber-classes. Given the package versioning stability issues that @bjhargrave is justly concerned about, it would probably best if they were put in their own package. A couple of candidates:

org.osgi.test.assertj
org.osgi.test.assertj.api

Edit: fixed link

[common] Unnecessary type safety violation in asMap()

I was reviewing #326 when I took a closer look at Dictionaries.asMap().

The method as it currently stands seems to unnecessarily leave open the potential for type safety violations. The following code snippet demonstrates the possibility:

Hashtable<Integer, Integer> intMap = new Hashtable<>();
Map<Number, Number> castMap = Dictionaries.asMap(intMap);
castMap.put(1.0, 1.9);

The above code compiles, and allows the caller to put floats into the underlying hashtable even though it was declared to accept only integers. Goodness only knows what kind of issues this might introduce further down the code path.

The fix for this is simple enough: the bounded wildcards on the dictionary need to be removed from asMap() so that the type parameters on the incoming dictionary exactly:

current:

	public static <K, V> Map<K, V> asMap(Dictionary<? extends K, ? extends V> dictionary) {

new:

	public static <K, V> Map<K, V> asMap(Dictionary<K, V> dictionary) {

This prevents the above code from compiling.

I'm struggling to the see the use case for including the bounds on the parameter to asMap(), but I raised this ticket in case someone else sees the need. If there is a genuine use case for it, I suggest that we push the burden of doing the unsafe cast back to the caller, eg:

Hashtable<Integer, Integer> intMap = new Hashtable<>();
Map<Number, Number> castMap = Dictionaries.asMap((Dictionary<Number, Number>)intMap);
castMap.put(1.0, 1.9);

or:

Hashtable<Integer, Integer> intMap = new Hashtable<>();
Map<Number, Number> castMap = (Map<Number, Number>)Dictionaries.asMap(intMap);
castMap.put(1.0, 1.9);

That way, if the caller wants to do something silly like this for whatever reason, at least the potential for compile-time type safety circumvention will be made explicit in their code. With the current API, we are hiding the type safety circumvention behind our own unsafe cast.

Because this is a breaking change (albeit a completely justifiable one IMO), I feel that it would be good to try and squeeze this into the 1.0 release (if possible) by cherry picking it?

[junit5] Extensions to properly handle parallelism

While having a look at #113, I realised that our current implementation of the JUnit 5 extensions and use of the Store is not parallel-safe. Because neither the key nor the namespace that we are using for the context lookup makes reference to the test method, every test method within the test class will always get the same BundleContext instance (or whatever state object it is storing). If their execution happens to overlap, this will cause them to interfere with each other.

The fix should be simple - to include the method name in the Namespace. See the TimingExtension.getStore() example in the JUnit 5 user doc here

I will look at this in conjunction with #113.

deduplicate the test code between junit4 and junit5 test harnesses

If ServiceUseRule and ServiceUseExtension had a common base class, then their respective test classes could too. That way you can avoid duplicating the test code between each and also give a greater guarantee that their behaviour is consistent.

Originally posted by @kriegfrj in #34

google/truth vs assertj

Is the assertion Framework fixed to AssertJ or is Google Truth an option?

https://github.com/google/truth

use osgi test bom in examples

When we release the bom we should use it in the example.

https://github.com/osgi/osgi-test/blob/main/examples/osgi-test-example-mvn/pom.xml

[junit5] Define an OSGi service extension

[workflow] Reinstate coverage for PRs

Looks like the build work in #129 killed code coverage stats for PRs. I'm assuming that this was an unintended side effect of overloading the canonical flag - coverage is very useful on PRs to see if you've left any gaping holes in your testing before you merge it.

Add at least a minimal example to the wiki

Currently the wiki only list the releases, but a minimal example would be nice to present there, or even a link to some blog-posts.

Create gradle version of examples/osgi-test-example-mvn

We need a Gradle (non-Bnd workspace) equivalent of examples/osgi-test-example-mvn for the users who are Gradle users.

[junit4] Resolution not picking up new dependency

I'm not sure if this is an osgi-test configuration issue, or a Bndtools/m2e issue, but:

I added Mockito as a test dependency. When I look at the Resolution window for org.osgi.test.junit4-1.0.0-SNAPSHOT-test.jar, it is correctly declaring org.mockito as an import. However, when I ran the resolver on test.bndrun, it failed to pick up Mockito as a required bundle.

[junit5] ServiceUseExtension is not stateless

The ServiceUseExtension keeps the state variables for the BundleContextExtension and the TrackServices instances.

State variables are not permitted in Jupiter extensions, because Jupiter makes no lifecycle guarantees about individual extension object instances (see https://junit.org/junit5/docs/current/user-guide/#extensions-keeping-state). Instead, extensions are supposed to store their state in the supplied ExtensionContext object.

[junit] Use junit-platform-testkit for testing extensions

I have learned in the last couple of years that mocking as a testing technique is fraught, especially as your level of functionality increases:

you spend as much time debugging your mocks as you spend debugging your actual code;
in spite of 1, your mocks are an approximation, and you can end up with tests that pass but the deliverable will still fail in the real world.

For this reason, I think it is almost always better to test against real code (where practical) rather than mocks.

Having wrestled briefly with #113 and #114, I think we have reached that point now where our mock-based tests are getting difficult to maintain and error-prone. Though it will involve a bit of work, I think we would be well served for the long time by putting in that effort now to port them to use junit-platform-testkit. I think that AssertJ's tests for SoftAssertionsExtension are a good model to follow (they were written by one of the core Jupiter developers): https://github.com/joel-costigliola/assertj-core/tree/master/src/test/java/org/assertj/core/api/junit/jupiter

	/**
	* Return a Map wrapper around a Dictionary.
	*
	* @param <K> The type of the key.
	* @param <V> The type of the value.
	* @param dictionary The dictionary to wrap.
	* @return A Map object which wraps the specified dictionary. If the
	* specified dictionary can be cast to a Map, then the specified
	* dictionary is returned.
	*/
	public static <K, V> Map<K, V> asMap(Dictionary<? extends K, ? extends V> dictionary) {