Async tests (wait) sometimes fail about expecta HOT 12 OPEN

I've just had it happen again, so this change does not fix the problem. Needs more investigation as to what the differences are between the two approaches. My SenTestKit category definitely doesn't show this behaviour, it only started showing up when converting one of our tests to expecta format. I much prefer expecta's syntax and conciseness - will try and get this to work.

from expecta.

@sas71, any progress? I've started seeing this as well, although it could be related to upgrading to Xcode 5.

I've tried 3 or 4 different Obj-C unit testing solutions, looking for something that can deal with asynchronicity in a robust fashion, and have yet to find it.

from expecta.

Not yet, unfortunately. Have you tried the version I've linked to above? Just to get an idea if there's a working version to compare to or if there's a general problem.

from expecta.

could you give specta's AsyncBlock feature a try?

from expecta.

I've had some time to revisit this and just noticed that these failures only seem to happen when using Xcode 5. I'll try to dig deeper. I'll also look at AsyncBlock as soon as I find the time, Peter.

from expecta.

Any progress? I have the same problems when I rig my test using an AsyncBlock from Specta.

from expecta.

I’m afraid not. We’ve had to abandon this for now due to lack of time to investigate further.

Dr. Sven A. Schmidt
[email protected] | http://feinstruktur.com

On 7 Jan 2014, at 17:56, Brennon Bortz [email protected] wrote:

Any progress? I have the same problems when I rig my test using an AsyncBlock from Specta.

—
Reply to this email directly or view it on GitHub.

from expecta.

FWIW, it appears that there may have been an underlying bug in Xcode. If you have some time, give the same tests another go under Xcode 5.1 — I'm seeing much more consistent results now.

from expecta.

Nope, I'm still seeing this in Xcode 6.0.1. Something is not right, but I haven't been able to figure out what.

@petejkim what are the differences in how Specta's AsyncBlock handles this compared to Expecta's will/willNot?

from expecta.

will/willNot is not reliable for me at all. It works for a while — long enough for me to believe that a test has been fixed — but when I implement a few more tests, the tests involving will start hanging indefinitely, preventing normal test execution from occurring.

If it hangs once, it consistently hangs.

Pretty mysterious. I've not set the timeout value at all either.

UPDATE: curiously still, I tried using https://github.com/hfossli/AGAsyncTestHelper to replace the will calls, but I get the nonterminating behaviour.

Could it be because I'm trying to check against a Core Data attribute that's causing the loop?

With Expecta:

expect(recipe.needsSync).will.equal(@YES);

With AGAsyncTestHelper:

AGWW_WAIT_WHILE_NOT_EQUALS(recipe.needsSync, @YES, 0.5);

from expecta.

Is there any update about it? I am sometimes getting random fails when using will

Also, it sometimes get stuck when applyMatcher:to gets called. Here:

Line 123

 [[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:0.01]];

from expecta.

I started using Expecta very recently and was very happy and impressed, but I'm having some nondeterministic trouble using will matching as well, in my case it only happens for a specific test running on Travis. It passes on Xcode and fastlane locally, but fails on CI.

from expecta.