Consistency issue using aetest about goon HOT 10 CLOSED

Sure you are not modifying the local cache? See my misunderstanding here. If you're not, please issue a pull request with a failing test.

See issue #27

from goon.

No I am not touching the local cache and in goon_test.go you can see :

        // Sleep a bit to wait for the HRD emulation to get out of our way
        time.Sleep(1000 * time.Millisecond)

I’m hosting an event today so I’ll send the pull request tomorrow. Cheers!

On 15 Aug 2014, at 23:31, mzimmerman [email protected] wrote:

Sure you are not modifying the local cache? See my misunderstanding here. If you're not, please issue a pull request with a failing test.

See issue #27 #27
—
Reply to this email directly or view it on GitHub #41 (comment).

from goon.

The TestCaches test seems to test exactly what you're stating is the issue and it passes for me.
goapp test -run Caches

The only time we're doing time.Sleep in the goon tests is so that we can run queries successfully. As the thread you linked to states, the datastore needs time to build the index so your queries are successful. It's not that we're waiting to fetch the data by key.

What version of the SDK are you using?

from goon.

What I was trying to say is that the proper way to do the test would be to fetch the entity created right away using get with the key returned by the put. Performing this get ensures that the entity is written to the datastore and indexed. Once you do that queries will work without having to rely on a sleep timer.

Using a datastore get after a put works, using a goon get, the test fails. I'll post the PR tomorrow.

On 16 Aug 2014, at 13:28, mzimmerman [email protected] wrote:

The TestCaches test seems to test exactly what you're stating is the issue and it passes for me.
goapp test -run Caches

The only time we're doing time.Sleep in the goon tests is so that we can run queries successfully. As the thread you linked to states, the datastore needs time to build the index so your queries are successful. It's not that we're waiting to fetch the data by key.

What version of the SDK are you using?

—
Reply to this email directly or view it on GitHub.

from goon.

Has anyone actually had the goon tests fail because of the sleep? I have never seen it. It's important to note that the goon tests will never produce a false positive due to this sleep. That said, I guess it wouldn't be a problem to swap the sleep out for a get, if there's more evidence that a get will actually commit the indexes.

However, @zboralski, it seems to me from your initial post that your problem isn't with the goon tests but with your own tests.

The thing with goon's Put is that it populates the local cache. The reason why just doing a Get afterwards doesn't achieve your goal has to do with the fact that the Get will just retrieve the entity from the local cache and no underlying datastore.Get is ever made. You would need to clear the local cache with FlushLocalCache before doing the Get.

from goon.

Go figure. As I understood it, doing a Get by key would cause writing all the pending writes to the datastore, but I didn't think it would ensure that indexes are generated but it does indeed seem to do that (at least on the dev SDK). I've issued a pull request for that.

I have gotten queries to fail due to the sleep (not goon's but my own).

@xStrom's got it right. I was misreading what the error was, but now seeing his explanation I get it. Your own tests are failing since your goon.Get call doesn't actually go to the datastore since it pulls from local cache, so it doesn't ensure your indexes are being generated in time. Either flushing goon local cache or doing a datastore.Get (like you've done) is a good solution.

from goon.

@xStrom flushing the local cache works. I should have thought of that. Thank you.

@mzimmerman Thanks, that's perfect!

from goon.

@mzimmerman gets executed in a request following a write in the previous request fail quite often... that's in an app not a test.

Also @xStrom flushing the local cache before every get really defeats the purpose of having a local cache...

from goon.

It's important that we remember the scope of the conversation - which is unit tests. I suggested using FlushLocalCache precisely because the local cache was getting in your way. However if you have a different goal, where you want to use the local cache, then of course flushing it works against your goal.

from goon.

Yes, this is not a goon issue. What you're essentially now asking for is the goon.Get() function to go back to the datastore when the key is already in the cache. If we did that, goon would really be failing in it's mission.

In your tests, when you need to query results of an entity, do the following.

datastore.Get(c,g.Key(data),data)

Then continue on in your tests as normal. This does not destroy the local cache. Note that if you're having this issue in your tests, you'll also have it in production. Datastore queries are eventual consistency, not immediate -- this is regardless of using goon.

from goon.