sanity / athena Goto Github PK

Should we use GitHub for issues as well?

Out of curiosity, how do you want to implement unit testing going forwards? I figured that I would make that my first bit of code for the project, after you set up your pom.xml for Maven how you'd like it.

Currently tag->value pairs can only be added to the Store, they cannot be deleted, or modified.

Deleting should be easy enough, we just set this position in the Store.values ArrayList to null, and ensure that the StoreIterator knows to skip null values. The challenge is that over time the ArrayList will fill up with gaps - so we may need some kind of periodic "compaction" process to get rid of them.

The difficulty with modifying values is that, let's say we modify the tags on a value somewhere in the middle of the store. Now, we may have several shortcuts skipping over this value, even though after modification the value's tags might match one of those queries. The result will be that Store.find() fails to return this value when it should.

One option is, rather than modifying a value's tags, we just delete it, and then add it again using the new tags. It will be added to the end of Store.values.

Of course this assumes that we have a way to delete values.

Thoughts?

Athena should support concurrent finds, adds, and (eventually) modifications, and deletions.

Right now Athena uses non-concurrent datastructures internally, which is likely to cause intermittent ConcurrentModificaitonExceptions if a Store is accessed from multiple threads.

Obviously making Athena concurrent is about more than just using ConcurrentHashMaps instead of HashMaps - we need to think about how we can ensure Athena supports concurrent operations.

Upon creation a Store is passed an integer, maxShortcuts, which limits the number of shortcuts permitted for each Value in the the store.

We need to determine how this parameter impacts Athena's efficiency. The lower the value the less memory Athena will use, but the more tests it must do to respond to any given query.

We should create something that creates a store with lots of randomly tagged values.

We should then create a bunch of random queries, and hit the store with these, monitoring the counter() to determine how many tests Athena had to do to retrieve the items it retrieved.

We then watch how this varies as we vary maxShortcuts.

The method Query.trueIfTrue() returns a non-exhaustive set of other queries that must match if this query matches a given set of tags.

Similarly, Query.falseIfFalse() returns a non-exhaustive set of other queries that will not match if this query doesn't match a given set of tags.

Right now we come up with these lists of queries in a rather simple naive way (see the constructors of the various query types). Can it be improved?

Right now the only logging is through a primitive and ugly static final boolean StoreIterator.verbose. We should use proper logging (provided it doesn't slow anything down).