Re-structure database semantics around new `Table` trait about gamesmannova HOT 3 CLOSED

Ah, ok - I'd been taking it from the description above that Tables were a snapshot of access to a particular database, not representing separate spaces within a database. The description makes much more sense with that definition! And with that definition, I see why not all of the functions in TableAccess would be needed for every Table.

So just to confirm:
A Database can implement KVStore, but doesn't have to.
A Database can also implement Tabular, and thus manage a collection of Table-implementers, which themselves must implement KVStore.

So I'll rework the volatile database to be the second type (a mapping of separate Table objects) and build the vector one as the second type as well.

Does that all seem right?

from gamesmannova.

So as I'm thinking about Table stuff

If the idea is that Tables allow for scans, should implementing Table really require implementing KVStore? It seems like it only really needs to be able to get(), and also like get()-ing individual records is an inefficient part of a scan.

I may be misunderstanding the use of a Table - if it's not just a view of the items in the database at some particular time, that could invalidate this.

What if Table items implemented a separate "TableAccess" trait, which could allow for useful types of access? E.g. get() (as in KVStore), full_scan() to return an iterator on all key-record pairs, and maybe even something like state_condition_scan(condition_func) to return an iterator on all Records whose keys meet some condition.

So the Database itself allows one to get individual records, but to scan, one creates a read-only Table that crystallizes the Hashmap at that moment.

@maxfierrog

from gamesmannova.

The fundamental idea of a table that we are using here is having separate key spaces and schemas.

Namely, we only allow two records with identical keys in a database as long as they are in different tables, which means that they might also have different schemas.

Hope that helps with the concept of a Table we're working with here.

As for the separation of functions that you point out, not all of this is meant to allow for full scans. I wanted to create the following semantics:

let db_handle = Database::initialize(...);
let table_handle = database_handle.create_table(...);
table_handle.put(...);
let record_handle = table_handle.get(...);

The reason this is useful is because get and put now mean different things depending on what table they get called on (upholding a separation of keyspaces and schemas), but you still have the option of creating a KVStore database that is not tabular (here, put and get will just be global). This basically means that every Database: KVStore should not also be Tabular, because it is its tables that should be KVStore. This can be enforced by simply expecting Tabular in a generic site, and not KVStore.

Even more importantly, we can now work on many tables at the same time, and let the database figure out how to make that fast:

// OLD
db.select_table(1);
let r1 = db.get(4);
db.select_table(2); // Structured around the implication of not using table 1  anymore
let r2 = db.get(4); 

// NEW
let t1 = db.select_table(1);
let t2 = db.select_table(2);
let r1 = t1.get(4);
let r2 = t2.get(4);

Your idea of having a TableAccess trait is good, but impractical in my opinion; does everyone who wants to get also need to full_scan? Not really, so it will probably be better to keep KVStore (for the reasons above).

So how do we support full and other kinds of scans on tables? Different kinds of iterator implementations. We can for example define a FullScanIterator trait, where if we call <TableType as IntoFullScanIterator>.iter(table), the database knows that we want to do a full scan on the table (this would just look like table.iter() though, unless table implements other types of scanning iterators).

This is canonically done through intermediate types in Rust, but that's a bit hard to explain, and is basically equivalent to the above (as in, we would actually have a bunch of zero-copy types that we could get from a Table whose implementations of Iterator have different meanings).

As for the condition_scan(func), that would be something we implement on top of these iterator definitions.

This is somewhat reminiscent of how it is done in 186.

from gamesmannova.