markmeeus / marcellodb Goto Github PK

To avoid future code incompatibilities it is better to inject a single object.
If at some point we need inject more platform specific implementations, we can avoid breaking code if we can keep the signature consistent.

Therefore, instead of injecting the platform specific FileStorageStreamProvider, we should inject a Platform object.

var session = new MarcelloDB.Session(new MarcelloDB.Platform());

A new version of MarcelloDB will come with a new version of the specific platforms.

Would be nice to be able to inject a serializer implementation, so user could use the serializer they like best according to their needs.
For example, a binary serialization would be good for perf.

Using this TypeNameHandling has a big impact on serialization performance.
Instead, we should use TypeNameHandling.Auto whenever possible.
This seems not to work for the root element, so we may need to wrap the objects beeing serialized in a property.

When creating RecordIndexes, we should inject the root record instead of having the root record fetching logic inside the RecordIndex

After this refactor, it will be easier to manage the rootrecords / collection instead of per collection file

Instead of leaving the value null, throw an exception to inform the user about the possible error.
If it ever becomes necessary, we can add an attribute [IgnoreIndex] or something like that.

No point in having comparer objects being created all the time

Currently, a type matches a collection.
Collection instead should have names like this:
This way, a single collection file can have multiple collections of the same type.

session["store_data"].Collection<Article>("clothing");
session["store_data"].Collection<Article>("food");

This may enable serialization into a more generic object (maybe for data migration scenario's)

session["store_data"].Collection<Dictionary<string, object>>("clothing");

or

session["store_data"].Collection<object>("clothing");

The btree algorithm has a flaw when the last child node needs to be split.
It will add another entry in the parent, which makes to grow over it's max limit.

MarcelloDb assumes to find an ID property.
The case where the object to be persisted has no such thing is not property handled.

It should throw a custom IDPropertyMissingException when such a scenario occurs.

Every btree node which is loaded in a transaction will be written back to disk, even if the data hasn't changed.
The btree implementation interacts with the lists of childrenAddresses and entries directly, so the data provider has no way of knowing which nodes to write back to the stream.

Possible solutions:
=> Use dedicated methods on node to add/remove items to both lists (childrenAddress and Entries) and mark the node as dirty.

The typename of the collection is used as part of the index name, which has the casing as in the defining source code.
This part should be upper cased to have more consistent naming.

IndexesTest should be refactored to DRY-out all the data creation.

Since the maximum size of these nodes is predicable, we can avoid them being moved (and thus recycled) by allocating their records with the maximum size.
The way it is now, when a record is recycled, it's added to the empty record index. This may in turn cause the node record to outgrow it's allocated space, which in turn triggers a recycle.
There is a theoretical risk of empty record explosion.

CollectionFile, Collection and RecordIndex all depend on a root record.
CollectionFile => RootRecord should contain a Dictionary of <string, Int64> (Collection Root record addresses per collection name)
Collection => RootRecord should containt a Dictionary <string, Int64> (RecordIndex root record per index name)
RecordIndex => Contains the Rootnode.

In all cases, the root record may change during the transaction and needs to be written back to disk when the transaction succeeds.
Also, these records are supposed to be tiny, so they may be cached within a session for future use.
(CollectionFiles and Collection instances are cached anyway so a signle member will do it.)

If objects are being enumerated and modified inside the same thread, the index can get changed underneath the iterator.

To avoid unexpected behaviour, we should throw an InvalidOperation exception.

So this should throw

foreach(var a in articles.all){
   articles.destroy(a);
}

This is fine:

foreach(var a in articles.All.ToArray()){
   articles.destroy(a);
}

Constructor of CompoundValue can be made public to simplify API

• The node cache is a mess.
• Methods need to be cleaned
• Tests

While they should be returning the correct scope instance.
Descending and Key enumeration are not possible because of this.

There seems to be a very rare scenario in which a call to Destroy seems to hang.
It is probably related to the recycling of a node in the empty record index which makes it go in some sort of loop.
This issue may go away when #4 is implemented.

I am getting a sharing violation exception on the journal file in the database directory when I try to write to a collection when there are more than one collection. I have two collections that are singletons derived from a Database class which has a static instance Session.

//Database.cs
public abstract class Database<ITEMTYPE>
{
    static volatile Session session;
    static object _SessionLock = new Object ();

    protected Session _session{
        get{
            if (session == null) {
                lock (_SessionLock) {
                    if (session == null) {
                        string path = Environment.GetFolderPath (Environment.SpecialFolder.Personal);
                        #if __IOS__
                        // we need to put in /Library/ on iOS5.1 to meet Apple's iCloud terms
                        path = Path.Combine (path, "..", "Library"); // Library folder instead
                        #endif

                        //Create database path if it doesnt exist
                        path = Path.Combine (path, "Data");
                        System.IO.Directory.CreateDirectory (path);

                            session = new Session (new MarcelloDB.netfx.Platform (),path);
                    }
                }
            }
            lock (_SessionLock) {
                return session;
            }
        }
    }

    protected Collection<T, TID> InitalizeCollection <T, TID> (Func<T, TID> idFunct)
    {
        lock(_SessionLock) {
                    return _session [$"{_collectionName}.data"].Collection<T, TID> (_collectionName, idFunct);
        }
    }
}

//ConversationsCollection.cs
public class ConversationsCollection : Database<Conversation>
{
    Collection<Conversation, string> _Collection {
        get {
            lock (_collectionLock) {
                return InitalizeCollection<Conversation, string> (p => p.Id);
            }
        }
    }

    #region Singleton Constructor/Initilizer
    ConversationsCollection ()
    {
    }

    static volatile ConversationsCollection instance;
    static object _ThisLock = new Object ();

    public static ConversationsCollection Instance {
        get {
            if (instance == null) {
                lock (_ThisLock) {
                    if (instance == null)
                        instance = new ConversationsCollection ();
                }
            }
            lock (_ThisLock) {
                return instance;
            }
        }
    }

    public override void Upsert (Conversation item)
    {
        lock (_collectionLock){
                _Collection.Persist (item);
                CollectionChanged (this, new DataCollectionChangedEventArgs<Conversation> (DataCollectionChangedEventArgsActions.Upsert, item));
        }
    }
}

System.IO.IOException: Sharing violation on path /Users/toddhenderson/Library/Developer/CoreSimulator/Devices/F73FB783-9ABC-40CC-AB18-78C90213D6C6/data/Containers/Data/Application/515A85C1-6A8D-4022-B3A6-F44471A60029/Library/Data/journal
  at System.IO.FileStream..ctor (System.String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, Boolean anonymous, FileOptions options) [0x0026d] in /Library/Frameworks/Xamarin.iOS.framework/Versions/9.8.0.323/src/mono/mcs/class/corlib/System.IO/FileStream.cs:257 
  at System.IO.FileStream..ctor (System.String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, Boolean isAsync, Boolean anonymous) [0x00000] in /Library/Frameworks/Xamarin.iOS.framework/Versions/9.8.0.323/src/mono/mcs/class/corlib/System.IO/FileStream.cs:157 
  at System.IO.FileStream..ctor (System.String path, FileMode mode, FileAccess access) [0x00000] in /Library/Frameworks/Xamarin.iOS.framework/Versions/9.8.0.323/src/mono/mcs/class/corlib/System.IO/FileStream.cs:92 
  at MarcelloDB.netfx.Storage.FileStorageStream..ctor (System.String filePath) [0x00008] in <filename unknown>:0 
  at MarcelloDB.netfx.Storage.FileStorageStreamProvider.GetStream (System.String streamName) [0x00025] in <filename unknown>:0 
  at MarcelloDB.Storage.StreamActors.StreamActor.GetStream () [0x00012] in <filename unknown>:0 
  at MarcelloDB.Storage.StreamActors.Reader.Read (Int64 address, Int32 length) [0x00001] in <filename unknown>:0 
  at MarcelloDB.Transactions.Journal.LoadJournal () [0x00007] in <filename unknown>:0 
  at MarcelloDB.Transactions.Journal.Apply () [0x00001] in <filename unknown>:0 
  at MarcelloDB.Session.EnsureTransaction () [0x00018] in <filename unknown>:0 
  at MarcelloDB.Session.Transaction (System.Action action) [0x0001b] in <filename unknown>:0 
  at MarcelloDB.Collections.Collection`2[T,TID].Transacted (System.Action action) [0x0001b] in <filename unknown>:0 
  at MarcelloDB.Collections.Collection`2[T,TID].Persist (MarcelloDB.Collections.T obj) [0x00015] in <filename unknown>:0 
  at VideoChat.Data.ConversationsCollection.Upsert (VideoChat.Models.Conversation item) [0x0003d] in /Users/toddhenderson/Projects/Xamarin/VideoChat/VideoChat/Data/ConversationsCollection.cs:68 
  at VideoChat.Pages.MessagesPage.SyncConversations () [0x00035] in /Users/toddhenderson/Projects/Xamarin/VideoChat/VideoChat/Pages/MessagesPage.xaml.cs:46 
  at VideoChat.Pages.MessagesPage.OnAppearing () [0x00002] in /Users/toddhenderson/Projects/Xamarin/VideoChat/VideoChat/Pages/MessagesPage.xaml.cs:24 
  at Xamarin.Forms.Page.SendAppearing () [0x00024] in <filename unknown>:0 
  at Xamarin.Forms.Page.SendAppearing () [0x00054] in <filename unknown>:0 
  at Xamarin.Forms.Platform.iOS.NavigationRenderer.ViewDidAppear (Boolean animated) [0x0001e] in <filename unknown>:0 
  at (wrapper managed-to-native) UIKit.UIApplication:UIApplicationMain (int,string[],intptr,intptr)
  at UIKit.UIApplication.Main (System.String[] args, IntPtr principal, IntPtr delegate) [0x00005] in /Users/builder/data/lanes/3339/39ebb778/source/maccore/src/UIKit/UIApplication.cs:79 
  at UIKit.UIApplication.Main (System.String[] args, System.String principalClassName, System.String delegateClassName) [0x00038] in /Users/builder/data/lanes/3339/39ebb778/source/maccore/src/UIKit/UIApplication.cs:63 
  at VideoChat.iOS.Application.Main (System.String[] args) [0x00014] in /Users/toddhenderson/Projects/Xamarin/VideoChat/iOS/Main.cs:23 

When dealing with datafiles directly (downloading updated data f.i) it is convenient to group collections in a single file.
Since a collection is re;ferenced by it's ID-tree's root address (which is a named record internally), it should be fairly easy to support multiple collections with the same mechanism.

Currently, the filenames are the typename of the class for which the collection is created.
The filename should be an explicit parameter for instantiating a collection.

Ideally, the api for creating a collection should look like this:

 //Articles and Locations are stored in a single file named "shop-data"
var articles = session["shop-data"].Collection<Article>();
var locations = session["shop-data"].Collection<Location>();

//Orders are stored in the "orders" file
var orders = session["orders"].Collection<Order>()

Btree nodes are currently allocated with the default DoubleSizeAllocationStrategy.

A better alternative would be to use a simple heuristic based on the percentage of filled values and child adresses to make a guess for the max size of the node.

There is a lot of overhead in the files if a custom type is used as key.
The list of entries in initialized as a list of Entry<object, Int64>, this causes the serializer to insert the type in the serialization.
A solution would be to use a specific serializer of Serializer<Node<CustomClass, Int64>>>.

Can we used this library in PCL project for Xamarin which will be used in Xamarin.iOS and Xamarin.Android projects?

How to fix
marcelloDB.collections.indexIDvalue does not contain a definition for setContext

also i wanna get all database objects to list to show it in recycleview

Hi Mark,

How to ignore class properties in MarcelloDB? For Sqlite they have ignore attribute but I see only ID attribute in MarcelloDB.

Thanks,
Pravin.

After data is added or removed from an index, the index blocks are persisted via RecordManager.
If a data block outgrows it's allocated size, RecordManger will append the record instead of update the record in place.

There is currently an EMPTY_RECORDS_BY_SIZE index, which is used by Collection to reuse delete records when appending new record.

This index is only populated when records are deleted, but not when records are moved.

Ideally, this behaviour would be completely abstracted by RecordManager.

Currently, it throws an InvalidCastException

Consider following class structure

public class Parent
{
  public int ParentID {get; set;}
}

public class Child : Parent
{
}

When persisting a child, it should use the ParentId property as ID;
There will be no support for using ID properties on multiple classes in the hiërarchy.

The current implementation of the transactions uses a collection to store the updated blocks of data.
A simple bson serialized blob saved to a file should do the trick.

Empty nodes currently serialize in +600 bytes.

It seems OutOfMemory may occur in case of two parallel threads (in my case one thread is reading while another thread is writing)

System.ArgumentException: An item with the same key has already been added.
at System.ThrowHelper.ThrowArgumentException (System.ExceptionResource resource) [0x0000b] in :0
at System.Collections.Generic.Dictionary2[TKey,TValue].Insert (TKey key, TValue value, System.Boolean add) [0x0008e] in <c78deec413e14f3190663c79cd1ab03d>:0 at System.Collections.Generic.Dictionary2[TKey,TValue].Add (TKey key, TValue value) [0x00000] in :0
at MarcelloDB.NodePersistence1[TK].Persist (MarcelloDB.Index.Node1[TK] node, System.Collections.Generic.Dictionary2[TKey,TValue] loadedNodes, MarcelloDB.Serialization.IObjectSerializer1[T] serializer, System.Collections.Generic.Dictionary2[TKey,TValue] touchedNodes, MarcelloDB.Records.IndexMetaRecord metaRecord) [0x00022] in <3388e64d99a64e1bafded80b38dd4acd>:0 at MarcelloDB.NodePersistence1[TK].SaveChildren (MarcelloDB.Index.Node1[TK] node, System.Collections.Generic.Dictionary2[TKey,TValue] loadedNodes, MarcelloDB.Serialization.IObjectSerializer1[T] serializer, System.Collections.Generic.Dictionary2[TKey,TValue] touchedNodes, MarcelloDB.Records.IndexMetaRecord metaRecord) [0x00046] in <3388e64d99a64e1bafded80b38dd4acd>:0
at MarcelloDB.NodePersistence1[TK].Persist (MarcelloDB.Index.Node1[TK] node, System.Collections.Generic.Dictionary2[TKey,TValue] loadedNodes, MarcelloDB.Serialization.IObjectSerializer1[T] serializer, System.Collections.Generic.Dictionary2[TKey,TValue] touchedNodes, MarcelloDB.Records.IndexMetaRecord metaRecord) [0x00001] in <3388e64d99a64e1bafded80b38dd4acd>:0 at MarcelloDB.NodePersistence1[TK].SaveChildren (MarcelloDB.Index.Node1[TK] node, System.Collections.Generic.Dictionary2[TKey,TValue] loadedNodes, MarcelloDB.Serialization.IObjectSerializer1[T] serializer, System.Collections.Generic.Dictionary2[TKey,TValue] touchedNodes, MarcelloDB.Records.IndexMetaRecord metaRecord) [0x00046] in <3388e64d99a64e1bafded80b38dd4acd>:0
at MarcelloDB.NodePersistence1[TK].Persist (MarcelloDB.Index.Node1[TK] node, System.Collections.Generic.Dictionary2[TKey,TValue] loadedNodes, MarcelloDB.Serialization.IObjectSerializer1[T] serializer, System.Collections.Generic.Dictionary2[TKey,TValue] touchedNodes, MarcelloDB.Records.IndexMetaRecord metaRecord) [0x00001] in <3388e64d99a64e1bafded80b38dd4acd>:0 at MarcelloDB.NodePersistence1[TK].Persist (MarcelloDB.Index.Node1[TK] rootNode, System.Collections.Generic.Dictionary2[TKey,TValue] loadedNodes, MarcelloDB.Serialization.IObjectSerializer1[T] serializer, MarcelloDB.Records.IndexMetaRecord metaRecord) [0x00007] in <3388e64d99a64e1bafded80b38dd4acd>:0 at MarcelloDB.Index.BTree.RecordBTreeDataProvider1[TNodeKey].Flush () [0x0007d] in <3388e64d99a64e1bafded80b38dd4acd>:0
at MarcelloDB.Index.RecordIndex1[TNodeKey].FlushProvider () [0x00007] in <3388e64d99a64e1bafded80b38dd4acd>:0 at MarcelloDB.Index.RecordIndex1[TNodeKey].Register (TNodeKey keyValue, System.Int64 recordAddress) [0x0000f] in <3388e64d99a64e1bafded80b38dd4acd>:0
at MarcelloDB.Records.RecordManager+<>c__DisplayClass6.b__5 () [0x0001d] in <3388e64d99a64e1bafded80b38dd4acd>:0
at MarcelloDB.Records.RecordManager+<>c__DisplayClass9.b__8 () [0x00001] in <3388e64d99a64e1bafded80b38dd4acd>:0
at MarcelloDB.Records.RecordManager.UsingEmptyRecordIndex[T] (System.Func`1[TResult] func) [0x00022] in <3388e64d99a64e1bafded80b38dd4acd>:0
at MarcelloDB.Records.RecordManager.UsingEmptyRecordIndex (System.Action action) [0x0000e] in <3388e64d99a64e1bafded80b38dd4acd>:0
at MarcelloDB.Records.RecordManager.RegisterRecycledRecordsInEmptyRecordIndex () [0x0007a] in <3388e64d99a64e1bafded80b38dd4acd>:0
at MarcelloDB.Records.RecordManager.SaveState () [0x00008] in <3388e64d99a64e1bafded80b38dd4acd>:0
at MarcelloDB.Transactions.Transaction.Commit () [0x00021] in <3388e64d99a64e1bafded80b38dd4acd>:0
at MarcelloDB.Transactions.Transaction.Leave () [0x00030] in <3388e64d99a64e1bafded80b38dd4acd>:0
at MarcelloDB.Session.Transaction (System.Action action) [0x0003c] in <3388e64d99a64e1bafded80b38dd4acd>:0

The binary serialization mechanism has better precision than the bson one.
This causes the indexes to not unregister on update, which ends up in a Duplicate entry on updating an object.

Solution is to fix the ObjectComparer to use millisecond precision for DateTimes

The empty record index uses the size of the empty record as key.
This can (and will) cause multiple records on the same key.
(Records that have the same allocated size)

Although this doesn't seem to create any issues, the outcome of the btree lookups, deletes and inserts is undetermined in such situation.
If we want to not depend on the implicit orderering of unique keys in the btree, we need to makes sure they are unique, and have the btree validate uniqueness. (It should not find the same key when inserting)

Currently, there is only an issue for the empty record index, we can solve this by using a combination of the allocated space and the record address as key. So that the sort order is by allocated address and within the same allocated addresses, by record address.

After every transaction, CollectionRoot is written back to disk, even when it contains no changes.
Since reads are wrapped in a transaction, every read (Find) will trigger a write.

at MarcelloDB.Collections.IndexedValue2.<get_ValueFunction>b__2(TObj o) at MarcelloDB.Collections.IndexedValue2.GetValue(Object o)
at MarcelloDB.Collections.IndexedIDValue1.GetKey(Object o, Int64 address) at MarcelloDB.Collections.Collection1.RegisterInIndexes(T o, Record record, Int64 originalAddress, T originalObject)
at MarcelloDB.Collections.Collection1.PersistInternal(T obj) at MarcelloDB.Collections.Collection1.<>c__DisplayClass4.b__3()
at MarcelloDB.Collections.Collection1.<>c__DisplayClassa.<Transacted>b__9() at MarcelloDB.Session.Transaction(Action action) at MarcelloDB.Collections.Collection1.Transacted(Action action)
at MarcelloDB.Collections.Collection`1.Persist(T obj)

instance of class:
public sealed class Product
{
public Guid Id { get; set; }

    public string Article { get; set; }

    public string Name { get; set; }

    public string MetaKeywords { get; set; }

    public string MetaDescription { get; set; }

    public string Description { get; set; }

    public string Comment { get; set; }

    public DateTime DateAdded { get; set; }

    public DateTime DateExported { get; set; }

    public DateTime DateUpdated { get; set; }

    public bool HasErrors { get; set; }

    public Guid ProductStoreCategoryId { get; set; }

    public Guid ManufacturerId { get; set; }

    public List<ProductParameterValue> ParameterValues { get; set; }

    public List<ProductImage> ProductImages { get; set; }
}

Because the journal file is reserverd for the write ahead log

In the current implementation, the index entries point directly to the records location.
If the data moves to a new location, the index entry needs to be updated.

After indexed attributes are implemented, this approach will force us to update every index entry when an object moves.
Instead, we should have the indexes point to a 'forward record' which contains the current address of the record. This causes an extra read for every index lookup, but limits the writes to this single record when the record moves.

This needs to be implemented only for data-records.

I'm encountering with #31 again. So, it is figured out that 0.3.1 libraries have installed with 0.3.3 version nuget-package (Tested both in VS2013 and VS2015).

Could you pls check nuget package?

Hi. I just downloaded the latest version of MarcelloDB from Nuget, version 0.4.2. When I tried it, it didn't work, it couldn't load Newtonsoft.Json. I investigated a little and noticed Marcello is built against Newtonsoft.Json version 6.0.8, but the Nuget build script depends on 7.0.1. I manually got Newtonsoft.Json version 6.0.8, replaced the binary, and it worked. Just thought you might want to fix this.

Hi! I'm encountered with strange behavior of indecies.
So, I have a class like this

    public sealed class HistoryPoint
    {
        public Guid Id { get; set; }

        public LocalDate Date { get; set; }

        public decimal Price { get; set; }
    }

Inside the server code i'm trying to get record of type HistoryPoint from database by date (using index):

Collection().Indexes.Date.Find(localDate)

And then if record exists i'm trying to update it and save into database.

Then I'm using such code to get all records for the interval:

Collection().Indexes.Date.GreaterThanOrEqual(startDate)

In some cases, above code can return two records of the single object (ids are identical). It seems that it is bug in index rebuild algorithm. I believe that I don't have duplicated objects in the database, but I have two different(by Reference) duplicated objects in the memory.

P.S. Collection().All works well

string/object collections could be use to transform data between different version of the consuming app. Otherwise, a broken serializer (due to changes in the data class)may render the entire dataset useless.

I'm using this issue to write down some ideas about the index api.

Hello Mark!

I'm working on upgrading my data from 0.4.x to 0.6.x version of MarcelloDB. And I've decided to go the following way:

1. Export all data from 0.4.x MarcelloDB into json files
2. Upgrade MarcelloDB from 0.4.x to 0.6x
3. Import all data from json files into MarcelloDB 0.6.x

So, I was unable to do it without custom code. I wrote smth like this:

public static class CollectionJsonExportImport
    {
        public static void ExportToFile<T, TID, TIndexDef>(
            this MarcelloDB.Collections.Collection<T, TID, TIndexDef> collection, string filePath)
            where TIndexDef : IndexDefinition<T>, new()
        {
            var contents = ExportToJson(collection);
            File.WriteAllText(filePath, contents);
        }

        public static string ExportToJson<T, TID, TIndexDef>(
            this MarcelloDB.Collections.Collection<T, TID, TIndexDef> collection)
            where TIndexDef : IndexDefinition<T>, new()
        {
            var items = collection.All.ToList();
            var result = JsonConvert.SerializeObject(items,
                new JsonSerializerSettings {TypeNameHandling = TypeNameHandling.All});
            return result;
        }

        public static void ImportFromFile<T, TID, TIndexDef>(
            this MarcelloDB.Collections.Collection<T, TID, TIndexDef> collection, string filePath)
            where TIndexDef : IndexDefinition<T>, new()
        {
            var contents = File.ReadAllText(filePath);
            var items = JsonConvert.DeserializeObject<List<T>>(contents,
                new JsonSerializerSettings {TypeNameHandling = TypeNameHandling.All});

            foreach (var item in items)
            {
                collection.Persist(item);
            }
        }
    }

May be it would be useful for other guys. Pls consider to include such functionality into MarcelloDB library.

Is it possible to store private fields? Or maybe some way for configuring bson serializer exists?

It is interesting that I have to remove all data files because I was catching OutOfMemory exception on Persist call:

at MarcelloDB.netfx.Storage.FileStorageStream.Read(Int64 address, Int32 length)
at MarcelloDB.Storage.StreamActors.Reader.Read(Int64 address, Int32 length)
at MarcelloDB.Storage.StreamActors.JournalledReader.Read(Int64 address, Int32 length)
at MarcelloDB.Storage.StorageEngine.Read(Int64 address, Int32 length)
at MarcelloDB.Records.RecordManager.ReadEntireRecord(Int64 address)
at MarcelloDB.Records.RecordManager.GetRecord(Int64 address)
at MarcelloDB.Records.RecordManager.GetNamedRecordIndexRecord()
at MarcelloDB.Records.RecordManager.RegisterNamedRecordAddress(String name, Int64 recordAddress)
at MarcelloDB.Index.BTree.RecordBTreeDataProvider1.SaveMetaRecord() at MarcelloDB.Index.BTree.RecordBTreeDataProvider1.Flush()
at MarcelloDB.Index.RecordIndex1.FlushProvider() at MarcelloDB.Index.RecordIndex1.Register(TNodeKey keyValue, Int64 recordAddress)
at MarcelloDB.Records.RecordManager.<>c__DisplayClass6.b__5()
at MarcelloDB.Records.RecordManager.<>c__DisplayClass9.b__8()
at MarcelloDB.Records.RecordManager.UsingEmptyRecordIndexT
at MarcelloDB.Records.RecordManager.UsingEmptyRecordIndex(Action action)
at MarcelloDB.Records.RecordManager.RegisterRecycledRecordsInEmptyRecordIndex()
at MarcelloDB.Records.RecordManager.SaveState()
at MarcelloDB.Transactions.Transaction.Commit()
at MarcelloDB.Transactions.Transaction.Leave()
at MarcelloDB.Session.Transaction(Action action)
at MarcelloDB.Collections.Collection1.Transacted(Action action) at MarcelloDB.Collections.Collection1.Persist(T obj)
at Products.Infrastructure.FakeProductRepository.Save(Product entity) in

It seems workaround for OutOfMemory is to use IndexDefinition:

            using (var db = Database.Open())
            {
                var file = db.Session[FileName];
                var products = file.Collection<Product, ProductIndexDefinition>(CollectionName);
                products.Persist(entity);
                return entity.ID;
            }

• Streams should be flushed inside the transaction, before the write-ahead journal is cleared.
• Streams should be closed when the StreamProvider is Disposed.

I would love to try this out (instead of serialising/deserialising my JSON) but not working with my PCL (Xamarin.Forms) project?

Could not install package 'MarcelloDB 0.3.2'. You are trying to install this package into a project that targets 'portable-net45+win+wp80+MonoTouch10+MonoAndroid10+xamarinmac20+xamarintvos10+xamarinwatchos10+xamarinios10', but the package does not contain any assembly references or content files that are compatible with that framework. For more information, contact the package author.