cosh / fallen-8 Goto Github PK

The download should provide a single save point-file which could be loaded by the Load-Method

Implement all inconclusive unit tests

Due to the 2GB memory per object limitation there has to be some kind of sharding towards multiple arrays of graph elements.

It should be possible to load or save Fallen-8 via the REST-interface

What are the chances of the following ever being possible...

1. Create a new C# project in Visual Studio
2. Reference the Fallen dlls (C# or C++)
3. Create a Fallen instance with basic configuration (ie. potential peers, instance data store location, key to allow peers to talk securely) using a single command
4. Create a graph traversal Action that takes parameters Graph graph, Stack currentPath, PriorityQueue frontierQueue, T sharedData
5. Run the traversal using a single command (ie. graph.Traverse(action, sharedData) with sharedData being both the input and output. The traversal algorithm's designer has to deal with concurrency issues on their sharedData object.
6. Optionally create a separate project that follows steps one and two but creates an instance of a Fallen admin class that runs sharding (clustering?) algorithms in a separate process and and talks to the operational database as required.

I would love this and would be more than happy to contribute if there was an interest in this direction.

The matching calls to FinishRead/WriteResource() are not called when an exception occurs while holding the lock. This essentially deadlocks the whole application.

There have to be dedicated services for status information

Implement all inconclusive integration tests

Great code, but no samples.

The synchronization concept seems largely broken when it comes to operations requiring more than one step executed atomically. Because all synchronisation happens inside the methods, there is no way to ensure that a sequence like (pseudo code)

  var elms = graph.FindSetOfElements()
  graph.Delete(elms)

is executed atomically. The above scenario becomes a problem, when some other thread tries to manipulate or even read some of these elements at the same time.

The (bad) workaround is to add some R/W lock to the whole thing and use that as a sync mechanism instead of reyling on the builtin locks.

I reproduceably get a OOM crash when I reach a certain level of data in the database. It is always at the same place in the code, while attempting to Save() the data:

System.OutOfMemoryException wurde nicht von Benutzercode behandelt.
  HResult=-2147024882
  Message=Eine Ausnahme vom Typ "System.OutOfMemoryException" wurde ausgelöst.
  Source=mscorlib
  StackTrace:
       bei System.IO.FileStream.Write(Byte[] array, Int32 offset, Int32 count)
       bei System.IO.BinaryWriter.Write(Int32 value)
       bei Framework.Serialization.SerializationWriter..ctor(Stream stream, Boolean allowUpdateHeader) in d:\DEVELOP\MyProject\3rdparty\fallen-8\Dependencies\FastSerializer\Serialization\SerializationWriter.cs:Zeile 208.
       bei Framework.Serialization.SerializationWriter..ctor(Stream stream) in d:\DEVELOP\MyProject\3rdparty\fallen-8\Dependencies\FastSerializer\Serialization\SerializationWriter.cs:Zeile 186.
       bei NoSQL.GraphDB.Persistency.PersistencyFactory.SaveIndex(String indexName, IIndex index, String path) in d:\DEVELOP\MyProject\3rdparty\fallen-8\Fallen-8\Persistency\PersistencyFactory.cs:Zeile 277.
       bei NoSQL.GraphDB.Persistency.PersistencyFactory.<>c__DisplayClass7.<Save>b__1() in d:\DEVELOP\MyProject\3rdparty\fallen-8\Fallen-8\Persistency\PersistencyFactory.cs:Zeile 199.
       bei System.Threading.Tasks.Task`1.InnerInvoke()
       bei System.Threading.Tasks.Task.Execute()

Environment is 64bit Windows 8.1 with lots of free RAM, if that matters.

Implement a R*Tree index plugin

Inform the Frontend User when the amount of expanded Nodes is more than a specific number ( this number must be configurable )

I just noticed that I can modify properties via the REST service interface, at least it is part of the contract.

But when I link directly against the assembly, I can't do that, because TryAddProperty() (which should be named TryAddOrUpdateProperty()) and TryRemoveProperty() both are internal, not public, hence not accessible from the outside.

Is there a reason for that which I overlook, or is it just inconsistent and they could safely be set to public without running into trouble?

It should be possible to load/stop/start plugins via REST

some kind of regEx index

It shall be possible to replicate f8 on multiple machines / local vm instances. This is for the purpose of scaling read requests and to increase fault tolerance. For that, we have to develop a way to synchronize db states between multiple f8 instances including message transfer, sync-protocols, error detection and -correction.

The first part should be the investigation on the topic. We need to compare different technologies and different protocols. The result of this phase also includes be a decision between centralized (M/S) oder decentralized (P2P) organization.

After that we have to select a technology to use for the implementation. There are different frameworks available, but we currently prefer our own implementation based on WCF.

When we selected a technolgy, we build a first prototype :)

It should be possible to know how many edges are within an edge-property

System.ArgumentException: Sections cannot exceed 32 bits in total
at System.Collections.Specialized.BitVector32.CreateSection (Int16 maxValue, Section previous) [0x00041] in /private/tmp/monobuild/build/BUILD/mono-2.10.9/mcs/class/System/System.Collections.Specialized/BitVector32.cs:190
at Framework.Serialization.SerializationWriter..cctor () [0x00000] in :0

Create multiple REST-endpoints to be able to search (idx/scan) the fallen-8 via REST

Create a link/button to the fallen-8 Issues-Page:
https://github.com/cosh/fallen-8/issues

I'm a big fan of the idea of having a single data store for both oltp and olap, although query prioritisation immediately becomes a concept of interest when these are combined. In my mind a graph database is always really just processing on big priority queue of nodes. From this perspective I'm wondering how easy it would be to add naive traversal prioritisation?

I haven't got a full grasp of the current codebase yet, will have a look myself over the next few days.

Implement bidirectional bfs

Create a button/link to the fallen-8 wiki:
https://github.com/cosh/fallen-8/wiki

It should be possible to serialize and deserialize services. On deserialization it should be possible to specify whether the services should be started or not.