Assume that Redis is safe and require that everywhere messages are stored when they could be lost due to actor death

Make it C# friendly

Need to monitor pipeline for stale, started but unprocessed messages and errors as well.

All ACK must be atomic, as well as children continuations (now it is just a loop)

This is instead of supervision. Not critical now, but must have in the long run.

"ActorIndentity".PostAndGetReply<Tinput, Toutput>(input)... - generic extension method on string

Do not need to write a fake definition on one platform to call actor from another platform, only serializable Tinput/Toutput POCOs/POJOs

Bad that I skimmed the article about Pinterest and did't read every line: http://highscalability.com/blog/2013/4/15/scaling-pinterest-from-0-to-10s-of-billions-of-page-views-a.html

"Hacked up a copy of Pyres, a Python interface to Github’s Resque queue, a queue on built on top of redis. Supports priorities and retries. It was so good they replaced Celery and RabbitMQ with Pyres."

Resque's job ~= Actors

Python is much more relevant than JVM for things that are not covered with R, which is P/Invokable (and could be distributed via current .NET actors). It's very hard to make .NET and Pythons friends in any way other than message passing and shared data.

Multiprocessing + py-redis + built-in json should be just enough to make it work. Should create an abstract class similar to the current .NET version. A single abstract method should do all the job, assuming it receives a json string.

Data copying and message passing limit use cases for this. E.g. real-time stream processing is not suitable, but parsing web pages using Newspaper or batch running parameterized number crunching routines is perfectly fine.

Will only target Python 3.5+ (latest Anaconda distribution). No plans to support calling .NET from Python, since .NET is much more superior for infrastructure, but lacks many rich libraries from Python.

Redis is a special case of a message bus with Pub/Sub semantics. ZMQ, Azure, SNS, manual many-to-many connections could also implement channels.

Every actor instance could have an identity (could be just persisted GUID). A mailbox (F# actors style) could be implemented in the usual way (a queue inside an instance), but in Redis it could also be implemented and persisted separately for each actor instance by its ID (not only a shared queue for a virtual actor that potentially has many instances).

Less obvious thing is how to implement virtual actors on top of a message bus without a central node or a shared storage like Redis. One idea is to keep a list of all actor instance IDs for each virtual actor and round-robin or some other distribution algo. A combination of ZMQ pub/sub and fanout could work, with all NetMQ reliability goodies already done (watch out for LGPL plague if modified version is used!). Sad thing is that to get some kind of reliability in case of in instance dies, will have to reinvent a lot of wheels similar to Redis cluster, but in many cases Lazy Pirate (retry until success) or Fire and Forget patterns are good enough.

The main differences from RESTful services are the distributed nature, autodiscovery of worker instances, low latency and very light weight - this should all run from a single dll/exe with near-zero config and no admin privileges for http hosting. One single config is good, e.g. UDP port for service broadcast and actors discovery (ZMQ's beacon is just a special case, the discovery should also be hidden behind interfaces. E.g. in AWS there is no broadcast, but SNS replaces it.)

Orleans/MBrace is overkill for local use case (several processes on the same machine or LAN, many of them could be ephemeral short-lived). SignalR model is very good, but too many stuff that turns into garbage in any non-http case, while MSFT's bullshit with WebSockets-only-on-Win8 is why people hate the company. There could be much more work to throw away unneeded things from SignalR and implement TCP transport or embedded Katana-like hosting than to build a similar simpler model on top of ZMQ.

In addition to Post and PostAndGetReply also need Pub (post to all), Poll (post to all and get reply from all), PollN (Poll but return after N replies, not really needed, but Poll is likely a special case of PollN when implemented. But, how do we know the total number of instances!? Maybe Poll/PollN are both not needed, but only parallel PostAndGetReply.).

Especially SSv4 dependencies are not welcomed. This is a good stuff if used together with Services API (and when I am too lazy to reimplement Auth stuff), but it is not free stuff.

Performance is a problem that is nice to have, now I do not have such a problem.

start only if some tags in an environment are set
e.g. AWS tags for crawlers, app servers, front ends - the deployment code should be
identical everywhere but the actor behavior should depend on the tags in the environment

Either add another interface or redefine IPerformanceMonitor as IEnvironmentMonitor

TRoot without annotations conflicts with string, e.g. redis.HSet("a", "b", "c") needs redis.HSet("a", "b", "c")

where : new() constraint on TRoot rules out strings as roots

or just go with annotations?

That should be quite easy, almost like their thunk server example. But do not forget about medium trust from the very beginning!
A dynamic actor should just accept a lambda + options similar to static ones.

AWS Lambda et al. either make the Actors part of this project irrelevant, or there could be some simple way to integrate the two. Need more research.

Some interesting Redis-related pieces could be reused directly as a part of a bigger application, but not as a library.

profiler throws, need to investigate manually

Need WhenAll similar to TPL's Task but with the Ractor guarantees, with a signature like that: WhenAll(ActorName, Requests[]) : Task<Replies[]>. Implementation is just an Actor that accepts an array of requests, spawns subactors and awaits with TPL's WhenAll. But in case of its death we should check existing results of subactors, or just re-await by ids using existing logic so that stale jobs of dead subactors will be picked up and reposted by live ones.

Obvious use-case is a grid search for a maximum of a very expensive function, such as some crazy strategy on small timeframes. But the performance and costs benefits of horizontal partitioning must be significantly above the costs of data copying to workers.

Alternatively, WhenAll could be a separate class that inherits Actor and accepts a subactor as a type parameters. Then Ractor's ContinueWith will work with it. Could call it BatchActor.

While trying to do some 'clever' built-in auto serialization and namespaces, I got allocations on practically every Redis operation. Therefore, no future integration with Spreads or any latency-sensitive stuff with this project. I already use DynamicContext for rapid prototyping with automatic per-table migrations, but Dapper for stable things. Redis part should also be seen just a convenience short-cut. (Should at least add a buffer pool to JSON.NET later). Keep calm and trust GC in this case!

correct distributed blocking lock (honors the order of consumers trying to access a locked resource). ETA next release if the idea is feasible

Current ServiceStack and 'the Redis book' implementations only enter atomically, but lock by looping so that the third thread could steal a lock from the second thread if the two are waiting for the first.

Atomic lua scripts should work like Monitor.Enter/Exit-like behavior.

See Keys hash tags section http://redis.io/topics/cluster-spec

Not all migrations are implemented, still throwing dbo. prefix errors with version 6.9.6. They couldn't fix it for so many years and do not have an easy repo on GitHub just to submit a pull request. F^&k them, use PostgresQL as default target and write a paragraph on this in readme.

F# is a nice language, but async is much worse than TPL for CPU-bound tasks. And Ractor microbenchmarks show that, e.g. with optimistic local execution when there is not trip to Redis it still takes 1ms for ping/pong. Here is a task monad slightly optimized from original Fsharpx: https://github.com/buybackoff/FSharpAsyncVsTPL/blob/master/FSharpTasks/TaskBuilder.fs

Will be done in Scala with actors definition in any language (interface/abstract class). Should not have BS with C#/F# API.

Interop between CLR and JVM is done via JSON messages

No idea if that will work and when

It is just stupid, if we could allow for in-memory execution without acknowledging that we have received a task, we could do just Task.Run(...)...

The benchmarks of RedisQueue/AsyncDictionary show that we cannot get significantly above 12k messages per sec on my machine, so Spreads-like performance will never be achievable even on the same machine (there is already mmaped IPC from Aeron for this).

Should remove all this complexity and keep only the fully reliable default case when messages always go via Redis and are stored there.

Currently Ractor does processing on first come first served basis - any actor could take next available job before a previous had finished.

Need to add singleton (just a distributed lock on a tasks queue) and multicast

Sometime one needs to stop processing everywhere, or on some nodes

Actors should have tags and system messages should be tagable. (Tags could be read from AWS instances, that gives control where actors should start receiving and where only post is available)

With BLPOP/BRPOP it is trivial to create a distributed blocking collection.

Marc Gravell did a spectacular job with Tasks that so naturally fit into F#'s async. Such queues could trivially replace internal queue is MailBoxProcessors. If we need producer/consumer pattern and do not care much about order, we could use such a distributed queue without locking (that will behave like AWS SQS)

pub/sub is already build-in, but many other zmq-like patterns could be done safely with a lock when there are many distributed clients

Not so sure about anything other than blocking queue/stack, probably

Use Interlocked.Exchange. Just need to use the same logic as for GetSubsriberChannel in BookSleeve. This will allow usage like use cloned = +conn in a for/while loop without opening a new connection each time. On Dispose() cloned connection is returned to the standby slot, or disposed normally is the slot is occupied by another one.

Dynamic object is cool to add state to actors and make then truly actors, not just distributed workers. Need to find a prototype somewhere on my disk.

State could be per virtual actor, with a distributed lock that should behave similar to a plain monitor or event better as non-blocking async disposable. This is the main case. Dynamic is good for implementing Redis storage via the dynamic object methods. Orleans has interfaces and some codegen, Orleankka has a 100 pages presentation on some better way but tl;dr. We are ok with dynamics for that Redis is not typed.

(Or state could be per instance, which could add convenience in some cases, but not obvious ones. We should separate actor tasks into very granular units of work, and we already have result caching built in so on replay a chain of actors will pick all ready results and continue where an actor died.)

At first there was a mess with System.Threading.Tasks on .NET 4.0... Upgraded to 4.5, the whole thing compiles, runs, but doesn't work (while separate parts do work). Need to make basic use case as easy as download NuGet, copy samples, run...

The point was to simplify the data access layer as much as possible, so that I can always add Fredis via nuget and use the minimalist API whenever I need to work with data.

Shards are complicated thing. Without any chards just assume we are in epoch 0 and use main connection for everything.

Think about "Deprecated for Orleans"???

Redis-based message persistence is a cool thing, but every other scenario is covered by Orleans? Need to play with it more before adding any feature that does not utilize Redis awesomeness.

Orleans has the concept of pluggable storage providers to support grain state persistence. (https://convective.wordpress.com/2014/04/20/a-first-look-at-project-orleans/)

There is a lot to learn from Orleans, M-brace and Akka, and then again leverage Lua in Redis to make it simpler. This goes together with #39

With current design continuations with self won't work, e.g. greeter.ContinueWith(greeter) (they will share result id and won't be able to distinguish calls). And that is not a completely stupid case as it may look in the Greeter test. E.g. for an AddOne actor it makes more sense to call it twice to add 2 - still stupid, but not as Greeter.

Probably need some prefixing for resultId. Also need to think again why one should use continuations over calling one actor from another. One clear reason is that in such a case actors are no longer an independent unit of work, and to make them independent we will need additional actors that will call 1st and pass the result to the 2nd - that is exactly what continuators do. Without built-in continuations each such case will be add-hoc manual coding.

For DbMigrationsConfig, for Connections, everywhere.... Connections.fs is kind of static abstract factory, not very good. Autofac could registered named interfaces.

There must be a simple way to get messages in order and reliably, e.g. simple ever increasing sequence number that is assigned from withing Lua, then messages are persisted in a log (ZSET with int53 key) for a period of time - Kafka style. Clients use pub/sub and just query the zset for a missing sequence number. Everything is atomic inside Lua scripts in Redis. Really want to avoid using Kafka or another additional moving part for persistent messaging. Need to google or ask SO if there is already a messaging framework on top of Redis, leveraging Lua. It must be even simpler and more reliable than zmq! Then we could plug Spreads into Ractor.

Need to look into SE.Redis what fire and forget means exactly...

If a message comes and we return, the sender may discard it before it is saved and then the receiving agent could die

Not 100%, but almost always in case of my service

protobuf-net is obvious choice here. should be trivial with extensions or new methods on Connection type

Another obvious thing to do with Redis... Should take SQLProvider as a guide.

Brain dump:

• simple objects and special objects: simple objects are ones that could be used by any other client, special objects are Fredis specific
• prefix for special objects, e.g. f# or frn:... (Fredis Resource Name)
• frn: is generic, urn: is string only, or give up urn: completely for frn:
• special objects could be: locks on simple objects, queues from another issue ( #4 ), also several Redis keys could represent one F# objects, e.g. for a queue with s single consumer frn:queue:abc - for Redis list, and frn:queue:abc:lock for a Redis string used for lock.
• special objects in most cases should be containers or combinations of containers (to use Redis compression for small containers) with many values inside, so we should not worry about additional service keys per container and their overheads (which are quite big in Redis)
• from TP we access objects by type, e.g. TP.Lists.my_key maps to frn:list:my_key
• when we know types by key, we could show only relevant methods and protect from silly errors of calling wrong methods on wrong object types.
• BS connection methods could be mapped directly to new types: RList, RHash, RSet, RZset, etc
• Types should be generic, e.g. RList<'T> with automatic serialization ( #2 ).

TP could be useful to work with distributed data flows during design time (e.g. setting up queues, producer-consumer patterns, any other containers with runtime-constant keys). Also, TP could include an actors system as a part: RActor type added to the mix of RList, RHash, etc. Supervisor is elected by a lock with a timeout - if the state is stored in Redis then we do not care who is doing the processing. Periodically each client tries to acquire the supervisor lock, while the current supervisor extends his lock if it is alive. If the supervisor dies, some other client will replace it soon.

It would be "nice to have" containers of containers with relationship logic hidden since the use case is pretty common. This maps to foreign keys support in SQL (https://github.com/fsprojects/SQLProvider/search?q=foreign+key&ref=cmdform). Should be lazy (like in EntityFramework), should support multiple slaves. Should think about this after the core stuff is done (using JSON strings already works fine).

Just use Resque-web

OR

Take Yo.Net as the structure - define contracts and services that could be plugged into any SS host.

• Fredis() init call should add services to host if that is possible (or will have to manually import services from Fredis assembly in AppHost config - not a big deal)
• On Actor.Start() HINCR actor Id in a hash of running actors (and decrement on stop) - will get a number of worker nodes where an actor is working
• Create an actor that will poll running actors queues for stats, update their stats metrics and push via SignalR to dashboards

Kafka gives the same functionality that is used here for actors. Given that it is from JVM world and quite mature (even with v.0.8), I should work on interfaces and abstract away Redis/Kafka/SNS/SQS/ whatever distributed storage. Must be simple, the only thing I use now is pub/sub on channel with persistent storage - actually Kafka model with Redis as a superset of what Kafka could give. Kafka is persistent, while Redis is "assumed" to be persistent in memory with AWS multi-az insfrastructure or similar.

Some examples how people are doing this:

FShap.Core Query:
https://github.com/fsharp/fsharp/blob/master/src/fsharp/FSharp.Core/Query.fs
https://github.com/fsharp/fsharp/blob/master/src/fsharp/FSharp.Core/Query.fsi
ILBuilder: https://github.com/kbattocchi/ILBuilder
RX: http://mnajder.blogspot.ca/2011/09/when-reactive-framework-meets-f-30.html
DSLCheatsheet: https://github.com/dungpa/dsls-in-action-fsharp/blob/master/DSLCheatsheet.md

http://files.meetup.com/2922732/Computation%20Expressions%20(April).pdf

Need native Redis syntax with typed values (any serializable .NET object, not only POCOs)

The goal to have something like this:

let conn = .. // new connection
let version = // should equal pong
    fredis conn {
        PING
    }

let value:'T = new T()
let key = "mykey"
    fredis conn {
        SET key value // automatic serialization
    }   

let value2:'T = // atomatic deserialization to 'T
    fredis +conn {
        GET key
    }

let channel = "mychannel"
let value = 
    fredis %conn {
        SUBSCRIBE channel
    }

Need to finalize all Redis commands in Persistence project.

Fredis compiles and starts - should work just fine out of the box, but there are issues at Stackexchange.Redis level (e.g. their 42th issue + http://stackoverflow.com/questions/23871110/stackexchange-redis-failing-to-connect-with-mono-in-mac-os-x)

As a part of learning Hopac try to migrate from using all low-level sync (AutoResetEvents, etc) primitives to Hopac constructs