xornand / rdsn Goto Github PK

All pull requests please now go to https://github.com/imzhenyu/rdsn for automatic integration with latest version. We will preriodically update this repo. Thank you.

• [Case] RocksDB made replicated using rDSN!
• [Tutorial] Build a counter service with built-in tools (e.g., codegen, auto-test, fault injection, bug replay, tracing)
• [Tutorial] Build a scalable and reliable counter service with built-in replication support
• [Tutorial] Build a perfect failure detector with progressively added system complexity
• [Tutorial] Plugin my own network implementation for higher performance
• Installation

I am a developer | researcher | student =>

Robust Distributed System Nucleus (rDSN) is an open framework for quickly building and managing high performance and robust distributed systems. An early version of rDSN has been used in Bing for building a distributed data service, and the system has been online and running well. Based on the feedbacks, rDSN is improved and now made public with the MIT open source license. The idea is to advocate a coherent and principled meta stack that distributed applications, tools, and frameworks are developed independently and integrated (almost) transparently to benefit each other. Following are some highlights for different audience of this framework.

I am a developer | researcher | student =>

• Highlights (more)
• compatible code generation via Apache Thrift and Google Protocol Buffer
• automatic test against various failures and scheduling decisions with reproducable bug
• flexiable to plugin your own module to adapt to existing culture or for higher performance
• automatically turn single-node service to a scalable and reliable service with built-in replication
• Have concerns? =>

• Layer 1: quick service development, plus with support from many tools and policies: simulation, fault injection, tracing, profiling, replay, throttling, ... (see Tutorial)

• Layer 2: to a partitioned? and replicated? service with simple configuration and minor further development cost (see Tutorial)

• Layer 3: compose workflow across multiple services in a declarative way to handle end-to-end incoming workloads (coming later)

• multiple-platform support (Linux, Windows, Mac)
• Compatible code generation with Thrift and Protocol Buffer, extensible for others
• enhanced RPC library with multi-port, multi-channel, multi-language-client support
• adapt to your own environment, or rDSN does not perform good enough? rDSN is highly extensible that you can always plugin your own low level constructs (network, logging, task queue, lock, performanc counter etc.)
• flexible deployment - single executable, multiple roles, multiple instances
• more concerns, please do not hesitate to let us know

• reliably expose dependencies and non-determinisms in upper systems to the underlying tools at a semantic-rich level (i.e., task)
• completeness made easy for tools
• reduced state space for tools to handle
• easier to map tool results to application logic
• dedicated Tool API for tool and runtime policy development
• transparent integration of the tools with the upper applications to make real impact

• configurable system complexity to learn step by step
• single thread to multiple threads
• synchronous and reliable messages to asynchronous and unreliable ones
• easy test, debug, and monitoring for system understanding, as with the developers
• easy further tool development, as with the researchers

rDSN is provided in C++ on Windows, Linux, and Mac, with the MIT open source license. You can use the "issues" tab in github to report bugs. For non-bug issues, please send email to [email protected].

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.