Git Product home page Git Product logo

springfield's Introduction

Springfield

Springfield is "big rolla".

It takes the rolla storage engine design and scales it up for use in larger 64-bit systems with millions of keys and data files on the order of GBs+. Like rolla, it is designed assuming it runs on ~100us random access times of solid state disks.

Unlike rolla, it remembers buckets counts in db files, so you don't need to walk on eggshells making sure you use the same NUM_BUCKETS every time with a given db file.

Springfield is fully thread-safe. In fact, You can also compact and "upgrade" the db to higher bucket count while remaining online to get your performance back when the keyspace grows. And, on SSDs, when the page cache is not large enough to cover you, parallel gets on separate threads speed things up nearly linearly.

Springfield also uses CRC sums to validate data integrity of keys/values on disk.

Status

Springfield is alpha. There is at least one known bug/race in online compaction, but everything else seems to work fine.

Still, don't use it in production, yet!

Performance

It is pretty fast on SSDs. It outperforms Tokyo Cabinet's TCHDB on most workloads, and often by a wide margin as key counts and db sizes go up.

springfield's People

Watchers

 avatar James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.