Git Product home page Git Product logo

bigdatata / cubefs Goto Github PK

View Code? Open in Web Editor NEW

This project forked from cubefs/cubefs

0.0 0.0 0.0 123.89 MB

CubeFS is a cloud native distributed storage platform.

Home Page: https://cubefs.io

License: Apache License 2.0

Shell 1.61% JavaScript 0.02% C++ 57.94% Python 1.11% Perl 1.52% C 1.15% PHP 0.07% Java 6.92% Go 28.45% Assembly 0.10% PowerShell 0.08% Makefile 0.65% CMake 0.29% Dockerfile 0.04% NASL 0.05%

cubefs's Introduction

CubeFS

CNCF Status Build Status LICENSE Language Go Report Card Docs CII Best Practices

 Community Meeting
The CubeFS Project holds bi-weekly community online meeting. To join or watch previous meeting notes and recordings, please see meeting schedule and meeting minutes.
CubeFS

Overview

CubeFS (储宝文件系统 in Chinese) is a cloud-native storage platform hosted by the Cloud Native Computing Foundation (CNCF) as a incubating project.

CubeFS has been commonly used as the storage infrastructure for online applications, database or data processing services and machine learning jobs orchestrated by Kubernetes. An advantage of doing so is to separate storage from compute - one can scale up or down based on the workload and independent of the other, providing total flexibility in matching resources to the actual storage and compute capacity required at any given time.

Some key features of CubeFS include:

  • Multiple access protocols, allowing users to access the same file over multiple protocols, such as POSIX/HDFS/S3
  • Highly scalable metadata service with strong consistency
  • Excellent storage performance via optimizing large/small files and sequential/random writes
  • Multi-tenancy support with better resource utilization and tenant isolation
  • Hybrid cloud I/O acceleration through multi-level caching
  • Flexible storage policies, high-performance replication or low-cost erasure coding

Documents

Benchmark

Small file operation performance and scalability benchmark tested by mdtest.

File Size (KB) 1 2 4 8 16 32 64 128
Creation (TPS) 70383 70383 73738 74617 69479 67435 47540 27147
Read (TPS) 108600 118193 118346 122975 116374 110795 90462 62082
Removal (TPS) 87648 84651 83532 79279 85498 86523 80946 84441
Stat (TPS) 231961 263270 264207 252309 240244 244906 273576 242930

Refer to cubefs.readthedocs.io for performance and scalability of IO and Metadata.

Setup CubeFS

Community

Partners and Users

There is the list of users and success stories ADOPTERS.md.

Reference

Haifeng Liu, et al., CFS: A Distributed File System for Large Scale Container Platforms. SIGMOD‘19, June 30-July 5, 2019, Amsterdam, Netherlands.

For more information, please refer to https://dl.acm.org/citation.cfm?doid=3299869.3314046 and https://arxiv.org/abs/1911.03001

License

CubeFS is licensed under the Apache License, Version 2.0. For detail see LICENSE and NOTICE.

Note

The master branch may be in an unstable or even broken state during development. Please use releases instead of the master branch in order to get a stable set of binaries.

cubefs's People

Contributors

awzhgw avatar mervinkid avatar shuoranliu avatar leonrayang avatar wenjia322 avatar zhuhyc avatar victor1319 avatar zhuzhengyi avatar bladehliu avatar yinlei-github avatar ansjsun avatar shyodx avatar zhangtianjiong avatar xuxihao1 avatar leeyubo avatar xrefft avatar hooklee2000 avatar leelx avatar xiangpingzhang avatar skypigltp avatar zhengyi avatar vivian7755 avatar wding109 avatar jadewang198510 avatar tianjiongzhang avatar chengyu-l avatar littlejiajia322 avatar ulricazhz avatar yankunli avatar ekaakurniawan avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.