Git Product home page Git Product logo

alluxio-internal's Introduction

Alluxio-Internal

经过近期对alluxio的研究,本着自身学习记录,同时因为当前对alluxio似乎还没有比较全面、深入的分析。这边系统的对alluxio进行源码分析,同时给出在工作过程中遇到的坑,和解决的坑。alluxio使用的源码是1.4.0。

##简单介绍

alluxio和hdfs有些类似、都是分布式的文件系统,hdfs基于磁盘介质存储、alluxio基于内存介质存储;hdfs基于replica方式进行容错、alluixo基于lineage的方式进行容错(目前容错性处于test阶段,并不完善,建议重要数据还是需要持久化到底层的文件系统);alluxio和hdfs都有类似的文件操作api、类似的shell命令(目前alluixo并没分admin和非admin命令);alluxio和hdfs都是基于文件块的形式存储数据,都是典型的master slave集群架构,都有master ha等。同时它们也有很多细节的区别:alluixo RPC使用thrift,hdfs使用的是protobuf;alluxio更多的用来加速上层计算框架,hdfs则更多的用来持久化存储;alluxio对上层的计算框架的locality不如hdfs等。

##主要内容 对alluixo进行比较全面的分析,将从以下几个方面着手。

  1. Build And Deploy 编译部署alluxio
  2. How to use alluxio alluxio的使用
  3. Alluxio RPC alluxio RPC底层thrift介绍
  4. Allxuio Block alluxio block存储与管理
  5. Alluxio Client alluxio Client源码分析
  6. Alluxio Master alluxio Master源码分析(未写)
  7. Alluxio Worker alluxio Worker源码分析(未写)
  8. Alluxio security alluxio 认证授权源码分析 (未写)
  9. Alluxio bug and fix bug 挖坑、填坑
  10. Alluxio + kerberos(未写)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.