Python parody of Apache Hadoop Distributed File System (see design overview here). Uses AWS EC2 for server communication, and S3 for persistent storage.
The NameNode master server manages the namespace of the file system and mediates access to clients.
The DataNode facilitates storage and retrieval of files, which are internally split into blocks (128 MB) and replicated up to a factor of 3.
- rpyc
- boto
- S3Handler