SDFS is a hdfs-like file system. A few important characters:
-
Immutable files.
-
Transparent to user. As a user, you don't need to care about the details about server layout. Simple APIs are provided to do file upload and download:
void FileSystem::put(std::string localFile, std::string remoteFile);
void FileSystem::get(std::string localFile, std::string remoteFile);
-
Fault tolerant. Each file in distributed file system has three replicas. So we can handle two simultanous failure on our server. After machine's failure, all replicas on it will be replicated on other machines.
-
De-centralized. We use a Ring-based key-value store approach to store the data.
-
Efficient. The failure detection module is based on SWIM protocol, which has O(1) workload when there is no failure.
For design details, please check out the report (pdf file).
Store the IP address or hostname of you machines. One should be put in AddrIntro.add, all the others in Address.add.
Timing library.
Support for distributed file system manipulation.
Membership control module using SWIM-protocol. It detects machine joining/leaving/failure.
Helper methods for TCP and UDP connections, and robust data transfer.
Luis Remis (remis2 at illinois dot edu)
Xuewei Zhang (xzhan160 at illinois dot edu)