Cloud Storage Environment
The Cloud Storage system in CSTCloud is a distributed file system featuring elasticity,
reliability, and high performance. Based on the robust network connection, this cloud
storage service adapts to various native operating systems with standard transmission
protocols and supports both IPV4 and IPV6 access to ensure data consistency. With high
IOPS and low latency, this cloud storage service could reach the performance of 99.99999%
data durability to guarantee data safety and reliability. With the latest updates in
October 2018, our cloud storage service has provided services to different research groups.
For example, in the CASEarth program, up to 800 TB of disciplinary research data are shared
through our system.
Main features
-
Seamless integration of data access, mainstream applications, and workloads following
standard file system semantics and in favor ofboth FTP and HTTP protocols.
-
In favor of open data by supporting data access from multiple computing nodes simultaneously.
-
Encrypted and parallel data transfer are used to guarantee fast and safe curation of massive data.
-
Dual-stack access is available by transferring and managing data through both IPV4/IPV6 protocols.
Object-based Storage Technology and System
Based on the general cloud storage environment, an object-based storage system has been
developed on the standard X86 platform enabling the management of 200 nodes with a data
storage capacity reaching 24PB in total. This system enables data access via Web, RESTful
API, FTP, or internal mounting, guaranteeing a high fault tolerance rate of over 99.999%.
To improve the metadata access performance in this object-based storage system, especially
dealing with millions of buckets and trillions of objects, we designed a distributed
hierarchical metadata management mechanism. Thus, the platform can achieve a very high
throughput of over 1.2M IOPS in retrieving metadata.