IO500 - Pengcheng CloudBrain with OceanStor A800

System Purpose

Question asked

Please describe the purpose and general usage of the submitted system. This would include the types of typical applications it supports (e.g., defense applications, molecular dynamics, benchmarking, system test, systems research), and the general use and purpose of the data generated by the applications running on it.

The system is designed for research on advanced large artificial intelligence models, providing massive computing power and extreme storage throughput for large-scale model training.

Availability

Question asked

Please provide the deployment timeframe of the submitted system, or for on-demand cloud systems, the general period over which it is deployed and destroyed.

Please describe the availability of the system to users and who are its set of most regular users.

The system is available to early pilot users in March 2026. It targets a diverse user base, with a primary focus on large language model (LLM) research and development.

Storage System Software

Question asked

How is this storage software available? (e.g., commercially, open-source, not publicly available) Note that if the storage software is not open-source or commercially available, then a general description would be requested, but this would limit the submission’s reproducibility score.

Can anyone download/purchase this software?

List either product webpage or open-source repo of the exact software used in IO500.

Give any and all additional details of this specific storage deployment: (e.g., type of storage server product such IBM ESS or DDN SFA400X2, use of Ext4 or some other file system on each storage node, dual connected storage media to storage servers).

The storage software stack is a highly optimized in-house implementation built for extreme throughput and scalability, with its source code currently not publicly available. The underlying hardware is Huawei OceanStor A800; each server / chassis carries 48x NVMe SSDs and 32x 200 Gbit/s RoCE Ethernet ports.
Every OceanStor A800 server contains dual controllers. Each controller is equipped with four-socket Kunpeng 920 CPUs (64 cores per socket, 2.4GHz frequency, 2 threads per physical core) plus 1024 GB DRAM.
All NVMe SSDs adopt dual-port design, with each port wired to a separate controller to enable simultaneous read / write operations from both controllers. The dual-port NICs follow a similar mapping scheme: each port binds exclusively to its affiliated controller, yet only the assigned controller can initiate network traffic over that port.
Of the total 32 RoCE ports per server, 24 serve as uplinks connecting compute / client nodes, while the remaining 8 are reserved for internal storage cluster traffic such as erasure code for data redundancy. In this test deployment, high availability and data redundancy features were fully disabled, leaving these 8 internal ports idle with zero data transmission. This explains the reported 24x 200 Gbit/s RoCE ports / links per storage server.
Given the four-socket layout, Non-Uniform Memory Access (NUMA) latency introduces severe I/O performance penalties. We implement hardware affinity to colocate nearby NVMe drives and network ports per socket: each socket is assigned with 12 SSDs and 4 RoCE ports (3 uplinks + 1 reserved internal link).
During this test, the dual controllers operate in active-active mode, enabling both to jointly deliver storage performance.
For full product specifications, please refer to:
https://e.huawei.com/en/products/storage/ai-storage

Runtime Environment

Question asked

State here that you provided all scripts/documentation that would allow someone else to reproduce your environment and attempt to achieve a similar IO500 score as the submitted system.

NOTE: provide all files/documentation/scripts that would enable a user to build your environment and deploy the custom scripts, software, or config files once they have obtained the appropriate storage system hardware and software. These may be included into the io500.tgz or into the extraCodes upload on the submission form.

Several examples include:

Commands used to set striping information (either for the entire system or for particular directories)
File system config and tuning information (or a reason why this is not available due to lack of root access, etc) on each node type (e.g., non-default config parameters on all three types of Lustre nodes: MDS, OSS, and client)
Any additional scripts utilized that impact IO500 execution beyond the io500 config file. For example, with IBM Spectrum Scale, the output of mmlsconfig, mmlscluster, mmlsfs, and mmlsfileset commands (and possibly even a dump of the configuration if possible). Each file system probably has similar type of config/tuning information that would need to be shared for a user to fully reproduce the environment.

The submitted tarball contains the io500.sh execution script and related .ini config file for environment replication. The script uses default settings with no additional tuning. The entire benchmark is launched via OpenMPI 4.1.4 over one extra 200 Gb/s management Ethernet network using TCP protocol to isolate traffic from the storage system. Our testing verifies that consistent and comparable IO500 results can be reliably reproduced.

Fault Tolerance Mechanisms

Question asked

Does your system have a single point of failure as defined by “Definition 7” of a Production System? Please describe all mechanisms that provide fault tolerance for the submitted storage system. Be specific to your submission, not general storage system capabilities.

Power (For example, can power failure of a node or rack bring down the storage system?)
Networking (For example, can failure of a single NIC bring down the storage system? Remember that any networking failure that brings down multiple nodes is acceptable)
Inter storage data and metadata server (For example, what type of data protection mechanism does the storage system use to keep operating during node failures, e.g., active-active servers, client-directed RAID, declustered RAID, erasure-coding, 3-way replication)
Intra storage data and metadata server storage media (For example, how does the storage eysstem protect against fiailure of a single storage device within a node, e.g., raid5)

Please list any additional information needed to determine whether this system has a single point of failure.

All storage servers feature redundant fans and power supplies, plus multiple NICs for front-end client and back-end inter-storage connections. To achieve optimal performance, we use raw network links without advanced settings like multi-rail or bonding. Controllers in the same server / chassis run in active-active mode, and failover as well as high availability functions are disabled. No redundancy was set up for data layout during testing.

Execution

Question asked

Please provide a description of how the IO500 benchmark was executed, e.g., via system scheduler (e.g., SLURM) to run a job on the compute cluster, which initially ran a setup process to configure the client and file system, and then started the full benchmark.

During the IO500 benchmark execution was the system entirely dedicated to running the benchmark or were there other jobs running in the same cluster and storage system?

The IO500.sh benchmark script is launched from a designated client node, which hosts the rank 0 process. The system is currently undergoing testing and evaluation, and operates in exclusive mode during the benchmark run, with all resources fully dedicated to IO500 execution.

Caching

Question asked

Please describe all caching mechanisms in client/server that were utilized during the IO500 run. This could include caching in any storage medium (e.g., SSD, RAM).

A few examples would include:

Client data/metadata caching (in Linux page cache or in other memory cache)
Client side NVMe read-only data cache
Storage server metadata/data caching in RAM
Storage controller caching
RAID card caching

Client nodes run the openEuler Linux distribution, and all worker processes' IO activities are intercepted and redirected to our in-house implementation, so standard file system caching is not used. The file system client on each node maintains an in-memory cache that stores only metadata, with data object caching fully disabled. On the storage servers, the service software leverages maximum available memory for both data and metadata caching. There is no additional cache employed.

Data Source

Question asked

Is the submitted system a stand-alone storage system or an acceleration layer in front of another storage system that is the source of truth for all application data? This question relates to whether the submitted system is a burst buffer layered on primary storage or primary storage itself.

The benchmark involves no additional I/O layers such as burst buffers. All I/O requests are directed to and persisted directly on the storage system itself.

Trust

Question asked

Please describe any steps taken to ensure that the results are trustworthy.

Did you run the benchmark multiple times and get similar scores?
Did you validate the score is below the physical capabilities of the deployed hardware?
How did you validate that the storage system follows the IO500 rules regarding persistence, which states, "All data must be written to persistent storage within the measured time for the individual benchmark, e.g. if a file system caches data, it must ensure that data is persistently stored before acknowledging the close"?

While the submitted score was obtained after extensive tuning of various configurations, similar performance can be consistently reproduced under comparable setups. The measured performance is reasonable and well within the hardware's inherent capabilities. Specifically, the achieved bandwidth is bounded by the minimum of the aggregated raw network path bandwidth and the aggregate SSD bandwidth. Data persistence is guaranteed by explicitly flushing in the underlying system call interception code, which is further validated by the absence of tailing network traffic.