IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O Began : Mon Jul 6 03:06:45 2020 Command line : /IO500/bin/ior -r -R -s 500000 -a POSIX -v -i 1 -C -Q 1 -g -G 27 -k -e -t 47008 -b 47008 -o /mnt/cephfs/datafiles/2020.07.06-02.27.35-scr/ior_hard/IOR_file -O stoneWallingStatusFile=/mnt/cephfs/datafiles/2020.07.06-02.27.35-scr/ior_hard/stonewall Machine : Linux sr650-1 Start time skew across all tasks: 321.14 sec TestID : 0 StartTime : Mon Jul 6 03:06:45 2020 Path : /mnt/cephfs/datafiles/2020.07.06-02.27.35-scr/ior_hard FS : 61.9 TiB Used FS: 22.3% Inodes: 13.9 Mi Used Inodes: 100.0% Participating tasks: 200 Using reorderTasks '-C' (useful to avoid read cache in client) Options: api : POSIX apiVersion : test filename : /mnt/cephfs/datafiles/2020.07.06-02.27.35-scr/ior_hard/IOR_file access : single-shared-file type : independent segments : 500000 ordering in a file : sequential ordering inter file : constant task offset task offset : 1 nodes : 10 tasks : 200 clients per node : 20 repetitions : 1 xfersize : 47008 bytes blocksize : 47008 bytes aggregate filesize : 4.28 TiB Results: access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter ------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ---- Commencing read performance test: Mon Jul 6 03:06:45 2020 WARNING: Expected aggregate file size = 4700800000000. WARNING: Stat() of aggregate file size = 1041293011200. WARNING: Using actual aggregate bytes moved = 1041293011200. read 11704 261083 84.84 45.91 45.91 0.001829 84.84 0.000370 84.85 0 Max Read: 11704.16 MiB/sec (12272.70 MB/sec) Summary of all tests: Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum read 11704.16 11704.16 11704.16 0.00 261076.79 261076.79 261076.79 0.00 84.84630 NA NA 0 200 20 1 0 1 1 0 0 500000 47008 47008 993054.4 POSIX 0 Finished : Mon Jul 6 03:08:10 2020