Oakforest-PACS

Institution JCAHPC
Client Procs Per Node
Client Operating System CentOS
Client Operating System Version 7.6
Client Kernel Version 3.1

DATA SERVER

Storage Type NVMe-SSD
Volatile Memory
Storage Interface NVMe
Network Omnipath
Software Version
OS Version

INFORMATION

Client Nodes 2,048
Client Total Procs 4,096
Metadata Nodes 1
Metadata Storage Devices 20
Data Nodes 25
Data Storage Devices 1,200

METADATA

Easy Write 71.51 kIOP/s
Easy Stat 196.15 kIOP/s
Easy Delete 57.05 kIOP/s
Hard Write 11.61 kIOP/s
Hard Read 129.29 kIOP/s
Hard Stat 127.34 kIOP/s
Hard Delete 24.36 kIOP/s

Submitted Files

ior-easy-read
WARNING: fsync() not supported in selected backend.  Using value of 0.
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Oct 27 14:21:39 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/ior-easy/ior_file_easy -O stoneWallingStatusFile=/work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/ior-easy.stonewall -t 2m -b 9920000m -F -r -R -a IME
Machine             : Linux c0049.ofp
TestID              : 0
StartTime           : Tue Oct 27 14:21:39 2020
Path                : /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/ior-easy/ior_file_easy.00000000
FS                  : 23597.0 TiB   Used FS: 45.9%   Inodes: 12312.0 Mi   Used Inodes: 24.2%

Options: 
api                 : IME
apiVersion          : 1.3.2.1 (132037)
test filename       : /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 2048
tasks               : 4096
clients per node    : 2
repetitions         : 1
xfersize            : 2 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 38750 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
WARNING: Expected aggregate file size       = 42606075576320000
WARNING: Stat() of aggregate file size      = 448609334067200
WARNING: Using actual aggregate bytes moved = 448609334067200
read      1267126    633756     0.000068    10158080000 2048.00    0.145269   337.53     0.091967   337.64     0   
ior-easy-write
WARNING: fsync() not supported in selected backend.  Using value of 0.
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Oct 27 13:49:23 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/ior-easy/ior_file_easy -O stoneWallingStatusFile=/work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/ior-easy.stonewall -t 2m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a IME
Machine             : Linux c0049.ofp
TestID              : 0
StartTime           : Tue Oct 27 13:49:23 2020
WARNING: Backend returned error during statfs.

Options: 
api                 : IME
apiVersion          : 1.3.2.1 (132037)
test filename       : /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 2048
tasks               : 4096
clients per node    : 2
repetitions         : 1
xfersize            : 2 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 38750 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 29715 max: 52225 -- min data: 58.0 GiB mean data: 76.6 GiB time: 300.8s
WARNING: Expected aggregate file size       = 42606075576320000
WARNING: Stat() of aggregate file size      = 448609334067200
WARNING: Using actual aggregate bytes moved = 448609334067200
WARNING: Maybe caused by deadlineForStonewalling
write     957128     479168     0.000066    10158080000 2048.00    0.528476   446.43     0.063338   446.99     0   
ior-hard-read
WARNING: fsync() not supported in selected backend.  Using value of 0.
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Oct 27 14:29:54 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/ior-hard/file -O stoneWallingStatusFile=/work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a IME
Machine             : Linux c0049.ofp
TestID              : 0
StartTime           : Tue Oct 27 14:29:54 2020
Path                : /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/ior-hard/file
FS                  : 23597.0 TiB   Used FS: 45.9%   Inodes: 12312.0 Mi   Used Inodes: 24.2%

Options: 
api                 : IME
apiVersion          : 1.3.2.1 (132037)
test filename       : /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 2048
tasks               : 4096
clients per node    : 2
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 1751.18 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 1210171 max: 1210171 -- min data: 53.0 GiB mean data: 53.0 GiB time: 614.9s
WARNING: Expected aggregate file size       = 1925447680000000
WARNING: Stat() of aggregate file size      = 233012094435328
WARNING: Using actual aggregate bytes moved = 233012094435328
read      360649     8051284    615.57      45.91      45.91      0.381246   615.66     0.324391   616.16     0   
ior-hard-write
WARNING: fsync() not supported in selected backend.  Using value of 0.
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Oct 27 14:08:50 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/ior-hard/file -O stoneWallingStatusFile=/work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a IME
Machine             : Linux c0049.ofp
TestID              : 0
StartTime           : Tue Oct 27 14:08:50 2020
WARNING: Backend returned error during statfs.

Options: 
api                 : IME
apiVersion          : 1.3.2.1 (132037)
test filename       : /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 2048
tasks               : 4096
clients per node    : 2
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 1751.18 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 926188 max: 1210171 -- min data: 40.5 GiB mean data: 46.1 GiB time: 300.1s
WARNING: Expected aggregate file size       = 1925447680000000
WARNING: Stat() of aggregate file size      = 233012094435328
WARNING: Using actual aggregate bytes moved = 233012094435328
WARNING: Maybe caused by deadlineForStonewalling
write     593973     13298867   316.61      45.91      45.91      0.745732   372.73     0.730567   374.12     0   
mdtest-easy-delete
-- started at 10/27/2020 14:40:44 --

mdtest-3.4.0+dev was launched with 4096 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/mdtest-easy' '-x' '/work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/mdtest-easy.stonewall' '-r' '-a' 'POSIX'
Path: /cache/1/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00
FS: 23597.0 TiB   Used FS: 45.9%   Inodes: 12312.0 Mi   Used Inodes: 24.2%

Nodemap: 1100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 2 for each phase.
4096 tasks, 4096000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :      57051.556      57047.487      57047.827          0.130
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.556          0.556          0.556          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        538.427        538.389        538.424          0.001
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          1.799          1.799          1.799          0.000
-- finished at 10/27/2020 14:49:44 --

mdtest-easy-stat
-- started at 10/27/2020 14:27:17 --

mdtest-3.4.0+dev was launched with 4096 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/mdtest-easy' '-x' '/work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/mdtest-easy.stonewall' '-T' '-a' 'POSIX'
Path: /cache/1/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00
FS: 23597.0 TiB   Used FS: 45.9%   Inodes: 12312.0 Mi   Used Inodes: 24.2%

Nodemap: 1100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 2 for each phase.
4096 tasks, 4096000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :     196161.043     196147.044     196148.217          0.460
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :        156.596        156.585        156.595          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/27/2020 14:29:53 --

mdtest-easy-write
-- started at 10/27/2020 13:56:51 --

mdtest-3.4.0+dev was launched with 4096 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/mdtest-easy' '-x' '/work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/mdtest-easy.stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /cache/1/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00
FS: 23597.0 TiB   Used FS: 45.9%   Inodes: 12312.0 Mi   Used Inodes: 24.0%

Nodemap: 1100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 2 for each phase.
4096 tasks, 4096000000 files
Continue stonewall hit min: 5831 max: 7499 avg: 6710.6 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :      71515.712      71510.634      71511.060          0.163
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA      91563.437             NA
   Tree creation             :         12.266         12.266         12.266          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        429.529        429.499        429.527          0.001
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.191             NA
   Tree creation             :          0.082          0.082          0.082          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/27/2020 14:08:50 --

mdtest-hard-delete
-- started at 10/27/2020 14:50:18 --

mdtest-3.4.0+dev was launched with 4096 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '4000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/mdtest-hard' '-x' '/work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/mdtest-hard.stonewall' '-r' '-a' 'IME'
Path: /cache/1/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00
FS: 23597.0 TiB   Used FS: 45.9%   Inodes: 12312.0 Mi   Used Inodes: 24.0%

Nodemap: 1100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 2 for each phase.
4096 tasks, 16384000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :      24360.695      24358.948      24359.100          0.056
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          9.107          9.107          9.107          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        175.719        175.706        175.717          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.110          0.110          0.110          0.000
-- finished at 10/27/2020 14:53:14 --

mdtest-hard-read
-- started at 10/27/2020 14:49:45 --

mdtest-3.4.0+dev was launched with 4096 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '4000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/mdtest-hard' '-x' '/work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/mdtest-hard.stonewall' '-E' '-X' '-a' 'IME'
Path: /cache/1/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00
FS: 23597.0 TiB   Used FS: 45.9%   Inodes: 12312.0 Mi   Used Inodes: 24.0%

Nodemap: 1100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 2 for each phase.
4096 tasks, 16384000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :     129301.546     129276.098     129292.913          0.415
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :         33.110         33.103         33.106          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/27/2020 14:50:18 --

mdtest-hard-stat
-- started at 10/27/2020 14:40:10 --

mdtest-3.4.0+dev was launched with 4096 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '4000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/mdtest-hard' '-x' '/work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/mdtest-hard.stonewall' '-T' '-a' 'IME'
Path: /cache/1/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00
FS: 23597.0 TiB   Used FS: 45.9%   Inodes: 12312.0 Mi   Used Inodes: 24.2%

Nodemap: 1100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 2 for each phase.
4096 tasks, 16384000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :     127347.473     127337.773     127339.082          0.367
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :         33.614         33.611         33.614          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/27/2020 14:40:44 --

mdtest-hard-write
-- started at 10/27/2020 14:15:05 --

mdtest-3.4.0+dev was launched with 4096 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '4000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/mdtest-hard' '-x' '/work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/mdtest-hard.stonewall' '-C' '-Y' '-W' '300' '-a' 'IME'
Path: /cache/1/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00
FS: 23597.0 TiB   Used FS: 45.9%   Inodes: 12312.0 Mi   Used Inodes: 24.2%

Nodemap: 1100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 2 for each phase.
4096 tasks, 16384000 files
Continue stonewall hit min: 680 max: 1045 avg: 853.4 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :      11605.906      11604.947      11605.148          0.027
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA      11634.557             NA
   Tree creation             :        784.124        784.124        784.124          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        368.836        368.805        368.829          0.001
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.442             NA
   Tree creation             :          0.001          0.001          0.001          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/27/2020 14:21:14 --

result
version         = io500-sc20
config-hash     = B78D712
result-dir      = /work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00
; START 2020-10-27 13:49:23


[ior-easy-write]
t_start         = 2020-10-27 13:49:23
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/ior-easy/ior_file_easy -O stoneWallingStatusFile=/work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/ior-easy.stonewall -t 2m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a IME
throughput-stonewall = 2874881.66
score           = 934.695288
t_delta         = 447.6786
t_end           = 2020-10-27 13:56:51

[mdtest-easy-write]
t_start         = 2020-10-27 13:56:51
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/mdtest-easy -x /work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/mdtest-easy.stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 91.563437
score           = 71.511134
t_delta         = 719.4440
t_end           = 2020-10-27 14:08:50

[timestamp]
t_start         = 2020-10-27 14:08:50
t_delta         = 0.0018
t_end           = 2020-10-27 14:08:50

[ior-hard-write]
t_start         = 2020-10-27 14:08:50
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/ior-hard/file -O stoneWallingStatusFile=/work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a IME
throughput-stonewall = 2067155.66
score           = 580.051686
t_delta         = 374.4269
t_end           = 2020-10-27 14:15:05

[mdtest-hard-write]
t_start         = 2020-10-27 14:15:05
exe             = ./mdtest -n 4000 -t -w 3901 -e 3901 -P -N 1 -F -d /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/mdtest-hard -x /work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/mdtest-hard.stonewall -C -Y -W 300 -a IME
rate-stonewall  = 11.634557
score           = 11.605160
t_delta         = 369.1457
t_end           = 2020-10-27 14:21:14

[find]
t_start         = 2020-10-27 14:21:14
exe             = ./pfind /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00 -newer /work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/timestampfile -size 3901c -name *01* -C -q 10000
nproc           = 1024
found           = 163920
total-files     = 35000321
score           = 1405.117443
t_delta         = 24.9634
t_end           = 2020-10-27 14:21:39

[ior-easy-read]
t_start         = 2020-10-27 14:21:39
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/ior-easy/ior_file_easy -O stoneWallingStatusFile=/work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/ior-easy.stonewall -t 2m -b 9920000m -F -r -R -a IME
score           = 1237.427287
t_delta         = 337.9416
t_end           = 2020-10-27 14:27:17

[mdtest-easy-stat]
t_start         = 2020-10-27 14:27:17
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/mdtest-easy -x /work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/mdtest-easy.stonewall -T -a POSIX
score           = 196.148744
t_delta         = 156.8758
t_end           = 2020-10-27 14:29:53

[ior-hard-read]
t_start         = 2020-10-27 14:29:53
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/ior-hard/file -O stoneWallingStatusFile=/work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a IME
score           = 352.196454
t_delta         = 616.3482
t_end           = 2020-10-27 14:40:10

[mdtest-hard-stat]
t_start         = 2020-10-27 14:40:10
exe             = ./mdtest -n 4000 -t -w 3901 -e 3901 -P -N 1 -F -d /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/mdtest-hard -x /work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/mdtest-hard.stonewall -T -a IME
score           = 127.339133
t_delta         = 33.9271
t_end           = 2020-10-27 14:40:44

[mdtest-easy-delete]
t_start         = 2020-10-27 14:40:44
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/mdtest-easy -x /work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/mdtest-easy.stonewall -r -a POSIX
score           = 57.047900
t_delta         = 540.6085
t_end           = 2020-10-27 14:49:44

[mdtest-hard-read]
t_start         = 2020-10-27 14:49:44
exe             = ./mdtest -n 4000 -t -w 3901 -e 3901 -P -N 1 -F -d /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/mdtest-hard -x /work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/mdtest-hard.stonewall -E -X -a IME
score           = 129.293212
t_delta         = 33.5525
t_end           = 2020-10-27 14:50:18

[mdtest-hard-delete]
t_start         = 2020-10-27 14:50:18
exe             = ./mdtest -n 4000 -t -w 3901 -e 3901 -P -N 1 -F -d /cache/xg17i000/x10007/io500-sc20/datafiles/2020.10.27-13.49.00/mdtest-hard -x /work/xg17i000/x10007/io500-sc20/results/2020.10.27-13.49.00/mdtest-hard.stonewall -r -a IME
score           = 24.359128
t_delta         = 176.1611
t_end           = 2020-10-27 14:53:14

[SCORE]
MD              = 92.224275
BW              = 697.204776
SCORE           = 253.572879 
hash            = 7368EB0C
; END 2020-10-27 14:53:25
result_summary
IO500 version io500-sc20
[RESULT]       ior-easy-write      934.695288 GiB/s : time 447.679 seconds
[RESULT]    mdtest-easy-write       71.511134 kIOPS : time 719.444 seconds
[RESULT]       ior-hard-write      580.051686 GiB/s : time 374.427 seconds
[RESULT]    mdtest-hard-write       11.605160 kIOPS : time 369.146 seconds
[RESULT]                 find     1405.117443 kIOPS : time 24.963 seconds
[RESULT]        ior-easy-read     1237.427287 GiB/s : time 337.942 seconds
[RESULT]     mdtest-easy-stat      196.148744 kIOPS : time 156.876 seconds
[RESULT]        ior-hard-read      352.196454 GiB/s : time 616.348 seconds
[RESULT]     mdtest-hard-stat      127.339133 kIOPS : time 33.927 seconds
[RESULT]   mdtest-easy-delete       57.047900 kIOPS : time 540.608 seconds
[RESULT]     mdtest-hard-read      129.293212 kIOPS : time 33.552 seconds
[RESULT]   mdtest-hard-delete       24.359128 kIOPS : time 176.161 seconds
[SCORE] Bandwidth 697.204776 GB/s : IOPS 92.224275 kiops : TOTAL 253.572879