hyperwall4

Institution NASA
Client Procs Per Node
Client Operating System SUSE
Client Operating System Version SUSE
Client Kernel Version 4.12.14

DATA SERVER

Storage Type SSD
Volatile Memory 25.3 TB DRAM
Storage Interface NVMe
Network Infiniband
Software Version 7.1.5
OS Version 4.12.14

INFORMATION

Client Nodes 128
Client Total Procs 1,024

METADATA

Easy Write 35.18 kIOP/s
Easy Stat 59.42 kIOP/s
Easy Delete 57.56 kIOP/s
Hard Write 12.53 kIOP/s
Hard Read 27.98 kIOP/s
Hard Stat 57.98 kIOP/s
Hard Delete 17.06 kIOP/s

Submitted Files

ior-easy-read
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Fri Oct 30 18:24:29 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o ./datafiles/2020.10.30-18.01.17/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.30-18.01.17/ior-easy.stonewall -t 2m -b 9920000m -F -r -R -a POSIX
Machine             : Linux r803i0n0
TestID              : 0
StartTime           : Fri Oct 30 18:24:29 2020
Path                : ./datafiles/2020.10.30-18.01.17/ior-easy/ior_file_easy.00000000
FS                  : 1787.9 TiB   Used FS: 11.3%   Inodes: 0.0 Mi   Used Inodes: -nan%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datafiles/2020.10.30-18.01.17/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 128
tasks               : 1664
clients per node    : 13
repetitions         : 1
xfersize            : 2 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 15742.19 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
WARNING: Expected aggregate file size       = 17308718202880000
WARNING: Stat() of aggregate file size      = 214170960134144
WARNING: Using actual aggregate bytes moved = 214170960134144
read      705995     354023     0.000058    10158080000 2048.00    0.056220   288.47     0.784960   289.31     0   
ior-easy-write
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Fri Oct 30 18:01:18 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o ./datafiles/2020.10.30-18.01.17/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.30-18.01.17/ior-easy.stonewall -t 2m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX
Machine             : Linux r803i0n0
TestID              : 0
StartTime           : Fri Oct 30 18:01:18 2020
Path                : ./datafiles/2020.10.30-18.01.17/ior-easy/ior_file_easy.00000000
FS                  : 1787.9 TiB   Used FS: 0.0%   Inodes: 0.0 Mi   Used Inodes: -nan%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datafiles/2020.10.30-18.01.17/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 128
tasks               : 1664
clients per node    : 13
repetitions         : 1
xfersize            : 2 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 15742.19 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 35267 max: 61373 -- min data: 68.9 GiB mean data: 95.4 GiB time: 300.3s
WARNING: Expected aggregate file size       = 17308718202880000
WARNING: Stat() of aggregate file size      = 214170960134144
WARNING: Using actual aggregate bytes moved = 214170960134144
WARNING: Maybe caused by deadlineForStonewalling
write     505656     253355     0.000080    10158080000 2048.00    0.101691   403.09     0.738881   403.93     0   
ior-hard-read
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Fri Oct 30 18:32:41 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o ./datafiles/2020.10.30-18.01.17/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.30-18.01.17/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a POSIX
Machine             : Linux r803i0n0
TestID              : 0
StartTime           : Fri Oct 30 18:32:41 2020
Path                : ./datafiles/2020.10.30-18.01.17/ior-hard/file
FS                  : 1787.9 TiB   Used FS: 11.3%   Inodes: 0.0 Mi   Used Inodes: -nan%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datafiles/2020.10.30-18.01.17/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 128
tasks               : 1664
clients per node    : 13
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 711.42 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 103697 max: 103697 -- min data: 4.5 GiB mean data: 4.5 GiB time: 281.0s
WARNING: Expected aggregate file size       = 782213120000000
WARNING: Stat() of aggregate file size      = 8111315390464
WARNING: Using actual aggregate bytes moved = 8111315390464
read      27394      613578     281.22      45.91      45.91      1.06       281.22     0.103678   282.38     0   
ior-hard-write
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Fri Oct 30 18:13:45 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o ./datafiles/2020.10.30-18.01.17/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.30-18.01.17/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a POSIX
Machine             : Linux r803i0n0
TestID              : 0
StartTime           : Fri Oct 30 18:13:45 2020
Path                : ./datafiles/2020.10.30-18.01.17/ior-hard/file
FS                  : 1787.9 TiB   Used FS: 10.9%   Inodes: 0.0 Mi   Used Inodes: -nan%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datafiles/2020.10.30-18.01.17/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 128
tasks               : 1664
clients per node    : 13
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 711.42 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 101536 max: 103697 -- min data: 4.4 GiB mean data: 4.5 GiB time: 300.1s
WARNING: Expected aggregate file size       = 782213120000000
WARNING: Stat() of aggregate file size      = 8111315390464
WARNING: Using actual aggregate bytes moved = 8111315390464
WARNING: Maybe caused by deadlineForStonewalling
write     25101      562000     305.83      45.91      45.91      1.06       307.03     0.079162   308.17     0   
mdtest-easy-delete
-- started at 10/30/2020 18:38:30 --

mdtest-3.4.0+dev was launched with 1664 total task(s) on 128 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datafiles/2020.10.30-18.01.17/mdtest-easy' '-x' './results/2020.10.30-18.01.17/mdtest-easy.stonewall' '-r' '-a' 'POSIX'
Path: /nobackuphw/rciotti/io500/datafiles/2020.10.30-18.01.17
FS: 1787.9 TiB   Used FS: 11.3%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
1664 tasks, 1664000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :      57558.475      57558.344      57558.422          0.031
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.296          0.296          0.296          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        208.787        208.786        208.786          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          3.373          3.373          3.373          0.000
-- finished at 10/30/2020 18:42:02 --

mdtest-easy-stat
-- started at 10/30/2020 18:29:19 --

mdtest-3.4.0+dev was launched with 1664 total task(s) on 128 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datafiles/2020.10.30-18.01.17/mdtest-easy' '-x' './results/2020.10.30-18.01.17/mdtest-easy.stonewall' '-T' '-a' 'POSIX'
Path: /nobackuphw/rciotti/io500/datafiles/2020.10.30-18.01.17
FS: 1787.9 TiB   Used FS: 11.3%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
1664 tasks, 1664000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :      59417.162      59417.034      59417.118          0.029
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :        202.255        202.255        202.255          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/30/2020 18:32:41 --

mdtest-easy-write
-- started at 10/30/2020 18:08:02 --

mdtest-3.4.0+dev was launched with 1664 total task(s) on 128 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datafiles/2020.10.30-18.01.17/mdtest-easy' '-x' './results/2020.10.30-18.01.17/mdtest-easy.stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /nobackuphw/rciotti/io500/datafiles/2020.10.30-18.01.17
FS: 1787.9 TiB   Used FS: 10.9%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
1664 tasks, 1664000000 files
Continue stonewall hit min: 5429 max: 7222 avg: 6508.4 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :      35176.000      35175.913      35175.960          0.019
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA      35640.948             NA
   Tree creation             :         11.243         11.243         11.243          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        341.637        341.637        341.637          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        303.864             NA
   Tree creation             :          0.089          0.089          0.089          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/30/2020 18:13:45 --

mdtest-hard-delete
-- started at 10/30/2020 18:44:20 --

mdtest-3.4.0+dev was launched with 1664 total task(s) on 128 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datafiles/2020.10.30-18.01.17/mdtest-hard' '-x' './results/2020.10.30-18.01.17/mdtest-hard.stonewall' '-r' '-a' 'POSIX'
Path: /nobackuphw/rciotti/io500/datafiles/2020.10.30-18.01.17
FS: 1787.9 TiB   Used FS: 11.3%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
1664 tasks, 1664000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :      17062.492      17062.455      17062.478          0.009
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.228          0.228          0.228          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        225.281        225.280        225.280          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          4.395          4.395          4.395          0.000
-- finished at 10/30/2020 18:48:09 --

mdtest-hard-read
-- started at 10/30/2020 18:42:02 --

mdtest-3.4.0+dev was launched with 1664 total task(s) on 128 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datafiles/2020.10.30-18.01.17/mdtest-hard' '-x' './results/2020.10.30-18.01.17/mdtest-hard.stonewall' '-E' '-X' '-a' 'POSIX'
Path: /nobackuphw/rciotti/io500/datafiles/2020.10.30-18.01.17
FS: 1787.9 TiB   Used FS: 11.3%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
1664 tasks, 1664000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :      27983.571      27983.509      27983.545          0.015
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :        137.361        137.361        137.361          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/30/2020 18:44:20 --

mdtest-hard-stat
-- started at 10/30/2020 18:37:24 --

mdtest-3.4.0+dev was launched with 1664 total task(s) on 128 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datafiles/2020.10.30-18.01.17/mdtest-hard' '-x' './results/2020.10.30-18.01.17/mdtest-hard.stonewall' '-T' '-a' 'POSIX'
Path: /nobackuphw/rciotti/io500/datafiles/2020.10.30-18.01.17
FS: 1787.9 TiB   Used FS: 11.3%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
1664 tasks, 1664000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :      57983.413      57983.268      57983.352          0.033
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :         66.292         66.292         66.292          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/30/2020 18:38:30 --

mdtest-hard-write
-- started at 10/30/2020 18:18:53 --

mdtest-3.4.0+dev was launched with 1664 total task(s) on 128 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datafiles/2020.10.30-18.01.17/mdtest-hard' '-x' './results/2020.10.30-18.01.17/mdtest-hard.stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /nobackuphw/rciotti/io500/datafiles/2020.10.30-18.01.17
FS: 1787.9 TiB   Used FS: 11.3%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
1664 tasks, 1664000000 files
Continue stonewall hit min: 2224 max: 2310 avg: 2281.4 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :      12526.698      12526.668      12526.686          0.008
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA      12646.674             NA
   Tree creation             :       1543.975       1543.975       1543.975          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        306.853        306.852        306.852          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.175             NA
   Tree creation             :          0.001          0.001          0.001          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/30/2020 18:24:01 --

result
version         = io500-sc20_v3
config-hash     = B3C51B6B
result-dir      = ./results/2020.10.30-18.01.17
; START 2020-10-30 18:01:18


[ior-easy-write]
t_start         = 2020-10-30 18:01:18
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o ./datafiles/2020.10.30-18.01.17/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.30-18.01.17/ior-easy.stonewall -t 2m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX
throughput-stonewall = 653845.49
score           = 493.804294
t_delta         = 404.0469
t_end           = 2020-10-30 18:08:02

[mdtest-easy-write]
t_start         = 2020-10-30 18:08:02
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d ./datafiles/2020.10.30-18.01.17/mdtest-easy -x ./results/2020.10.30-18.01.17/mdtest-easy.stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 35.640948
score           = 35.175940
t_delta         = 342.6733
t_end           = 2020-10-30 18:13:45

[timestamp]
t_start         = 2020-10-30 18:13:45
t_delta         = 0.0540
t_end           = 2020-10-30 18:13:45

[ior-hard-write]
t_start         = 2020-10-30 18:13:45
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o ./datafiles/2020.10.30-18.01.17/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.30-18.01.17/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a POSIX
throughput-stonewall = 40629.56
score           = 24.512967
t_delta         = 308.2432
t_end           = 2020-10-30 18:18:53

[mdtest-hard-write]
t_start         = 2020-10-30 18:18:53
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datafiles/2020.10.30-18.01.17/mdtest-hard -x ./results/2020.10.30-18.01.17/mdtest-hard.stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 12.646674
score           = 12.526686
t_delta         = 307.3941
t_end           = 2020-10-30 18:24:01

[find]
t_start         = 2020-10-30 18:24:01
exe             = ./pfind ./datafiles/2020.10.30-18.01.17 -newer ./results/2020.10.30-18.01.17/timestampfile -size 3901c -name *01* -C -q 10000
found           = 130494
total-files     = 15862913
score           = 565.094949
t_delta         = 28.0899
t_end           = 2020-10-30 18:24:29

[ior-easy-read]
t_start         = 2020-10-30 18:24:29
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o ./datafiles/2020.10.30-18.01.17/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.30-18.01.17/ior-easy.stonewall -t 2m -b 9920000m -F -r -R -a POSIX
score           = 689.448531
t_delta         = 290.1972
t_end           = 2020-10-30 18:29:19

[mdtest-easy-stat]
t_start         = 2020-10-30 18:29:19
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d ./datafiles/2020.10.30-18.01.17/mdtest-easy -x ./results/2020.10.30-18.01.17/mdtest-easy.stonewall -T -a POSIX
score           = 59.417114
t_delta         = 202.2732
t_end           = 2020-10-30 18:32:41

[ior-hard-read]
t_start         = 2020-10-30 18:32:41
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o ./datafiles/2020.10.30-18.01.17/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.30-18.01.17/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a POSIX
score           = 26.751721
t_delta         = 282.4189
t_end           = 2020-10-30 18:37:24

[mdtest-hard-stat]
t_start         = 2020-10-30 18:37:24
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datafiles/2020.10.30-18.01.17/mdtest-hard -x ./results/2020.10.30-18.01.17/mdtest-hard.stonewall -T -a POSIX
score           = 57.983353
t_delta         = 66.3047
t_end           = 2020-10-30 18:38:30

[mdtest-easy-delete]
t_start         = 2020-10-30 18:38:30
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d ./datafiles/2020.10.30-18.01.17/mdtest-easy -x ./results/2020.10.30-18.01.17/mdtest-easy.stonewall -r -a POSIX
score           = 57.558416
t_delta         = 212.1754
t_end           = 2020-10-30 18:42:02

[mdtest-hard-read]
t_start         = 2020-10-30 18:42:02
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datafiles/2020.10.30-18.01.17/mdtest-hard -x ./results/2020.10.30-18.01.17/mdtest-hard.stonewall -E -X -a POSIX
score           = 27.983542
t_delta         = 137.3740
t_end           = 2020-10-30 18:44:20

[mdtest-hard-delete]
t_start         = 2020-10-30 18:44:20
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datafiles/2020.10.30-18.01.17/mdtest-hard -x ./results/2020.10.30-18.01.17/mdtest-hard.stonewall -r -a POSIX
score           = 17.062476
t_delta         = 229.6892
t_end           = 2020-10-30 18:48:09

[SCORE]
MD              = 46.941717
BW              = 122.236548
SCORE           = 75.749544 
hash            = A3D5B986
; END 2020-10-30 18:48:10
result_summary
IO500 version io500-sc20_v3
[RESULT]       ior-easy-write      493.804294 GiB/s : time 404.047 seconds
[RESULT]    mdtest-easy-write       35.175940 kIOPS : time 342.673 seconds
[RESULT]       ior-hard-write       24.512967 GiB/s : time 308.243 seconds
[RESULT]    mdtest-hard-write       12.526686 kIOPS : time 307.394 seconds
[RESULT]                 find      565.094949 kIOPS : time 28.090 seconds
[RESULT]        ior-easy-read      689.448531 GiB/s : time 290.197 seconds
[RESULT]     mdtest-easy-stat       59.417114 kIOPS : time 202.273 seconds
[RESULT]        ior-hard-read       26.751721 GiB/s : time 282.419 seconds
[RESULT]     mdtest-hard-stat       57.983353 kIOPS : time 66.305 seconds
[RESULT]   mdtest-easy-delete       57.558416 kIOPS : time 212.175 seconds
[RESULT]     mdtest-hard-read       27.983542 kIOPS : time 137.374 seconds
[RESULT]   mdtest-hard-delete       17.062476 kIOPS : time 229.689 seconds
[SCORE] Bandwidth 122.236548 GiB/s : IOPS 46.941717 kiops : TOTAL 75.749544