Vast

Institution Janelia Research Campus, HHMI
Client Procs Per Node
Client Operating System Scientific-Linux
Client Operating System Version 7.6
Client Kernel Version 3.10.0-957.10.1.el7.x86_64

DATA SERVER

Storage Type Flash
Volatile Memory 64GB
Storage Interface NVMe
Network 100GbE
Software Version 2.0.6
OS Version 7.6

INFORMATION

Client Nodes 10
Client Total Procs 620
Metadata Nodes 24
Metadata Storage Devices 72
Data Nodes 6
Data Storage Devices 1,320

METADATA

Easy Write 22.62 kIOP/s
Easy Stat 110.87 kIOP/s
Easy Delete 21.93 kIOP/s
Hard Write 0.64 kIOP/s
Hard Read 4.89 kIOP/s
Hard Stat 9.30 kIOP/s
Hard Delete 0.76 kIOP/s

Submitted Files

ior_easy_read
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Mon Nov 18 16:10:12 2019
Command line        : /io500/io500-sc19/bin/ior -r -R -t 1024k -b 9920000m -F -i 1 -C -Q 1 -g -G 27 -k -e -o /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/ior_easy/ior_file_easy -O stoneWallingStatusFile=/io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/ior_easy/stonewall
Machine             : Linux h10u07.int.janelia.org
TestID              : 0
StartTime           : Mon Nov 18 16:10:12 2019
Path                : /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/ior_easy
FS                  : 3092.5 TiB   Used FS: 51.4%   Inodes: 8270.0 Mi   Used Inodes: 12.8%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/ior_easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
tasks               : 620
clients per node    : 62
repetitions         : 1
xfersize            : 1 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 5865.48 TiB

Results: 

access    bw(MiB/s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s) iter
------    ---------  ---------- ---------  --------   --------   --------   -------- ----
WARNING: Expected aggregate file size       = 6449161830400000.
WARNING: Stat() of aggregate file size      = 5416775843840.
WARNING: Using actual aggregate bytes moved = 5416775843840.
read      20131      10158080000 1024.00    0.174365   256.44     0.003857   256.61     0   
Max Read:  20130.81 MiB/sec (21108.69 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
read        20130.81   20130.81   20130.81       0.00   20130.81   20130.81   20130.81       0.00  256.61356         NA            NA     0    620  62    1   1     1        1         0    0      1 10401873920000  1048576 5165840.0 POSIX      0
Finished            : Mon Nov 18 16:14:29 2019
ior_easy_write
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Mon Nov 18 15:37:19 2019
Command line        : /io500/io500-sc19/bin/ior -w -t 1024k -b 9920000m -F -i 1 -C -Q 1 -g -G 27 -k -e -o /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/ior_easy/ior_file_easy -O stoneWallingStatusFile=/io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/ior_easy/stonewall -O stoneWallingWearOut=1 -D 300
Machine             : Linux h10u07.int.janelia.org
TestID              : 0
StartTime           : Mon Nov 18 15:37:19 2019
Path                : /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/ior_easy
FS                  : 3085.1 TiB   Used FS: 51.3%   Inodes: 8270.0 Mi   Used Inodes: 12.6%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/ior_easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
tasks               : 620
clients per node    : 62
repetitions         : 1
xfersize            : 1 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 5865.48 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s) iter
------    ---------  ---------- ---------  --------   --------   --------   -------- ----
stonewalling pairs accessed min: 7035 max: 8332 -- min data: 6.9 GiB mean data: 7.4 GiB time: 300.2s
WARNING: Expected aggregate file size       = 6449161830400000.
WARNING: Stat() of aggregate file size      = 5416775843840.
WARNING: Using actual aggregate bytes moved = 5416775843840.
WARNING: maybe caused by deadlineForStonewalling
write     8607       10158080000 1024.00    0.908264   599.29     0.013917   600.21     0   
Max Write: 8606.68 MiB/sec (9024.76 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
write        8606.68    8606.68    8606.68       0.00    8606.68    8606.68    8606.68       0.00  600.21296     300.16      15577.55     0    620  62    1   1     1        1         0    0      1 10401873920000  1048576 5165840.0 POSIX      0
Finished            : Mon Nov 18 15:47:19 2019
ior_hard_read
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Mon Nov 18 16:15:50 2019
Command line        : /io500/io500-sc19/bin/ior -r -R -s 1900000 -a POSIX --posix.odirect -i 1 -C -Q 1 -g -G 27 -k -e -t 47008 -b 47008 -o /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/ior_hard/IOR_file -O stoneWallingStatusFile=/io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/ior_hard/stonewall
Machine             : Linux h10u07.int.janelia.org
TestID              : 0
StartTime           : Mon Nov 18 16:15:50 2019
Path                : /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/ior_hard
FS                  : 3091.4 TiB   Used FS: 51.4%   Inodes: 8270.0 Mi   Used Inodes: 12.8%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/ior_hard/IOR_file
access              : single-shared-file
type                : independent
segments            : 1900000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
tasks               : 620
clients per node    : 62
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 50.36 TiB

Results: 

access    bw(MiB/s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s) iter
------    ---------  ---------- ---------  --------   --------   --------   -------- ----
WARNING: Expected aggregate file size       = 55375424000000.
WARNING: Stat() of aggregate file size      = 654158627200.
WARNING: Using actual aggregate bytes moved = 654158627200.
read      4473       45.91      45.91      0.029075   139.43     0.000637   139.46     0   
Max Read:  4473.27 MiB/sec (4690.56 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
read         4473.27    4473.27    4473.27       0.00   99782.13   99782.13   99782.13       0.00  139.46285         NA            NA     0    620  62    1   0     1        1         0    0 1900000    47008    47008  623854.3 POSIX      0
Finished            : Mon Nov 18 16:18:10 2019
ior_hard_write
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Mon Nov 18 15:53:51 2019
Command line        : /io500/io500-sc19/bin/ior -w -s 1900000 -a POSIX --posix.odirect -i 1 -C -Q 1 -g -G 27 -k -e -t 47008 -b 47008 -o /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/ior_hard/IOR_file -O stoneWallingStatusFile=/io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/ior_hard/stonewall -O stoneWallingWearOut=1 -D 300
Machine             : Linux h10u07.int.janelia.org
TestID              : 0
StartTime           : Mon Nov 18 15:53:51 2019
Path                : /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/ior_hard
FS                  : 3087.9 TiB   Used FS: 51.4%   Inodes: 8270.0 Mi   Used Inodes: 12.7%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/ior_hard/IOR_file
access              : single-shared-file
type                : independent
segments            : 1900000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
tasks               : 620
clients per node    : 62
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 50.36 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s) iter
------    ---------  ---------- ---------  --------   --------   --------   -------- ----
stonewalling pairs accessed min: 17876 max: 22445 -- min data: 0.8 GiB mean data: 0.9 GiB time: 300.0s
WARNING: Expected aggregate file size       = 55375424000000.
WARNING: Stat() of aggregate file size      = 654158627200.
WARNING: Using actual aggregate bytes moved = 654158627200.
WARNING: maybe caused by deadlineForStonewalling
write     1795.23    45.91      45.91      0.045306   347.46     0.000422   347.51     0   
Max Write: 1795.23 MiB/sec (1882.44 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
write        1795.23    1795.23    1795.23       0.00   40045.10   40045.10   40045.10       0.00  347.50573     300.05       1829.69     0    620  62    1   0     1        1         0    0 1900000    47008    47008  623854.3 POSIX      0
Finished            : Mon Nov 18 15:59:38 2019
mdtest_easy_delete
-- started at 11/18/2019 16:18:35 --

mdtest-3.3.0+dev was launched with 620 total task(s) on 10 node(s)
Command line used: /io500/io500-sc19/bin/mdtest '-r' '-F' '-P' '-d' '/io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/mdt_easy' '-n' '900000' '-u' '-L' '-x' '/io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/mdt_easy-stonewall' '-N' '1'
Path: /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18
FS: 3091.8 TiB   Used FS: 51.4%   Inodes: 8270.0 Mi   Used Inodes: 12.8%

Nodemap: 11111111111111111111111111111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2128 Shifting ranks by 62 for each phase.
620 tasks, 558000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :      21930.126      21930.102      21930.116          0.005
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.839          0.839          0.839          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        399.620        399.619        399.619          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          1.192          1.192          1.192          0.000
-- finished at 11/18/2019 16:25:17 --

mdtest_easy_stat
-- started at 11/18/2019 16:14:30 --

mdtest-3.3.0+dev was launched with 620 total task(s) on 10 node(s)
Command line used: /io500/io500-sc19/bin/mdtest '-T' '-F' '-P' '-d' '/io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/mdt_easy' '-n' '900000' '-u' '-L' '-x' '/io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/mdt_easy-stonewall' '-N' '1'
Path: /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18
FS: 3092.1 TiB   Used FS: 51.4%   Inodes: 8270.0 Mi   Used Inodes: 12.8%

Nodemap: 11111111111111111111111111111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2128 Shifting ranks by 62 for each phase.
620 tasks, 558000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :     110870.075     110869.459     110869.917          0.114
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :         79.045         79.045         79.045          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 11/18/2019 16:15:49 --

mdtest_easy_write
-- started at 11/18/2019 15:47:21 --

mdtest-3.3.0+dev was launched with 620 total task(s) on 10 node(s)
Command line used: /io500/io500-sc19/bin/mdtest '-C' '-F' '-P' '-d' '/io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/mdt_easy' '-n' '900000' '-u' '-L' '-x' '/io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/mdt_easy-stonewall' '-N' '1' '-W' '300'
Path: /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18
FS: 3087.0 TiB   Used FS: 51.4%   Inodes: 8270.0 Mi   Used Inodes: 12.6%

Nodemap: 11111111111111111111111111111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2128 Shifting ranks by 62 for each phase.
620 tasks, 558000000 files
Continue stonewall hit min: 9888 max: 14135 avg: 12241.7 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :      22620.950      22620.924      22620.941          0.005
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA      25410.148             NA
   Tree creation             :          0.694          0.694          0.694          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        387.416        387.415        387.415          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        298.695             NA
   Tree creation             :          1.442          1.442          1.442          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 11/18/2019 15:53:50 --

mdtest_hard_delete
-- started at 11/18/2019 16:26:03 --

mdtest-3.3.0+dev was launched with 620 total task(s) on 10 node(s)
Command line used: /io500/io500-sc19/bin/mdtest '-r' '-t' '-F' '-P' '-w' '3901' '-e' '3901' '-d' '/io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/mdt_hard' '-n' '950000' '-x' '/io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/mdt_hard-stonewall' '-a' 'POSIX' '-N' '1'
Path: /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18
FS: 3090.8 TiB   Used FS: 51.4%   Inodes: 8270.0 Mi   Used Inodes: 12.7%

Nodemap: 11111111111111111111111111111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2128 Shifting ranks by 62 for each phase.
620 tasks, 589000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        758.778        758.777        758.777          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          6.987          6.987          6.987          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        290.889        290.889        290.889          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.143          0.143          0.143          0.000
-- finished at 11/18/2019 16:30:54 --

mdtest_hard_read
-- started at 11/18/2019 16:25:18 --

mdtest-3.3.0+dev was launched with 620 total task(s) on 10 node(s)
Command line used: /io500/io500-sc19/bin/mdtest '-X' '-E' '-t' '-F' '-P' '-w' '3901' '-e' '3901' '-d' '/io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/mdt_hard' '-n' '950000' '-x' '/io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/mdt_hard-stonewall' '-a' 'POSIX' '-N' '1'
Path: /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18
FS: 3092.6 TiB   Used FS: 51.4%   Inodes: 8270.0 Mi   Used Inodes: 12.7%

Nodemap: 11111111111111111111111111111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2128 Shifting ranks by 62 for each phase.
620 tasks, 589000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :       4893.510       4893.501       4893.505          0.002
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :         45.105         45.105         45.105          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 11/18/2019 16:26:03 --

mdtest_hard_stat
-- started at 11/18/2019 16:18:11 --

mdtest-3.3.0+dev was launched with 620 total task(s) on 10 node(s)
Command line used: /io500/io500-sc19/bin/mdtest '-T' '-t' '-F' '-P' '-w' '3901' '-e' '3901' '-d' '/io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/mdt_hard' '-n' '950000' '-x' '/io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/mdt_hard-stonewall' '-a' 'POSIX' '-N' '1'
Path: /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18
FS: 3091.8 TiB   Used FS: 51.4%   Inodes: 8270.0 Mi   Used Inodes: 12.8%

Nodemap: 11111111111111111111111111111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2128 Shifting ranks by 62 for each phase.
620 tasks, 589000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :       9303.827       9303.778       9303.809          0.013
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :         23.724         23.724         23.724          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 11/18/2019 16:18:34 --

mdtest_hard_write
-- started at 11/18/2019 15:59:39 --

mdtest-3.3.0+dev was launched with 620 total task(s) on 10 node(s)
Command line used: /io500/io500-sc19/bin/mdtest '-C' '-t' '-F' '-P' '-w' '3901' '-e' '3901' '-d' '/io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/mdt_hard' '-n' '950000' '-x' '/io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18/mdt_hard-stonewall' '-a' 'POSIX' '-N' '1' '-W' '300'
Path: /io500/io500-sc19/datafiles/io500.2019.11.18-15.37.18
FS: 3090.3 TiB   Used FS: 51.4%   Inodes: 8270.0 Mi   Used Inodes: 12.8%

Nodemap: 11111111111111111111111111111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2128 Shifting ranks by 62 for each phase.
620 tasks, 589000000 files
Continue stonewall hit min: 324 max: 356 avg: 339.6 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        638.792        638.792        638.792          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        694.774             NA
   Tree creation             :        129.099        129.099        129.099          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        345.527        345.527        345.527          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        303.025             NA
   Tree creation             :          0.008          0.008          0.008          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 11/18/2019 16:05:25 --

result_summary
[RESULT] BW   phase 1            ior_easy_write                8.405 GB/s : time 600.21 seconds
[RESULT] IOPS phase 1         mdtest_easy_write               22.621 kiops : time 387.42 seconds
[RESULT] BW   phase 2            ior_hard_write                1.753 GB/s : time 347.51 seconds
[RESULT] IOPS phase 2         mdtest_hard_write                0.639 kiops : time 345.53 seconds
[RESULT] IOPS phase 3                      find               31.140 kiops : time 286.70 seconds
[RESULT] BW   phase 3             ior_easy_read               19.659 GB/s : time 256.61 seconds
[RESULT] IOPS phase 4          mdtest_easy_stat              110.870 kiops : time  79.04 seconds
[RESULT] BW   phase 4             ior_hard_read                4.368 GB/s : time 139.46 seconds
[RESULT] IOPS phase 5          mdtest_hard_stat                9.304 kiops : time  23.72 seconds
[RESULT] IOPS phase 6        mdtest_easy_delete               21.930 kiops : time 399.62 seconds
[RESULT] IOPS phase 7          mdtest_hard_read                4.894 kiops : time  45.10 seconds
[RESULT] IOPS phase 8        mdtest_hard_delete                0.759 kiops : time 291.64 seconds
[SCORE] Bandwidth 5.96429 GB/s : IOPS 8.85483 kiops : TOTAL 7.26724