Vast

Institution Janelia Research Campus, HHMI
Client Procs Per Node
Client Operating System Scientific-Linux
Client Operating System Version 7.8
Client Kernel Version 3.10.0-957.10.1.el7.x86_64

DATA SERVER

Storage Type Flash
Volatile Memory 64GB
Storage Interface NVMe
Network 100GbE
Software Version 3.0.6
OS Version 7.8

INFORMATION

Client Nodes 10
Client Total Procs 260
Metadata Nodes 40
Metadata Storage Devices 132
Data Nodes 11
Data Storage Devices 484

METADATA

Easy Write 28.15 kIOP/s
Easy Stat 143.84 kIOP/s
Easy Delete 30.87 kIOP/s
Hard Write 1.19 kIOP/s
Hard Read 8.44 kIOP/s
Hard Stat 9.70 kIOP/s
Hard Delete 1.53 kIOP/s

Submitted Files

ior-easy-read
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Wed Oct 28 14:11:29 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.28-13.40.49/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.28-13.40.49/ior-easy.stonewall -t 1m -b 9920000m -F -r -R -a POSIX
Machine             : Linux h09u01.int.janelia.org
TestID              : 0
StartTime           : Wed Oct 28 14:11:29 2020
Path                : ./datadir/2020.10.28-13.40.49/ior-easy/ior_file_easy.00000000
FS                  : 6700.2 TiB   Used FS: 24.7%   Inodes: 16758.4 Mi   Used Inodes: 8.1%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datadir/2020.10.28-13.40.49/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 260
clients per node    : 26
repetitions         : 1
xfersize            : 1 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 2459.72 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
WARNING: Expected aggregate file size       = 2704487219200000
WARNING: Stat() of aggregate file size      = 7843012935680
WARNING: Using actual aggregate bytes moved = 7843012935680
read      21677      21692      0.000035    10158080000 1024.00    0.115188   344.82     0.113768   345.04     0   
ior-easy-write
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Wed Oct 28 13:40:49 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.28-13.40.49/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.28-13.40.49/ior-easy.stonewall -t 1m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX
Machine             : Linux h09u01.int.janelia.org
TestID              : 0
StartTime           : Wed Oct 28 13:40:49 2020
Path                : ./datadir/2020.10.28-13.40.49/ior-easy/ior_file_easy.00000000
FS                  : 6677.3 TiB   Used FS: 24.6%   Inodes: 16758.4 Mi   Used Inodes: 8.0%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datadir/2020.10.28-13.40.49/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 260
clients per node    : 26
repetitions         : 1
xfersize            : 1 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 2459.72 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 23141 max: 28768 -- min data: 22.6 GiB mean data: 26.2 GiB time: 368.0s
WARNING: Expected aggregate file size       = 2704487219200000
WARNING: Stat() of aggregate file size      = 7843012935680
WARNING: Using actual aggregate bytes moved = 7843012935680
WARNING: Maybe caused by deadlineForStonewalling
write     15444      15455      0.000049    10158080000 1024.00    0.229820   483.95     0.114565   484.30     0   
ior-hard-read
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Wed Oct 28 14:18:55 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.28-13.40.49/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.28-13.40.49/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a POSIX --posix.odirect
Machine             : Linux h09u01.int.janelia.org
TestID              : 0
StartTime           : Wed Oct 28 14:18:55 2020
Path                : ./datadir/2020.10.28-13.40.49/ior-hard/file
FS                  : 6700.2 TiB   Used FS: 24.7%   Inodes: 16758.4 Mi   Used Inodes: 8.1%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datadir/2020.10.28-13.40.49/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 260
clients per node    : 26
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 111.16 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 57717 max: 57717 -- min data: 2.5 GiB mean data: 2.5 GiB time: 112.2s
WARNING: Expected aggregate file size       = 122220800000000
WARNING: Stat() of aggregate file size      = 705421791360
WARNING: Using actual aggregate bytes moved = 705421791360
read      5969       133412     112.48      45.91      45.91      0.218126   112.48     0.000314   112.70     0   
ior-hard-write
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Wed Oct 28 13:57:41 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.28-13.40.49/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.28-13.40.49/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a POSIX --posix.odirect
Machine             : Linux h09u01.int.janelia.org
TestID              : 0
StartTime           : Wed Oct 28 13:57:41 2020
Path                : ./datadir/2020.10.28-13.40.49/ior-hard/file
FS                  : 6697.7 TiB   Used FS: 24.7%   Inodes: 16758.4 Mi   Used Inodes: 8.1%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datadir/2020.10.28-13.40.49/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 260
clients per node    : 26
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 111.16 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 36791 max: 57717 -- min data: 1.6 GiB mean data: 2.2 GiB time: 300.0s
WARNING: Expected aggregate file size       = 122220800000000
WARNING: Stat() of aggregate file size      = 705421791360
WARNING: Using actual aggregate bytes moved = 705421791360
WARNING: Maybe caused by deadlineForStonewalling
write     1683.27    37565      337.83      45.91      45.91      0.182048   399.48     0.000270   399.67     0   
mdtest-easy-delete
-- started at 10/28/2020 14:21:27 --

mdtest-3.4.0+dev was launched with 260 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datadir/2020.10.28-13.40.49/mdtest-easy' '-x' './results/2020.10.28-13.40.49/mdtest-easy.stonewall' '-r' '-a' 'POSIX'
Path: /io500/nrs/testing/io500-sc20/datadir/2020.10.28-13.40.49
FS: 6700.2 TiB   Used FS: 24.7%   Inodes: 16758.4 Mi   Used Inodes: 8.1%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 26 for each phase.
260 tasks, 260000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :      30866.369      30866.362      30866.366          0.002
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          4.078          4.078          4.078          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        468.662        468.662        468.662          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.245          0.245          0.245          0.000
-- finished at 10/28/2020 14:29:16 --

mdtest-easy-stat
-- started at 10/28/2020 14:17:14 --

mdtest-3.4.0+dev was launched with 260 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datadir/2020.10.28-13.40.49/mdtest-easy' '-x' './results/2020.10.28-13.40.49/mdtest-easy.stonewall' '-T' '-a' 'POSIX'
Path: /io500/nrs/testing/io500-sc20/datadir/2020.10.28-13.40.49
FS: 6700.2 TiB   Used FS: 24.7%   Inodes: 16758.4 Mi   Used Inodes: 8.1%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 26 for each phase.
260 tasks, 260000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :     143844.248     143844.165     143844.214          0.014
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :        100.566        100.566        100.566          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/28/2020 14:18:55 --

mdtest-easy-write
-- started at 10/28/2020 13:48:53 --

mdtest-3.4.0+dev was launched with 260 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datadir/2020.10.28-13.40.49/mdtest-easy' '-x' './results/2020.10.28-13.40.49/mdtest-easy.stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /io500/nrs/testing/io500-sc20/datadir/2020.10.28-13.40.49
FS: 6695.9 TiB   Used FS: 24.7%   Inodes: 16758.4 Mi   Used Inodes: 8.0%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 26 for each phase.
260 tasks, 260000000 files
Continue stonewall hit min: 32658 max: 55638 avg: 40924.7 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :      28154.295      28154.286      28154.290          0.002
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA      35465.574             NA
   Tree creation             :          3.455          3.455          3.455          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        513.807        513.807        513.807          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.021             NA
   Tree creation             :          0.289          0.289          0.289          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/28/2020 13:57:41 --

mdtest-hard-delete
-- started at 10/28/2020 14:30:01 --

mdtest-3.4.0+dev was launched with 260 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.28-13.40.49/mdtest-hard' '-x' './results/2020.10.28-13.40.49/mdtest-hard.stonewall' '-r' '-a' 'POSIX'
Path: /io500/nrs/testing/io500-sc20/datadir/2020.10.28-13.40.49
FS: 6700.2 TiB   Used FS: 24.7%   Inodes: 16758.4 Mi   Used Inodes: 8.0%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 26 for each phase.
260 tasks, 260000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :       1528.030       1528.029       1528.030          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          6.977          6.977          6.977          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        246.213        246.212        246.212          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.143          0.143          0.143          0.000
-- finished at 10/28/2020 14:34:07 --

mdtest-hard-read
-- started at 10/28/2020 14:29:16 --

mdtest-3.4.0+dev was launched with 260 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.28-13.40.49/mdtest-hard' '-x' './results/2020.10.28-13.40.49/mdtest-hard.stonewall' '-E' '-X' '-a' 'POSIX'
Path: /io500/nrs/testing/io500-sc20/datadir/2020.10.28-13.40.49
FS: 6700.2 TiB   Used FS: 24.7%   Inodes: 16758.4 Mi   Used Inodes: 8.0%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 26 for each phase.
260 tasks, 260000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :       8444.135       8444.129       8444.132          0.001
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :         44.554         44.554         44.554          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/28/2020 14:30:01 --

mdtest-hard-stat
-- started at 10/28/2020 14:20:48 --

mdtest-3.4.0+dev was launched with 260 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.28-13.40.49/mdtest-hard' '-x' './results/2020.10.28-13.40.49/mdtest-hard.stonewall' '-T' '-a' 'POSIX'
Path: /io500/nrs/testing/io500-sc20/datadir/2020.10.28-13.40.49
FS: 6700.2 TiB   Used FS: 24.7%   Inodes: 16758.4 Mi   Used Inodes: 8.1%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 26 for each phase.
260 tasks, 260000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :       9700.974       9700.966       9700.970          0.002
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :         38.782         38.782         38.782          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/28/2020 14:21:27 --

mdtest-hard-write
-- started at 10/28/2020 14:04:21 --

mdtest-3.4.0+dev was launched with 260 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.28-13.40.49/mdtest-hard' '-x' './results/2020.10.28-13.40.49/mdtest-hard.stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /io500/nrs/testing/io500-sc20/datadir/2020.10.28-13.40.49
FS: 6698.6 TiB   Used FS: 24.7%   Inodes: 16758.4 Mi   Used Inodes: 8.1%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 26 for each phase.
260 tasks, 260000000 files
Continue stonewall hit min: 1372 max: 1447 avg: 1403.2 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :       1185.730       1185.730       1185.730          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA       1215.186             NA
   Tree creation             :         99.048         99.048         99.048          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        317.290        317.290        317.290          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.236             NA
   Tree creation             :          0.010          0.010          0.010          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/28/2020 14:09:53 --

result
version         = io500-sc20_v3-31
config-hash     = BC4E5478
result-dir      = ./results/2020.10.28-13.40.49
; START 2020-10-28 13:40:49


[ior-easy-write]
t_start         = 2020-10-28 13:40:49
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.28-13.40.49/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.28-13.40.49/ior-easy.stonewall -t 1m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX
result-file     = ./results/2020.10.28-13.40.49/ior-easy-write.txt
accessed-pairs  = 28768
throughput-stonewall = 3651.60
score           = 15.082483
t_delta         = 484.4809
t_end           = 2020-10-28 13:48:53

[mdtest-easy-write]
t_start         = 2020-10-28 13:48:53
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d ./datadir/2020.10.28-13.40.49/mdtest-easy -x ./results/2020.10.28-13.40.49/mdtest-easy.stonewall -C -Y -W 300 -a POSIX
result-file     = ./results/2020.10.28-13.40.49/mdtest-easy-write.txt
time            = 513.807246
rate-stonewall  = 35.465574
score           = 28.154293
t_delta         = 527.7861
t_end           = 2020-10-28 13:57:41

[timestamp]
t_start         = 2020-10-28 13:57:41
timestamp-file  = ./results/2020.10.28-13.40.49/timestampfile
t_delta         = 0.1090
t_end           = 2020-10-28 13:57:41

[ior-hard-write]
t_start         = 2020-10-28 13:57:41
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.28-13.40.49/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.28-13.40.49/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a POSIX --posix.odirect
result-file     = ./results/2020.10.28-13.40.49/ior-hard-write.txt
accessed-pairs  = 57717
throughput-stonewall = 364.14
score           = 1.643815
t_delta         = 400.0077
t_end           = 2020-10-28 14:04:21

[mdtest-hard-write]
t_start         = 2020-10-28 14:04:21
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.28-13.40.49/mdtest-hard -x ./results/2020.10.28-13.40.49/mdtest-hard.stonewall -C -Y -W 300 -a POSIX
result-file     = ./results/2020.10.28-13.40.49/mdtest-hard-write.txt
time            = 317.289704
rate-stonewall  = 1.215186
score           = 1.185730
t_delta         = 331.6665
t_end           = 2020-10-28 14:09:53

[find]
t_start         = 2020-10-28 14:09:53
exe             = ./pfind ./datadir/2020.10.28-13.40.49 -newer ./results/2020.10.28-13.40.49/timestampfile -size 3901c -name *01* -C -N -H 1 -q 20000
found           = 9086
total-files     = 14842361
score           = 155.085665
t_delta         = 95.9802
t_end           = 2020-10-28 14:11:29

[ior-easy-read]
t_start         = 2020-10-28 14:11:29
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.28-13.40.49/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.28-13.40.49/ior-easy.stonewall -t 1m -b 9920000m -F -r -R -a POSIX
result-file     = ./results/2020.10.28-13.40.49/ior-easy-read.txt
score           = 21.169392
t_delta         = 345.1701
t_end           = 2020-10-28 14:17:14

[mdtest-easy-stat]
t_start         = 2020-10-28 14:17:14
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d ./datadir/2020.10.28-13.40.49/mdtest-easy -x ./results/2020.10.28-13.40.49/mdtest-easy.stonewall -T -a POSIX
result-file     = ./results/2020.10.28-13.40.49/mdtest-easy-stat.txt
time            = 100.566274
score           = 143.844248
t_delta         = 100.7944
t_end           = 2020-10-28 14:18:55

[ior-hard-read]
t_start         = 2020-10-28 14:18:55
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.28-13.40.49/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.28-13.40.49/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a POSIX --posix.odirect
result-file     = ./results/2020.10.28-13.40.49/ior-hard-read.txt
score           = 5.829394
t_delta         = 112.8312
t_end           = 2020-10-28 14:20:48

[mdtest-hard-stat]
t_start         = 2020-10-28 14:20:48
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.28-13.40.49/mdtest-hard -x ./results/2020.10.28-13.40.49/mdtest-hard.stonewall -T -a POSIX
result-file     = ./results/2020.10.28-13.40.49/mdtest-hard-stat.txt
time            = 38.781674
score           = 9.700974
t_delta         = 39.0102
t_end           = 2020-10-28 14:21:27

[mdtest-easy-delete]
t_start         = 2020-10-28 14:21:27
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d ./datadir/2020.10.28-13.40.49/mdtest-easy -x ./results/2020.10.28-13.40.49/mdtest-easy.stonewall -r -a POSIX
result-file     = ./results/2020.10.28-13.40.49/mdtest-easy-delete.txt
time            = 468.661532
score           = 30.866369
t_delta         = 469.6422
t_end           = 2020-10-28 14:29:16

[mdtest-hard-read]
t_start         = 2020-10-28 14:29:16
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.28-13.40.49/mdtest-hard -x ./results/2020.10.28-13.40.49/mdtest-hard.stonewall -E -X -a POSIX
result-file     = ./results/2020.10.28-13.40.49/mdtest-hard-read.txt
time            = 44.554011
score           = 8.444133
t_delta         = 44.5671
t_end           = 2020-10-28 14:30:01

[mdtest-hard-delete]
t_start         = 2020-10-28 14:30:01
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.28-13.40.49/mdtest-hard -x ./results/2020.10.28-13.40.49/mdtest-hard.stonewall -r -a POSIX
result-file     = ./results/2020.10.28-13.40.49/mdtest-hard-delete.txt
time            = 246.212460
score           = 1.528030
t_delta         = 246.4855
t_end           = 2020-10-28 14:34:07

[SCORE]
MD              = 15.218534
BW              = 7.437284
SCORE           = 10.638823 
hash            = 2DB9987D
; END 2020-10-28 14:34:08
result_summary
IO500 version io500-sc20_v3-31
[RESULT]       ior-easy-write       15.082483 GiB/s : time 484.481 seconds
[RESULT]    mdtest-easy-write       28.154293 kIOPS : time 527.786 seconds
[RESULT]       ior-hard-write        1.643815 GiB/s : time 400.008 seconds
[RESULT]    mdtest-hard-write        1.185730 kIOPS : time 331.666 seconds
[RESULT]                 find      155.085665 kIOPS : time 95.980 seconds
[RESULT]        ior-easy-read       21.169392 GiB/s : time 345.170 seconds
[RESULT]     mdtest-easy-stat      143.844248 kIOPS : time 100.794 seconds
[RESULT]        ior-hard-read        5.829394 GiB/s : time 112.831 seconds
[RESULT]     mdtest-hard-stat        9.700974 kIOPS : time 39.010 seconds
[RESULT]   mdtest-easy-delete       30.866369 kIOPS : time 469.642 seconds
[RESULT]     mdtest-hard-read        8.444133 kIOPS : time 44.567 seconds
[RESULT]   mdtest-hard-delete        1.528030 kIOPS : time 246.485 seconds
[SCORE] Bandwidth 7.437284 GiB/s : IOPS 15.218534 kiops : TOTAL 10.638823