SharedScratch

Institution Janelia Research Campus, HHMI
Client Procs Per Node
Client Operating System Scientific-Linux
Client Operating System Version 7.7
Client Kernel Version 3.10.0-1062.9.1.el7.x86_64

DATA SERVER

Storage Type Flash
Volatile Memory 192GB
Storage Interface NVMe
Network 100GbE
Software Version 3.9.0
OS Version 7.7

INFORMATION

Client Nodes 10
Client Total Procs 320

METADATA

Easy Write 195.52 kIOP/s
Easy Stat 1,570.35 kIOP/s
Easy Delete 294.30 kIOP/s
Hard Write 16.16 kIOP/s
Hard Read 92.88 kIOP/s
Hard Stat 124.76 kIOP/s
Hard Delete 19.69 kIOP/s

Submitted Files

io500
#!/bin/bash
bsub -q test -R "span[ptile=32]" -n 320 -J '100Gb-weka-10n32p' -app sharedscratch mpirun -n 320 ./io500 ./config-hhmi.ini
ior-easy-read
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Sat Oct 17 17:20:57 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.17-16.47.32/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.17-16.47.32/ior-easy.stonewall -t 32m -b 9920000m -F -r -R -a POSIX
Machine             : Linux f12u10.int.janelia.org
TestID              : 0
StartTime           : Sat Oct 17 17:20:57 2020
Path                : ./datadir/2020.10.17-16.47.32/ior-easy/ior_file_easy.00000000
FS                  : 150.0 TiB   Used FS: 46.6%   Inodes: 858.3 Mi   Used Inodes: 16.5%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datadir/2020.10.17-16.47.32/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 320
clients per node    : 32
repetitions         : 1
xfersize            : 32 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 3027.34 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
WARNING: Expected aggregate file size       = 3328599654400000
WARNING: Stat() of aggregate file size      = 31729070899200
WARNING: Using actual aggregate bytes moved = 31729070899200
read      49170      1536.59    0.001985    10158080000 32768      0.004219   615.39     0.002162   615.39     0   
ior-easy-write
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Sat Oct 17 16:47:32 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.17-16.47.32/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.17-16.47.32/ior-easy.stonewall -t 32m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX --posix.odirect
Machine             : Linux f12u10.int.janelia.org
TestID              : 0
StartTime           : Sat Oct 17 16:47:32 2020
Path                : ./datadir/2020.10.17-16.47.32/ior-easy/ior_file_easy.00000000
FS                  : 150.0 TiB   Used FS: 25.0%   Inodes: 858.3 Mi   Used Inodes: 4.3%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datadir/2020.10.17-16.47.32/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 320
clients per node    : 32
repetitions         : 1
xfersize            : 32 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 3027.34 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 1574 max: 2955 -- min data: 49.2 GiB mean data: 66.8 GiB time: 300.1s
WARNING: Expected aggregate file size       = 3328599654400000
WARNING: Stat() of aggregate file size      = 31729070899200
WARNING: Using actual aggregate bytes moved = 31729070899200
WARNING: Maybe caused by deadlineForStonewalling
write     59845      1870.29    0.001138    10158080000 32768      0.030260   505.59     0.001279   505.62     0   
ior-hard-read
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Sat Oct 17 17:31:56 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.17-16.47.32/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.17-16.47.32/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a POSIX
Machine             : Linux f12u10.int.janelia.org
TestID              : 0
StartTime           : Sat Oct 17 17:31:56 2020
Path                : ./datadir/2020.10.17-16.47.32/ior-hard/file
FS                  : 150.0 TiB   Used FS: 46.6%   Inodes: 858.3 Mi   Used Inodes: 16.5%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datadir/2020.10.17-16.47.32/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 320
clients per node    : 32
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 136.81 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 230676 max: 230676 -- min data: 10.1 GiB mean data: 10.1 GiB time: 328.1s
WARNING: Expected aggregate file size       = 150425600000000
WARNING: Stat() of aggregate file size      = 3469957570560
WARNING: Using actual aggregate bytes moved = 3469957570560
read      10062      224454     328.81      45.91      45.91      0.003303   328.87     0.001727   328.87     0   
ior-hard-write
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Sat Oct 17 17:03:51 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.17-16.47.32/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.17-16.47.32/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a POSIX
Machine             : Linux f12u10.int.janelia.org
TestID              : 0
StartTime           : Sat Oct 17 17:03:51 2020
Path                : ./datadir/2020.10.17-16.47.32/ior-hard/file
FS                  : 150.0 TiB   Used FS: 44.5%   Inodes: 858.3 Mi   Used Inodes: 15.5%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datadir/2020.10.17-16.47.32/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 320
clients per node    : 32
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 136.81 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 29 max: 230676 -- min data: 0.0 GiB mean data: 0.3 GiB time: 300.0s
WARNING: Expected aggregate file size       = 150425600000000
WARNING: Stat() of aggregate file size      = 3469957570560
WARNING: Using actual aggregate bytes moved = 3469957570560
WARNING: Maybe caused by deadlineForStonewalling
write     6946       154931     411.54      45.91      45.91      0.002365   476.45     0.001080   476.45     0   
mdtest-easy-delete
-- started at 10/17/2020 17:38:11 --

mdtest-3.4.0+dev was launched with 320 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datadir/2020.10.17-16.47.32/mdtest-easy' '-x' './results/2020.10.17-16.47.32/mdtest-easy.stonewall' '-r' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.17-16.47.32
FS: 150.0 TiB   Used FS: 46.6%   Inodes: 858.3 Mi   Used Inodes: 16.5%

Nodemap: 11111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 32 for each phase.
320 tasks, 320000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :     294300.231     294300.040     294300.150          0.057
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.657          0.657          0.657          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        234.294        234.294        234.294          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          1.523          1.523          1.523          0.000
-- finished at 10/17/2020 17:42:11 --

mdtest-easy-stat
-- started at 10/17/2020 17:31:12 --

mdtest-3.4.0+dev was launched with 320 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datadir/2020.10.17-16.47.32/mdtest-easy' '-x' './results/2020.10.17-16.47.32/mdtest-easy.stonewall' '-T' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.17-16.47.32
FS: 150.0 TiB   Used FS: 46.6%   Inodes: 858.3 Mi   Used Inodes: 16.5%

Nodemap: 11111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 32 for each phase.
320 tasks, 320000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :    1570352.091    1570348.620    1570349.542          1.034
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :         43.909         43.909         43.909          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/17/2020 17:31:56 --

mdtest-easy-write
-- started at 10/17/2020 16:55:58 --

mdtest-3.4.0+dev was launched with 320 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datadir/2020.10.17-16.47.32/mdtest-easy' '-x' './results/2020.10.17-16.47.32/mdtest-easy.stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.17-16.47.32
FS: 150.0 TiB   Used FS: 44.0%   Inodes: 858.3 Mi   Used Inodes: 7.7%

Nodemap: 11111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 32 for each phase.
320 tasks, 320000000 files
Continue stonewall hit min: 205653 max: 215477 avg: 211368.5 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :     195515.885     195515.703     195515.764          0.058
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA     225454.674             NA
   Tree creation             :         39.354         39.354         39.354          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        352.671        352.670        352.670          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.007             NA
   Tree creation             :          0.025          0.025          0.025          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/17/2020 17:03:51 --

mdtest-hard-delete
-- started at 10/17/2020 17:43:13 --

mdtest-3.4.0+dev was launched with 320 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.17-16.47.32/mdtest-hard' '-x' './results/2020.10.17-16.47.32/mdtest-hard.stonewall' '-r' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.17-16.47.32
FS: 150.0 TiB   Used FS: 46.4%   Inodes: 858.3 Mi   Used Inodes: 8.7%

Nodemap: 11111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 32 for each phase.
320 tasks, 320000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :      19688.771      19688.749      19688.757          0.005
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.542          0.542          0.542          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        293.967        293.967        293.967          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          1.845          1.845          1.845          0.000
-- finished at 10/17/2020 17:48:09 --

mdtest-hard-read
-- started at 10/17/2020 17:42:11 --

mdtest-3.4.0+dev was launched with 320 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.17-16.47.32/mdtest-hard' '-x' './results/2020.10.17-16.47.32/mdtest-hard.stonewall' '-E' '-X' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.17-16.47.32
FS: 150.0 TiB   Used FS: 46.4%   Inodes: 858.3 Mi   Used Inodes: 8.7%

Nodemap: 11111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 32 for each phase.
320 tasks, 320000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :      92880.315      92880.205      92880.257          0.030
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :         62.315         62.315         62.315          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/17/2020 17:43:13 --

mdtest-hard-stat
-- started at 10/17/2020 17:37:25 --

mdtest-3.4.0+dev was launched with 320 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.17-16.47.32/mdtest-hard' '-x' './results/2020.10.17-16.47.32/mdtest-hard.stonewall' '-T' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.17-16.47.32
FS: 150.0 TiB   Used FS: 46.6%   Inodes: 858.3 Mi   Used Inodes: 16.5%

Nodemap: 11111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 32 for each phase.
320 tasks, 320000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :     124759.678     124759.557     124759.618          0.028
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :         46.392         46.392         46.392          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/17/2020 17:38:11 --

mdtest-hard-write
-- started at 10/17/2020 17:11:48 --

mdtest-3.4.0+dev was launched with 320 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.17-16.47.32/mdtest-hard' '-x' './results/2020.10.17-16.47.32/mdtest-hard.stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.17-16.47.32
FS: 150.0 TiB   Used FS: 46.6%   Inodes: 858.3 Mi   Used Inodes: 15.9%

Nodemap: 11111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 32 for each phase.
320 tasks, 320000000 files
Continue stonewall hit min: 17322 max: 18087 avg: 17741.5 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :      16164.516      16164.500      16164.509          0.005
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA      18922.994             NA
   Tree creation             :        782.519        782.519        782.519          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        358.059        358.058        358.059          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.020             NA
   Tree creation             :          0.001          0.001          0.001          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/17/2020 17:20:03 --

result
version         = io500-sc20
config-hash     = F5F16DD
result-dir      = ./results/2020.10.17-16.47.32
; START 2020-10-17 16:47:32


[ior-easy-write]
t_start         = 2020-10-17 16:47:32
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.17-16.47.32/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.17-16.47.32/ior-easy.stonewall -t 32m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX --posix.odirect
throughput-stonewall = 13519.46
score           = 58.442848
t_delta         = 505.6889
t_end           = 2020-10-17 16:55:58

[mdtest-easy-write]
t_start         = 2020-10-17 16:55:58
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d ./datadir/2020.10.17-16.47.32/mdtest-easy -x ./results/2020.10.17-16.47.32/mdtest-easy.stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 225.454674
score           = 195.515703
t_delta         = 473.5320
t_end           = 2020-10-17 17:03:51

[timestamp]
t_start         = 2020-10-17 17:03:51
t_delta         = 0.0014
t_end           = 2020-10-17 17:03:51

[ior-hard-write]
t_start         = 2020-10-17 17:03:51
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.17-16.47.32/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.17-16.47.32/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a POSIX
throughput-stonewall = 68.37
score           = 6.782759
t_delta         = 476.4584
t_end           = 2020-10-17 17:11:48

[mdtest-hard-write]
t_start         = 2020-10-17 17:11:48
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.17-16.47.32/mdtest-hard -x ./results/2020.10.17-16.47.32/mdtest-hard.stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 18.922994
score           = 16.164503
t_delta         = 495.4364
t_end           = 2020-10-17 17:20:03

[find]
t_start         = 2020-10-17 17:20:03
exe             = ./pfind ./datadir/2020.10.17-16.47.32 -newer ./results/2020.10.17-16.47.32/timestampfile -size 3901c -name *01* -C -q 10000
found           = 199764
total-files     = 74740801
score           = 1403.535811
t_delta         = 53.2727
t_end           = 2020-10-17 17:20:57

[ior-easy-read]
t_start         = 2020-10-17 17:20:57
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.17-16.47.32/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.17-16.47.32/ior-easy.stonewall -t 32m -b 9920000m -F -r -R -a POSIX
score           = 48.018003
t_delta         = 615.4253
t_end           = 2020-10-17 17:31:12

[mdtest-easy-stat]
t_start         = 2020-10-17 17:31:12
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d ./datadir/2020.10.17-16.47.32/mdtest-easy -x ./results/2020.10.17-16.47.32/mdtest-easy.stonewall -T -a POSIX
score           = 1570.349371
t_delta         = 43.9210
t_end           = 2020-10-17 17:31:56

[ior-hard-read]
t_start         = 2020-10-17 17:31:56
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.17-16.47.32/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.17-16.47.32/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a POSIX
score           = 9.826414
t_delta         = 328.8817
t_end           = 2020-10-17 17:37:25

[mdtest-hard-stat]
t_start         = 2020-10-17 17:37:25
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.17-16.47.32/mdtest-hard -x ./results/2020.10.17-16.47.32/mdtest-hard.stonewall -T -a POSIX
score           = 124.759602
t_delta         = 46.3968
t_end           = 2020-10-17 17:38:11

[mdtest-easy-delete]
t_start         = 2020-10-17 17:38:11
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d ./datadir/2020.10.17-16.47.32/mdtest-easy -x ./results/2020.10.17-16.47.32/mdtest-easy.stonewall -r -a POSIX
score           = 294.300080
t_delta         = 239.3170
t_end           = 2020-10-17 17:42:11

[mdtest-hard-read]
t_start         = 2020-10-17 17:42:11
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.17-16.47.32/mdtest-hard -x ./results/2020.10.17-16.47.32/mdtest-hard.stonewall -E -X -a POSIX
score           = 92.880208
t_delta         = 62.3207
t_end           = 2020-10-17 17:43:13

[mdtest-hard-delete]
t_start         = 2020-10-17 17:43:13
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.17-16.47.32/mdtest-hard -x ./results/2020.10.17-16.47.32/mdtest-hard.stonewall -r -a POSIX
score           = 19.688750
t_delta         = 295.8183
t_end           = 2020-10-17 17:48:09

[SCORE]
MD              = 161.713465
BW              = 20.796212
SCORE           = 57.991615 
hash            = 70DF4E12
; END 2020-10-17 17:48:13
result_summary
IO500 version io500-sc20
[RESULT]       ior-easy-write       58.442848 GiB/s : time 505.689 seconds
[RESULT]    mdtest-easy-write      195.515703 kIOPS : time 473.532 seconds
[RESULT]       ior-hard-write        6.782759 GiB/s : time 476.458 seconds
[RESULT]    mdtest-hard-write       16.164503 kIOPS : time 495.436 seconds
[RESULT]                 find     1403.535811 kIOPS : time 53.273 seconds
[RESULT]        ior-easy-read       48.018003 GiB/s : time 615.425 seconds
[RESULT]     mdtest-easy-stat     1570.349371 kIOPS : time 43.921 seconds
[RESULT]        ior-hard-read        9.826414 GiB/s : time 328.882 seconds
[RESULT]     mdtest-hard-stat      124.759602 kIOPS : time 46.397 seconds
[RESULT]   mdtest-easy-delete      294.300080 kIOPS : time 239.317 seconds
[RESULT]     mdtest-hard-read       92.880208 kIOPS : time 62.321 seconds
[RESULT]   mdtest-hard-delete       19.688750 kIOPS : time 295.818 seconds
[SCORE] Bandwidth 20.796212 GB/s : IOPS 161.713465 kiops : TOTAL 57.991615