SharedScratch

Institution Janelia Research Campus, HHMI
Client Procs Per Node
Client Operating System Scientific-Linux
Client Operating System Version 7.7
Client Kernel Version 3.10.0-1062.9.1.el7.x86_64

DATA SERVER

Storage Type Flash
Volatile Memory 192GB
Storage Interface NVMe
Network 100GbE
Software Version 3.9.0
OS Version 7.7

INFORMATION

Client Nodes 10
Client Total Procs 320
Metadata Nodes 0
Metadata Storage Devices 0
Data Nodes 20
Data Storage Devices 80

METADATA

Easy Write 195.52 kIOP/s
Easy Stat 1,570.35 kIOP/s
Easy Delete 294.30 kIOP/s
Hard Write 16.16 kIOP/s
Hard Read 92.88 kIOP/s
Hard Stat 124.76 kIOP/s
Hard Delete 19.69 kIOP/s

Submitted Files

io500
#!/bin/bash
bsub -q test -R "span[ptile=32]" -n 320 -J '100Gb-weka-10n32p' -app sharedscratch mpirun -n 320 ./io500 ./config-hhmi.ini
ior-easy-read
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Sat Oct 17 17:20:57 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.17-16.47.32/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.17-16.47.32/ior-easy.stonewall -t 32m -b 9920000m -F -r -R -a POSIX
Machine             : Linux f12u10.int.janelia.org
TestID              : 0
StartTime           : Sat Oct 17 17:20:57 2020
Path                : ./datadir/2020.10.17-16.47.32/ior-easy/ior_file_easy.00000000
FS                  : 150.0 TiB   Used FS: 46.6%   Inodes: 858.3 Mi   Used Inodes: 16.5%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datadir/2020.10.17-16.47.32/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 320
clients per node    : 32
repetitions         : 1
xfersize            : 32 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 3027.34 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
WARNING: Expected aggregate file size       = 3328599654400000
WARNING: Stat() of aggregate file size      = 31729070899200
WARNING: Using actual aggregate bytes moved = 31729070899200
read      49170      1536.59    0.001985    10158080000 32768      0.004219   615.39     0.002162   615.39     0   
ior-easy-write
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Sat Oct 17 16:47:32 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.17-16.47.32/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.17-16.47.32/ior-easy.stonewall -t 32m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX --posix.odirect
Machine             : Linux f12u10.int.janelia.org
TestID              : 0
StartTime           : Sat Oct 17 16:47:32 2020
Path                : ./datadir/2020.10.17-16.47.32/ior-easy/ior_file_easy.00000000
FS                  : 150.0 TiB   Used FS: 25.0%   Inodes: 858.3 Mi   Used Inodes: 4.3%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datadir/2020.10.17-16.47.32/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 320
clients per node    : 32
repetitions         : 1
xfersize            : 32 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 3027.34 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 1574 max: 2955 -- min data: 49.2 GiB mean data: 66.8 GiB time: 300.1s
WARNING: Expected aggregate file size       = 3328599654400000
WARNING: Stat() of aggregate file size      = 31729070899200
WARNING: Using actual aggregate bytes moved = 31729070899200
WARNING: Maybe caused by deadlineForStonewalling
write     59845      1870.29    0.001138    10158080000 32768      0.030260   505.59     0.001279   505.62     0   
ior-hard-read
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Sat Oct 17 17:31:56 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.17-16.47.32/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.17-16.47.32/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a POSIX
Machine             : Linux f12u10.int.janelia.org
TestID              : 0
StartTime           : Sat Oct 17 17:31:56 2020
Path                : ./datadir/2020.10.17-16.47.32/ior-hard/file
FS                  : 150.0 TiB   Used FS: 46.6%   Inodes: 858.3 Mi   Used Inodes: 16.5%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datadir/2020.10.17-16.47.32/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 320
clients per node    : 32
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 136.81 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 230676 max: 230676 -- min data: 10.1 GiB mean data: 10.1 GiB time: 328.1s
WARNING: Expected aggregate file size       = 150425600000000
WARNING: Stat() of aggregate file size      = 3469957570560
WARNING: Using actual aggregate bytes moved = 3469957570560
read      10062      224454     328.81      45.91      45.91      0.003303   328.87     0.001727   328.87     0   
ior-hard-write
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Sat Oct 17 17:03:51 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.17-16.47.32/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.17-16.47.32/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a POSIX
Machine             : Linux f12u10.int.janelia.org
TestID              : 0
StartTime           : Sat Oct 17 17:03:51 2020
Path                : ./datadir/2020.10.17-16.47.32/ior-hard/file
FS                  : 150.0 TiB   Used FS: 44.5%   Inodes: 858.3 Mi   Used Inodes: 15.5%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datadir/2020.10.17-16.47.32/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 320
clients per node    : 32
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 136.81 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 29 max: 230676 -- min data: 0.0 GiB mean data: 0.3 GiB time: 300.0s
WARNING: Expected aggregate file size       = 150425600000000
WARNING: Stat() of aggregate file size      = 3469957570560
WARNING: Using actual aggregate bytes moved = 3469957570560
WARNING: Maybe caused by deadlineForStonewalling
write     6946       154931     411.54      45.91      45.91      0.002365   476.45     0.001080   476.45     0   
mdtest-easy-delete
-- started at 10/17/2020 17:38:11 --

mdtest-3.4.0+dev was launched with 320 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datadir/2020.10.17-16.47.32/mdtest-easy' '-x' './results/2020.10.17-16.47.32/mdtest-easy.stonewall' '-r' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.17-16.47.32
FS: 150.0 TiB   Used FS: 46.6%   Inodes: 858.3 Mi   Used Inodes: 16.5%

Nodemap: 11111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 32 for each phase.
320 tasks, 320000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :     294300.231     294300.040     294300.150          0.057
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.657          0.657          0.657          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        234.294        234.294        234.294          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          1.523          1.523          1.523          0.000
-- finished at 10/17/2020 17:42:11 --

mdtest-easy-stat
-- started at 10/17/2020 17:31:12 --

mdtest-3.4.0+dev was launched with 320 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datadir/2020.10.17-16.47.32/mdtest-easy' '-x' './results/2020.10.17-16.47.32/mdtest-easy.stonewall' '-T' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.17-16.47.32
FS: 150.0 TiB   Used FS: 46.6%   Inodes: 858.3 Mi   Used Inodes: 16.5%

Nodemap: 11111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 32 for each phase.
320 tasks, 320000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :    1570352.091    1570348.620    1570349.542          1.034
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :         43.909         43.909         43.909          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/17/2020 17:31:56 --

mdtest-easy-write
-- started at 10/17/2020 16:55:58 --

mdtest-3.4.0+dev was launched with 320 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datadir/2020.10.17-16.47.32/mdtest-easy' '-x' './results/2020.10.17-16.47.32/mdtest-easy.stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.17-16.47.32
FS: 150.0 TiB   Used FS: 44.0%   Inodes: 858.3 Mi   Used Inodes: 7.7%

Nodemap: 11111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 32 for each phase.
320 tasks, 320000000 files
Continue stonewall hit min: 205653 max: 215477 avg: 211368.5 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :     195515.885     195515.703     195515.764          0.058
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA     225454.674             NA
   Tree creation             :         39.354         39.354         39.354          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        352.671        352.670        352.670          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.007             NA
   Tree creation             :          0.025          0.025          0.025          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/17/2020 17:03:51 --

mdtest-hard-delete
-- started at 10/17/2020 17:43:13 --

mdtest-3.4.0+dev was launched with 320 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.17-16.47.32/mdtest-hard' '-x' './results/2020.10.17-16.47.32/mdtest-hard.stonewall' '-r' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.17-16.47.32
FS: 150.0 TiB   Used FS: 46.4%   Inodes: 858.3 Mi   Used Inodes: 8.7%

Nodemap: 11111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 32 for each phase.
320 tasks, 320000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :      19688.771      19688.749      19688.757          0.005
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.542          0.542          0.542          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        293.967        293.967        293.967          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          1.845          1.845          1.845          0.000
-- finished at 10/17/2020 17:48:09 --

mdtest-hard-read
-- started at 10/17/2020 17:42:11 --

mdtest-3.4.0+dev was launched with 320 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.17-16.47.32/mdtest-hard' '-x' './results/2020.10.17-16.47.32/mdtest-hard.stonewall' '-E' '-X' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.17-16.47.32
FS: 150.0 TiB   Used FS: 46.4%   Inodes: 858.3 Mi   Used Inodes: 8.7%

Nodemap: 11111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 32 for each phase.
320 tasks, 320000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :      92880.315      92880.205      92880.257          0.030
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :         62.315         62.315         62.315          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/17/2020 17:43:13 --

mdtest-hard-stat
-- started at 10/17/2020 17:37:25 --

mdtest-3.4.0+dev was launched with 320 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.17-16.47.32/mdtest-hard' '-x' './results/2020.10.17-16.47.32/mdtest-hard.stonewall' '-T' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.17-16.47.32
FS: 150.0 TiB   Used FS: 46.6%   Inodes: 858.3 Mi   Used Inodes: 16.5%

Nodemap: 11111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 32 for each phase.
320 tasks, 320000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :     124759.678     124759.557     124759.618          0.028
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :         46.392         46.392         46.392          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/17/2020 17:38:11 --

mdtest-hard-write
-- started at 10/17/2020 17:11:48 --

mdtest-3.4.0+dev was launched with 320 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.17-16.47.32/mdtest-hard' '-x' './results/2020.10.17-16.47.32/mdtest-hard.stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.17-16.47.32
FS: 150.0 TiB   Used FS: 46.6%   Inodes: 858.3 Mi   Used Inodes: 15.9%

Nodemap: 11111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 32 for each phase.
320 tasks, 320000000 files
Continue stonewall hit min: 17322 max: 18087 avg: 17741.5 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :      16164.516      16164.500      16164.509          0.005
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA      18922.994             NA
   Tree creation             :        782.519        782.519        782.519          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        358.059        358.058        358.059          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.020             NA
   Tree creation             :          0.001          0.001          0.001          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/17/2020 17:20:03 --

result
version         = io500-sc20
config-hash     = F5F16DD
result-dir      = ./results/2020.10.17-16.47.32
; START 2020-10-17 16:47:32


[ior-easy-write]
t_start         = 2020-10-17 16:47:32
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.17-16.47.32/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.17-16.47.32/ior-easy.stonewall -t 32m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX --posix.odirect
throughput-stonewall = 13519.46
score           = 58.442848
t_delta         = 505.6889
t_end           = 2020-10-17 16:55:58

[mdtest-easy-write]
t_start         = 2020-10-17 16:55:58
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d ./datadir/2020.10.17-16.47.32/mdtest-easy -x ./results/2020.10.17-16.47.32/mdtest-easy.stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 225.454674
score           = 195.515703
t_delta         = 473.5320
t_end           = 2020-10-17 17:03:51

[timestamp]
t_start         = 2020-10-17 17:03:51
t_delta         = 0.0014
t_end           = 2020-10-17 17:03:51

[ior-hard-write]
t_start         = 2020-10-17 17:03:51
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.17-16.47.32/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.17-16.47.32/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a POSIX
throughput-stonewall = 68.37
score           = 6.782759
t_delta         = 476.4584
t_end           = 2020-10-17 17:11:48

[mdtest-hard-write]
t_start         = 2020-10-17 17:11:48
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.17-16.47.32/mdtest-hard -x ./results/2020.10.17-16.47.32/mdtest-hard.stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 18.922994
score           = 16.164503
t_delta         = 495.4364
t_end           = 2020-10-17 17:20:03

[find]
t_start         = 2020-10-17 17:20:03
exe             = ./pfind ./datadir/2020.10.17-16.47.32 -newer ./results/2020.10.17-16.47.32/timestampfile -size 3901c -name *01* -C -q 10000
found           = 199764
total-files     = 74740801
score           = 1403.535811
t_delta         = 53.2727
t_end           = 2020-10-17 17:20:57

[ior-easy-read]
t_start         = 2020-10-17 17:20:57
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.17-16.47.32/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.17-16.47.32/ior-easy.stonewall -t 32m -b 9920000m -F -r -R -a POSIX
score           = 48.018003
t_delta         = 615.4253
t_end           = 2020-10-17 17:31:12

[mdtest-easy-stat]
t_start         = 2020-10-17 17:31:12
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d ./datadir/2020.10.17-16.47.32/mdtest-easy -x ./results/2020.10.17-16.47.32/mdtest-easy.stonewall -T -a POSIX
score           = 1570.349371
t_delta         = 43.9210
t_end           = 2020-10-17 17:31:56

[ior-hard-read]
t_start         = 2020-10-17 17:31:56
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.17-16.47.32/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.17-16.47.32/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a POSIX
score           = 9.826414
t_delta         = 328.8817
t_end           = 2020-10-17 17:37:25

[mdtest-hard-stat]
t_start         = 2020-10-17 17:37:25
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.17-16.47.32/mdtest-hard -x ./results/2020.10.17-16.47.32/mdtest-hard.stonewall -T -a POSIX
score           = 124.759602
t_delta         = 46.3968
t_end           = 2020-10-17 17:38:11

[mdtest-easy-delete]
t_start         = 2020-10-17 17:38:11
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d ./datadir/2020.10.17-16.47.32/mdtest-easy -x ./results/2020.10.17-16.47.32/mdtest-easy.stonewall -r -a POSIX
score           = 294.300080
t_delta         = 239.3170
t_end           = 2020-10-17 17:42:11

[mdtest-hard-read]
t_start         = 2020-10-17 17:42:11
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.17-16.47.32/mdtest-hard -x ./results/2020.10.17-16.47.32/mdtest-hard.stonewall -E -X -a POSIX
score           = 92.880208
t_delta         = 62.3207
t_end           = 2020-10-17 17:43:13

[mdtest-hard-delete]
t_start         = 2020-10-17 17:43:13
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.17-16.47.32/mdtest-hard -x ./results/2020.10.17-16.47.32/mdtest-hard.stonewall -r -a POSIX
score           = 19.688750
t_delta         = 295.8183
t_end           = 2020-10-17 17:48:09

[SCORE]
MD              = 161.713465
BW              = 20.796212
SCORE           = 57.991615 
hash            = 70DF4E12
; END 2020-10-17 17:48:13
result_summary
IO500 version io500-sc20
[RESULT]       ior-easy-write       58.442848 GiB/s : time 505.689 seconds
[RESULT]    mdtest-easy-write      195.515703 kIOPS : time 473.532 seconds
[RESULT]       ior-hard-write        6.782759 GiB/s : time 476.458 seconds
[RESULT]    mdtest-hard-write       16.164503 kIOPS : time 495.436 seconds
[RESULT]                 find     1403.535811 kIOPS : time 53.273 seconds
[RESULT]        ior-easy-read       48.018003 GiB/s : time 615.425 seconds
[RESULT]     mdtest-easy-stat     1570.349371 kIOPS : time 43.921 seconds
[RESULT]        ior-hard-read        9.826414 GiB/s : time 328.882 seconds
[RESULT]     mdtest-hard-stat      124.759602 kIOPS : time 46.397 seconds
[RESULT]   mdtest-easy-delete      294.300080 kIOPS : time 239.317 seconds
[RESULT]     mdtest-hard-read       92.880208 kIOPS : time 62.321 seconds
[RESULT]   mdtest-hard-delete       19.688750 kIOPS : time 295.818 seconds
[SCORE] Bandwidth 20.796212 GB/s : IOPS 161.713465 kiops : TOTAL 57.991615