SharedScratch

Institution Janelia Research Campus, HHMI
Client Procs Per Node
Client Operating System Scientific-Linux
Client Operating System Version 7.7
Client Kernel Version 3.10.0-1062.9.1.el7.x86_64

DATA SERVER

Storage Type Flash
Volatile Memory 192GB
Storage Interface NVMe
Network 100GbE
Software Version 3.8.1
OS Version 7.7

INFORMATION

Client Nodes 40
Client Total Procs 1,040

METADATA

Easy Write 289.75 kIOP/s
Easy Stat 3,435.11 kIOP/s
Easy Delete 489.99 kIOP/s
Hard Write 50.16 kIOP/s
Hard Read 329.95 kIOP/s
Hard Stat 469.73 kIOP/s
Hard Delete 65.70 kIOP/s

Submitted Files

ior-easy-read
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Oct 13 21:14:12 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.13-20.35.18/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.13-20.35.18/ior-easy.stonewall -t 32m -b 9920000m -F -r -R -a POSIX
Machine             : Linux h09u01.int.janelia.org
TestID              : 0
StartTime           : Tue Oct 13 21:14:12 2020
Path                : ./datadir/2020.10.13-20.35.18/ior-easy/ior_file_easy.00000000
FS                  : 150.0 TiB   Used FS: 30.6%   Inodes: 858.3 Mi   Used Inodes: 19.4%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datadir/2020.10.13-20.35.18/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 40
tasks               : 1040
clients per node    : 26
repetitions         : 1
xfersize            : 32 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 9838.87 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
WARNING: Expected aggregate file size       = 10817948876800000
WARNING: Stat() of aggregate file size      = 36187783823360
WARNING: Using actual aggregate bytes moved = 36187783823360
read      77043      2407.62    0.001445    10158080000 32768      0.005263   447.94     0.002478   447.95     0   
ior-easy-write
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Oct 13 20:35:18 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.13-20.35.18/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.13-20.35.18/ior-easy.stonewall -t 32m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX
Machine             : Linux h09u01.int.janelia.org
TestID              : 0
StartTime           : Tue Oct 13 20:35:18 2020
Path                : ./datadir/2020.10.13-20.35.18/ior-easy/ior_file_easy.00000000
FS                  : 150.0 TiB   Used FS: 0.0%   Inodes: 858.3 Mi   Used Inodes: 0.0%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datadir/2020.10.13-20.35.18/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 40
tasks               : 1040
clients per node    : 26
repetitions         : 1
xfersize            : 32 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 9838.87 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 557 max: 1037 -- min data: 17.4 GiB mean data: 23.9 GiB time: 302.3s
WARNING: Expected aggregate file size       = 10817948876800000
WARNING: Stat() of aggregate file size      = 36187783823360
WARNING: Using actual aggregate bytes moved = 36187783823360
WARNING: Maybe caused by deadlineForStonewalling
write     77393      2418.78    0.001362    10158080000 32768      0.042550   445.88     0.002954   445.92     0   
ior-hard-read
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Oct 13 21:22:12 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.13-20.35.18/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.13-20.35.18/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a POSIX --posix.odirect=TRUE
Machine             : Linux h09u01.int.janelia.org
TestID              : 0
StartTime           : Tue Oct 13 21:22:12 2020
Path                : ./datadir/2020.10.13-20.35.18/ior-hard/file
FS                  : 150.0 TiB   Used FS: 30.6%   Inodes: 858.3 Mi   Used Inodes: 19.4%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datadir/2020.10.13-20.35.18/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 40
tasks               : 1040
clients per node    : 26
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 444.64 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 275946 max: 275946 -- min data: 12.1 GiB mean data: 12.1 GiB time: 347.3s
WARNING: Expected aggregate file size       = 488883200000000
WARNING: Stat() of aggregate file size      = 13490536350720
WARNING: Using actual aggregate bytes moved = 13490536350720
read      37006      825489     347.63      45.91      45.91      0.005567   347.65     0.002212   347.66     0   
ior-hard-write
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Oct 13 20:51:10 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.13-20.35.18/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.13-20.35.18/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a POSIX --posix.odirect=TRUE
Machine             : Linux h09u01.int.janelia.org
TestID              : 0
StartTime           : Tue Oct 13 20:51:10 2020
Path                : ./datadir/2020.10.13-20.35.18/ior-hard/file
FS                  : 150.0 TiB   Used FS: 22.3%   Inodes: 858.3 Mi   Used Inodes: 16.0%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datadir/2020.10.13-20.35.18/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 40
tasks               : 1040
clients per node    : 26
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 444.64 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 19 max: 275946 -- min data: 0.0 GiB mean data: 0.5 GiB time: 300.5s
WARNING: Expected aggregate file size       = 488883200000000
WARNING: Stat() of aggregate file size      = 13490536350720
WARNING: Using actual aggregate bytes moved = 13490536350720
WARNING: Maybe caused by deadlineForStonewalling
write     18051      402664     669.50      45.91      45.91      0.009886   712.71     0.001852   712.72     0   
mdtest-easy-delete
-- started at 10/13/2020 21:28:39 --

mdtest-3.4.0+dev was launched with 1040 total task(s) on 40 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datadir/2020.10.13-20.35.18/mdtest-easy' '-x' './results/2020.10.13-20.35.18/mdtest-easy.stonewall' '-r' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.13-20.35.18
FS: 150.0 TiB   Used FS: 30.6%   Inodes: 858.3 Mi   Used Inodes: 19.4%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 26 for each phase.
1040 tasks, 1040000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :     489987.523     489986.379     489987.186          0.197
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          1.254          1.254          1.254          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        218.438        218.437        218.438          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.798          0.798          0.798          0.000
-- finished at 10/13/2020 21:32:19 --

mdtest-easy-stat
-- started at 10/13/2020 21:21:40 --

mdtest-3.4.0+dev was launched with 1040 total task(s) on 40 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datadir/2020.10.13-20.35.18/mdtest-easy' '-x' './results/2020.10.13-20.35.18/mdtest-easy.stonewall' '-T' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.13-20.35.18
FS: 150.0 TiB   Used FS: 30.6%   Inodes: 858.3 Mi   Used Inodes: 19.4%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 26 for each phase.
1040 tasks, 1040000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :    3435109.776    3435097.974    3435103.660          2.451
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :         31.158         31.158         31.158          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/13/2020 21:22:11 --

mdtest-easy-write
-- started at 10/13/2020 20:42:44 --

mdtest-3.4.0+dev was launched with 1040 total task(s) on 40 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datadir/2020.10.13-20.35.18/mdtest-easy' '-x' './results/2020.10.13-20.35.18/mdtest-easy.stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.13-20.35.18
FS: 150.0 TiB   Used FS: 22.0%   Inodes: 858.3 Mi   Used Inodes: 3.8%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 26 for each phase.
1040 tasks, 1040000000 files
Continue stonewall hit min: 86734 max: 102915 avg: 96458.4 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :     289748.966     289748.493     289748.717          0.107
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA     334381.652             NA
   Tree creation             :         23.693         23.693         23.693          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        369.395        369.394        369.395          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.007             NA
   Tree creation             :          0.042          0.042          0.042          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/13/2020 20:51:10 --

mdtest-hard-delete
-- started at 10/13/2020 21:33:16 --

mdtest-3.4.0+dev was launched with 1040 total task(s) on 40 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.13-20.35.18/mdtest-hard' '-x' './results/2020.10.13-20.35.18/mdtest-hard.stonewall' '-r' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.13-20.35.18
FS: 150.0 TiB   Used FS: 30.3%   Inodes: 858.3 Mi   Used Inodes: 7.3%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 26 for each phase.
1040 tasks, 1040000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :      65697.338      65697.082      65697.284          0.038
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.358          0.358          0.358          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        273.182        273.181        273.181          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          2.796          2.796          2.796          0.000
-- finished at 10/13/2020 21:37:53 --

mdtest-hard-read
-- started at 10/13/2020 21:32:19 --

mdtest-3.4.0+dev was launched with 1040 total task(s) on 40 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.13-20.35.18/mdtest-hard' '-x' './results/2020.10.13-20.35.18/mdtest-hard.stonewall' '-E' '-X' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.13-20.35.18
FS: 150.0 TiB   Used FS: 30.3%   Inodes: 858.3 Mi   Used Inodes: 7.3%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 26 for each phase.
1040 tasks, 1040000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :     329946.880     329944.167     329946.281          0.398
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :         54.395         54.394         54.395          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/13/2020 21:33:15 --

mdtest-hard-stat
-- started at 10/13/2020 21:28:01 --

mdtest-3.4.0+dev was launched with 1040 total task(s) on 40 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.13-20.35.18/mdtest-hard' '-x' './results/2020.10.13-20.35.18/mdtest-hard.stonewall' '-T' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.13-20.35.18
FS: 150.0 TiB   Used FS: 30.6%   Inodes: 858.3 Mi   Used Inodes: 19.4%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 26 for each phase.
1040 tasks, 1040000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :     469734.761     469733.703     469734.197          0.220
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :         38.207         38.207         38.207          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/13/2020 21:28:39 --

mdtest-hard-write
-- started at 10/13/2020 21:03:02 --

mdtest-3.4.0+dev was launched with 1040 total task(s) on 40 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.13-20.35.18/mdtest-hard' '-x' './results/2020.10.13-20.35.18/mdtest-hard.stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.13-20.35.18
FS: 150.0 TiB   Used FS: 30.5%   Inodes: 858.3 Mi   Used Inodes: 17.4%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2216 Shifting ranks by 26 for each phase.
1040 tasks, 1040000000 files
Continue stonewall hit min: 15966 max: 17257 avg: 16599.3 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :      50162.887      50162.626      50162.826          0.037
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA      57540.176             NA
   Tree creation             :        786.776        786.776        786.776          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        357.782        357.780        357.780          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.021             NA
   Tree creation             :          0.001          0.001          0.001          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/13/2020 21:11:52 --

result
version         = io500-sc20
config-hash     = 16684A95
result-dir      = ./results/2020.10.13-20.35.18
; START 2020-10-13 20:35:18


[ior-easy-write]
t_start         = 2020-10-13 20:35:18
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.13-20.35.18/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.13-20.35.18/ior-easy.stonewall -t 32m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX
throughput-stonewall = 58066.77
score           = 75.579211
t_delta         = 446.0466
t_end           = 2020-10-13 20:42:44

[mdtest-easy-write]
t_start         = 2020-10-13 20:42:44
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d ./datadir/2020.10.13-20.35.18/mdtest-easy -x ./results/2020.10.13-20.35.18/mdtest-easy.stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 334.381652
score           = 289.748670
t_delta         = 505.3250
t_end           = 2020-10-13 20:51:10

[timestamp]
t_start         = 2020-10-13 20:51:10
t_delta         = 0.0012
t_end           = 2020-10-13 20:51:10

[ior-hard-write]
t_start         = 2020-10-13 20:51:10
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.13-20.35.18/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.13-20.35.18/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a POSIX --posix.odirect=TRUE
throughput-stonewall = 720.40
score           = 17.628231
t_delta         = 712.7429
t_end           = 2020-10-13 21:03:02

[mdtest-hard-write]
t_start         = 2020-10-13 21:03:02
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.13-20.35.18/mdtest-hard -x ./results/2020.10.13-20.35.18/mdtest-hard.stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 57.540176
score           = 50.162840
t_delta         = 529.5017
t_end           = 2020-10-13 21:11:52

[find]
t_start         = 2020-10-13 21:11:52
exe             = ./pfind ./datadir/2020.10.13-20.35.18 -newer ./results/2020.10.13-20.35.18/timestampfile -size 3901c -name *01* -C -q 10000
found           = 794960
total-files     = 124979921
score           = 892.413986
t_delta         = 140.0797
t_end           = 2020-10-13 21:14:12

[ior-easy-read]
t_start         = 2020-10-13 21:14:12
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.13-20.35.18/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.13-20.35.18/ior-easy.stonewall -t 32m -b 9920000m -F -r -R -a POSIX
score           = 75.237432
t_delta         = 448.0087
t_end           = 2020-10-13 21:21:40

[mdtest-easy-stat]
t_start         = 2020-10-13 21:21:40
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d ./datadir/2020.10.13-20.35.18/mdtest-easy -x ./results/2020.10.13-20.35.18/mdtest-easy.stonewall -T -a POSIX
score           = 3435.108094
t_delta         = 31.1893
t_end           = 2020-10-13 21:22:11

[ior-hard-read]
t_start         = 2020-10-13 21:22:11
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.13-20.35.18/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.13-20.35.18/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a POSIX --posix.odirect=TRUE
score           = 36.139061
t_delta         = 348.6890
t_end           = 2020-10-13 21:28:00

[mdtest-hard-stat]
t_start         = 2020-10-13 21:28:00
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.13-20.35.18/mdtest-hard -x ./results/2020.10.13-20.35.18/mdtest-hard.stonewall -T -a POSIX
score           = 469.734184
t_delta         = 39.2383
t_end           = 2020-10-13 21:28:39

[mdtest-easy-delete]
t_start         = 2020-10-13 21:28:39
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d ./datadir/2020.10.13-20.35.18/mdtest-easy -x ./results/2020.10.13-20.35.18/mdtest-easy.stonewall -r -a POSIX
score           = 489.987314
t_delta         = 220.1139
t_end           = 2020-10-13 21:32:19

[mdtest-hard-read]
t_start         = 2020-10-13 21:32:19
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.13-20.35.18/mdtest-hard -x ./results/2020.10.13-20.35.18/mdtest-hard.stonewall -E -X -a POSIX
score           = 329.946655
t_delta         = 55.5482
t_end           = 2020-10-13 21:33:15

[mdtest-hard-delete]
t_start         = 2020-10-13 21:33:15
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.13-20.35.18/mdtest-hard -x ./results/2020.10.13-20.35.18/mdtest-hard.stonewall -r -a POSIX
score           = 65.697296
t_delta         = 278.0114
t_end           = 2020-10-13 21:37:53

[SCORE]
MD              = 349.435739
BW              = 43.627013
SCORE           = 123.469986 
hash            = DF66D45B
; END 2020-10-13 21:37:57
result_summary
IO500 version io500-sc20
[RESULT]       ior-easy-write       75.579211 GiB/s : time 446.047 seconds
[RESULT]    mdtest-easy-write      289.748670 kIOPS : time 505.325 seconds
[RESULT]       ior-hard-write       17.628231 GiB/s : time 712.743 seconds
[RESULT]    mdtest-hard-write       50.162840 kIOPS : time 529.502 seconds
[RESULT]                 find      892.413986 kIOPS : time 140.080 seconds
[RESULT]        ior-easy-read       75.237432 GiB/s : time 448.009 seconds
[RESULT]     mdtest-easy-stat     3435.108094 kIOPS : time 31.189 seconds
[RESULT]        ior-hard-read       36.139061 GiB/s : time 348.689 seconds
[RESULT]     mdtest-hard-stat      469.734184 kIOPS : time 39.238 seconds
[RESULT]   mdtest-easy-delete      489.987314 kIOPS : time 220.114 seconds
[RESULT]     mdtest-hard-read      329.946655 kIOPS : time 55.548 seconds
[RESULT]   mdtest-hard-delete       65.697296 kIOPS : time 278.011 seconds
[SCORE] Bandwidth 43.627013 GB/s : IOPS 349.435739 kiops : TOTAL 123.469986