MOGON II

Institution Johannes Gutenberg University Mainz
Client Procs Per Node
Client Operating System CentOS
Client Operating System Version
Client Kernel Version 3.10.0-1127.8.2.el7.x86_64

DATA SERVER

Storage Type Intel SSD DC S4600 Series
Volatile Memory
Storage Interface SATA
Network Omnipath
Software Version 0.8dev
OS Version

INFORMATION

Client Nodes 10
Client Total Procs 160
Metadata Nodes 200
Metadata Storage Devices 1
Data Nodes 200
Data Storage Devices 1

METADATA

Easy Write 1,493.89 kIOP/s
Easy Stat 1,433.29 kIOP/s
Easy Delete 790.06 kIOP/s
Hard Write 246.26 kIOP/s
Hard Read 286.47 kIOP/s
Hard Stat 1,439.66 kIOP/s
Hard Delete 270.70 kIOP/s

Submitted Files

ior-easy-read
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Jul 14 00:13:51 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/vef_mountdir/io500-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/vef_mountdir/io500-app/ior-easy/stonewall -t 64m -b 9920000m -F -r -R -a POSIX
Machine             : Linux x0003.mogon
TestID              : 0
StartTime           : Tue Jul 14 00:13:51 2020
Path                : /tmp/vef_mountdir/io500-app/ior-easy
FS                  : 83.2 TiB   Used FS: 22.2%   Inodes: 0.0 Mi   Used Inodes: -nan%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /tmp/vef_mountdir/io500-app/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 160
clients per node    : 16
repetitions         : 1
xfersize            : 64 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 1513.67 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
WARNING: Expected aggregate file size       = 1664299827200000.
WARNING: Stat() of aggregate file size      = 18930068357120.
WARNING: Using actual aggregate bytes moved = 18930068357120.
read      23362      365.03     0.004986    10158080000 65536      0.002898   772.76     0.000274   772.76     0   
Max Read:  23361.73 MiB/sec (24496.54 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
read        23361.73   23361.73   23361.73       0.00     365.03     365.03     365.03       0.00  772.76483         NA            NA     0    160  16    1   1     1        1         0    0      1 10401873920000 67108864 18053120.0 POSIX      0
Finished            : Tue Jul 14 00:26:50 2020
ior-easy-write
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Mon Jul 13 23:48:52 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/vef_mountdir/io500-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/vef_mountdir/io500-app/ior-easy/stonewall -t 64m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX
Machine             : Linux x0003.mogon
TestID              : 0
StartTime           : Mon Jul 13 23:48:52 2020
Path                : /tmp/vef_mountdir/io500-app/ior-easy
FS                  : 83.2 TiB   Used FS: 0.0%   Inodes: 0.0 Mi   Used Inodes: -nan%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /tmp/vef_mountdir/io500-app/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 160
clients per node    : 16
repetitions         : 1
xfersize            : 64 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 1513.67 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 1475 max: 1763 -- min data: 92.2 GiB mean data: 99.0 GiB time: 300.3s
WARNING: Expected aggregate file size       = 1664299827200000.
WARNING: Stat() of aggregate file size      = 18930068357120.
WARNING: Using actual aggregate bytes moved = 18930068357120.
WARNING: maybe caused by deadlineForStonewalling
write     50012      781.45     0.002286    10158080000 65536      0.003253   360.97     0.000241   360.98     0   
Max Write: 50012.03 MiB/sec (52441.41 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
write       50012.03   50012.03   50012.03       0.00     781.44     781.44     781.44       0.00  360.97555     300.33      54014.45     0    160  16    1   1     1        1         0    0      1 10401873920000 67108864 18053120.0 POSIX      0
Finished            : Mon Jul 13 23:54:53 2020
ior-hard-read
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Jul 14 00:32:44 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/vef_mountdir/io500-app/ior-hard/file -O stoneWallingStatusFile=/tmp/vef_mountdir/io500-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a POSIX
Machine             : Linux x0003.mogon
TestID              : 0
StartTime           : Tue Jul 14 00:32:44 2020
Path                : /tmp/vef_mountdir/io500-app/ior-hard
FS                  : 83.2 TiB   Used FS: 22.2%   Inodes: 0.0 Mi   Used Inodes: -nan%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /tmp/vef_mountdir/io500-app/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 160
clients per node    : 16
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 68.41 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 123743 max: 123743 -- min data: 5.4 GiB mean data: 5.4 GiB time: 192.6s
WARNING: Expected aggregate file size       = 75212800000000.
WARNING: Stat() of aggregate file size      = 930705751040.
WARNING: Using actual aggregate bytes moved = 930705751040.
read      4609       102802     192.59      45.91      45.91      0.002671   192.59     0.000280   192.60     0   
Max Read:  4608.58 MiB/sec (4832.44 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
read         4608.58    4608.58    4608.58       0.00  102800.43  102800.43  102800.43       0.00  192.59531     192.56       4609.53     0    160  16    1   0     1        1         0    0 10000000    47008    47008  887590.2 POSIX      0
Finished            : Tue Jul 14 00:35:56 2020
ior-hard-write
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Jul 14 00:00:30 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/vef_mountdir/io500-app/ior-hard/file -O stoneWallingStatusFile=/tmp/vef_mountdir/io500-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a POSIX
Machine             : Linux x0003.mogon
TestID              : 0
StartTime           : Tue Jul 14 00:00:30 2020
Path                : /tmp/vef_mountdir/io500-app/ior-hard
FS                  : 83.2 TiB   Used FS: 20.7%   Inodes: 0.0 Mi   Used Inodes: -nan%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /tmp/vef_mountdir/io500-app/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 160
clients per node    : 16
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 68.41 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 122230 max: 123743 -- min data: 5.4 GiB mean data: 5.4 GiB time: 300.0s
WARNING: Expected aggregate file size       = 75212800000000.
WARNING: Stat() of aggregate file size      = 930705751040.
WARNING: Using actual aggregate bytes moved = 930705751040.
WARNING: maybe caused by deadlineForStonewalling
write     2944.17    65678      300.53      45.91      45.91      0.020139   301.45     0.000196   301.47     0   
Max Write: 2944.17 MiB/sec (3087.18 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
write        2944.17    2944.17    2944.17       0.00   65673.55   65673.55   65673.55       0.00  301.47418     300.02       2952.39     0    160  16    1   0     1        1         0    0 10000000    47008    47008  887590.2 POSIX      0
Finished            : Tue Jul 14 00:05:31 2020
mdtest-easy-delete
-- started at 07/14/2020 00:36:51 --

mdtest-3.3.0+dev was launched with 160 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '5000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/vef_mountdir/io500-app/mdtest-easy' '-x' '/tmp/vef_mountdir/io500-app/mdtest-easy-stonewall' '-r' '-a' 'POSIX'
Path: /tmp/vef_mountdir/io500-app
FS: 83.2 TiB   Used FS: 22.2%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 1000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000
V-0: Rank   0 Line  2166 Shifting ranks by 1 for each phase.
160 tasks, 800000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :     790057.512     790057.378     790057.445          0.034
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.052          0.052          0.052          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        631.993        631.993        631.993          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :         19.153         19.153         19.153          0.000
-- finished at 07/14/2020 00:47:43 --

mdtest-easy-stat
-- started at 07/14/2020 00:26:50 --

mdtest-3.3.0+dev was launched with 160 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '5000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/vef_mountdir/io500-app/mdtest-easy' '-x' '/tmp/vef_mountdir/io500-app/mdtest-easy-stonewall' '-T' '-a' 'POSIX'
Path: /tmp/vef_mountdir/io500-app
FS: 83.2 TiB   Used FS: 22.2%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 1000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000
V-0: Rank   0 Line  2166 Shifting ranks by 1 for each phase.
160 tasks, 800000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :    1433295.280    1433294.811    1433295.067          0.116
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :        348.366        348.365        348.366          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/14/2020 00:32:44 --

mdtest-easy-write
-- started at 07/13/2020 23:54:53 --

mdtest-3.3.0+dev was launched with 160 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '5000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/vef_mountdir/io500-app/mdtest-easy' '-x' '/tmp/vef_mountdir/io500-app/mdtest-easy-stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /tmp/vef_mountdir/io500-app
FS: 83.2 TiB   Used FS: 20.8%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 1000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000
V-0: Rank   0 Line  2166 Shifting ranks by 1 for each phase.
160 tasks, 800000000 files
Continue stonewall hit min: 2924988 max: 3120691 avg: 3043411.5 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :    1493886.707    1493885.613    1493886.223          0.249
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA    1624314.008             NA
   Tree creation             :          4.668          4.668          4.668          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        334.236        334.236        334.236          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        299.786             NA
   Tree creation             :          0.214          0.214          0.214          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/14/2020 00:00:30 --

mdtest-hard-delete
-- started at 07/14/2020 00:52:15 --

mdtest-3.3.0+dev was launched with 160 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/vef_mountdir/io500-app/mdtest-hard' '-x' '/tmp/vef_mountdir/io500-app/mdtest-hard-stonewall' '-r' '-a' 'POSIX'
Path: /tmp/vef_mountdir/io500-app
FS: 83.2 TiB   Used FS: 22.2%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 1000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000
V-0: Rank   0 Line  2166 Shifting ranks by 1 for each phase.
160 tasks, 160000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :     270704.092     270703.963     270704.024          0.027
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          4.149          4.149          4.149          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        287.890        287.890        287.890          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.241          0.241          0.241          0.000
-- finished at 07/14/2020 00:57:03 --

mdtest-hard-read
-- started at 07/14/2020 00:47:43 --

mdtest-3.3.0+dev was launched with 160 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/vef_mountdir/io500-app/mdtest-hard' '-x' '/tmp/vef_mountdir/io500-app/mdtest-hard-stonewall' '-E' '-X' '-a' 'POSIX'
Path: /tmp/vef_mountdir/io500-app
FS: 83.2 TiB   Used FS: 22.2%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 1000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000
V-0: Rank   0 Line  2166 Shifting ranks by 1 for each phase.
160 tasks, 160000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :     286467.237     286467.117     286467.174          0.029
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :        272.049        272.048        272.048          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/14/2020 00:52:15 --

mdtest-hard-stat
-- started at 07/14/2020 00:35:57 --

mdtest-3.3.0+dev was launched with 160 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/vef_mountdir/io500-app/mdtest-hard' '-x' '/tmp/vef_mountdir/io500-app/mdtest-hard-stonewall' '-T' '-a' 'POSIX'
Path: /tmp/vef_mountdir/io500-app
FS: 83.2 TiB   Used FS: 22.2%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 1000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000
V-0: Rank   0 Line  2166 Shifting ranks by 1 for each phase.
160 tasks, 160000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :    1439666.789    1439663.765    1439664.850          0.816
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :         54.133         54.133         54.133          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/14/2020 00:36:51 --

mdtest-hard-write
-- started at 07/14/2020 00:05:31 --

mdtest-3.3.0+dev was launched with 160 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/vef_mountdir/io500-app/mdtest-hard' '-x' '/tmp/vef_mountdir/io500-app/mdtest-hard-stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /tmp/vef_mountdir/io500-app
FS: 83.2 TiB   Used FS: 21.8%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 1000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000
V-0: Rank   0 Line  2166 Shifting ranks by 1 for each phase.
160 tasks, 160000000 files
Continue stonewall hit min: 471853 max: 487081 avg: 479481.1 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :     246264.600     246264.526     246264.567          0.019
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA     255724.078             NA
   Tree creation             :       1849.495       1849.495       1849.495          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        316.460        316.460        316.460          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        299.999             NA
   Tree creation             :          0.001          0.001          0.001          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/14/2020 00:10:50 --

result
version         = io500-isc20_v4
config-hash     = A1AB27EC
result-dir      = ./results/2020.07.13-23.48.39-app
; START 2020-07-13 23:48:52


[ior-easy-write]
t_start         = 2020-07-13 23:48:52
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/vef_mountdir/io500-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/vef_mountdir/io500-app/ior-easy/stonewall -t 64m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX
throughput-stonewall = 7021.83
score           = 48.839874
t_delta         = 361.1574
t_end           = 2020-07-13 23:54:53

[mdtest-easy-write]
t_start         = 2020-07-13 23:54:53
exe             = ./mdtest -n 5000000 -u -L -F -P -N 1 -d /tmp/vef_mountdir/io500-app/mdtest-easy -x /tmp/vef_mountdir/io500-app/mdtest-easy-stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 1624.314008
score           = 1493.886217
t_delta         = 336.9057
t_end           = 2020-07-14 00:00:30

[timestamp]
t_start         = 2020-07-14 00:00:30
t_delta         = 0.0007
t_end           = 2020-07-14 00:00:30

[ior-hard-write]
t_start         = 2020-07-14 00:00:30
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/vef_mountdir/io500-app/ior-hard/file -O stoneWallingStatusFile=/tmp/vef_mountdir/io500-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a POSIX
throughput-stonewall = 459.09
score           = 2.875163
t_delta         = 301.4943
t_end           = 2020-07-14 00:05:31

[mdtest-hard-write]
t_start         = 2020-07-14 00:05:31
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/vef_mountdir/io500-app/mdtest-hard -x /tmp/vef_mountdir/io500-app/mdtest-hard-stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 255.724078
score           = 246.264556
t_delta         = 319.0073
t_end           = 2020-07-14 00:10:50

[find]
t_start         = 2020-07-14 00:10:50
exe             = ./pfind /tmp/vef_mountdir/io500-app -newer /tmp/vef_mountdir/io500-app/timestampfile -size 3901c -name *01* -C -q 100000
nproc           = 160
found           = 3414589
total-files     = 577243686
score           = 3187.153144
t_delta         = 181.1240
t_end           = 2020-07-14 00:13:51

[ior-easy-read]
t_start         = 2020-07-14 00:13:51
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/vef_mountdir/io500-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/vef_mountdir/io500-app/ior-easy/stonewall -t 64m -b 9920000m -F -r -R -a POSIX
score           = 22.814185
t_delta         = 778.5340
t_end           = 2020-07-14 00:26:50

[mdtest-easy-stat]
t_start         = 2020-07-14 00:26:50
exe             = ./mdtest -n 5000000 -u -L -F -P -N 1 -d /tmp/vef_mountdir/io500-app/mdtest-easy -x /tmp/vef_mountdir/io500-app/mdtest-easy-stonewall -T -a POSIX
score           = 1433.294961
t_delta         = 354.0802
t_end           = 2020-07-14 00:32:44

[ior-hard-read]
t_start         = 2020-07-14 00:32:44
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/vef_mountdir/io500-app/ior-hard/file -O stoneWallingStatusFile=/tmp/vef_mountdir/io500-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a POSIX
score           = 4.500563
t_delta         = 192.6170
t_end           = 2020-07-14 00:35:56

[mdtest-hard-stat]
t_start         = 2020-07-14 00:35:56
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/vef_mountdir/io500-app/mdtest-hard -x /tmp/vef_mountdir/io500-app/mdtest-hard-stonewall -T -a POSIX
score           = 1439.663948
t_delta         = 54.2411
t_end           = 2020-07-14 00:36:51

[mdtest-easy-delete]
t_start         = 2020-07-14 00:36:51
exe             = ./mdtest -n 5000000 -u -L -F -P -N 1 -d /tmp/vef_mountdir/io500-app/mdtest-easy -x /tmp/vef_mountdir/io500-app/mdtest-easy-stonewall -r -a POSIX
score           = 790.057479
t_delta         = 652.0450
t_end           = 2020-07-14 00:47:43

[mdtest-hard-read]
t_start         = 2020-07-14 00:47:43
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/vef_mountdir/io500-app/mdtest-hard -x /tmp/vef_mountdir/io500-app/mdtest-hard-stonewall -E -X -a POSIX
score           = 286.467149
t_delta         = 272.0849
t_end           = 2020-07-14 00:52:15

[mdtest-hard-delete]
t_start         = 2020-07-14 00:52:15
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/vef_mountdir/io500-app/mdtest-hard -x /tmp/vef_mountdir/io500-app/mdtest-hard-stonewall -r -a POSIX
score           = 270.704087
t_delta         = 288.3892
t_end           = 2020-07-14 00:57:03

[SCORE]
MD              = 787.713746
BW              = 10.957896
SCORE           = 92.906864 
hash            = 913F6918
; END 2020-07-14 00:57:43
result_summary
IO500 version io500-isc20_v4
[RESULT]       ior-easy-write       48.839874 GiB/s  : time 361.157 seconds
[RESULT]    mdtest-easy-write     1493.886217 kIOPS : time 336.906 seconds
[RESULT]       ior-hard-write        2.875163 GiB/s  : time 301.494 seconds
[RESULT]    mdtest-hard-write      246.264556 kIOPS : time 319.007 seconds
[RESULT]                 find     3187.153144 kIOPS : time 181.124 seconds
[RESULT]        ior-easy-read       22.814185 GiB/s  : time 778.534 seconds
[RESULT]     mdtest-easy-stat     1433.294961 kIOPS : time 354.080 seconds
[RESULT]        ior-hard-read        4.500563 GiB/s  : time 192.617 seconds
[RESULT]     mdtest-hard-stat     1439.663948 kIOPS : time 54.241 seconds
[RESULT]   mdtest-easy-delete      790.057479 kIOPS : time 652.045 seconds
[RESULT]     mdtest-hard-read      286.467149 kIOPS : time 272.085 seconds
[RESULT]   mdtest-hard-delete      270.704087 kIOPS : time 288.389 seconds
[SCORE] Bandwidth 10.957896 GB/s : IOPS 787.713746 kiops : TOTAL 92.906864