bracewell

Institution CSIRO
Client Procs Per Node
Client Operating System CentOS
Client Operating System Version 7.7
Client Kernel Version 3.10

DATA SERVER

Storage Type NVMe
Volatile Memory 192
Storage Interface NVMe
Network IB-EDR
Software Version 2.12.3
OS Version 7.7

INFORMATION

Client Nodes 10
Client Total Procs 400
Metadata Nodes 8
Metadata Storage Devices 4
Data Nodes 8
Data Storage Devices 20

METADATA

Easy Write 226.97 kIOP/s
Easy Stat 460.22 kIOP/s
Easy Delete 181.27 kIOP/s
Hard Write 157.85 kIOP/s
Hard Read 205.72 kIOP/s
Hard Stat 334.92 kIOP/s
Hard Delete 104.03 kIOP/s

Submitted Files

ior-easy-read
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Jul 14 14:16:48 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o ./datafiles/2020.07.14-13.32.08-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=./datafiles/2020.07.14-13.32.08-app/ior-easy/stonewall -t 16m -b 130000m -F -r -R -a POSIX --posix.odirect
Machine             : Linux bss022
TestID              : 0
StartTime           : Tue Jul 14 14:16:48 2020
Path                : /scratch2/leh015/io500-app/datafiles/2020.07.14-13.32.08-app/ior-easy
FS                  : 457.2 TiB   Used FS: 11.6%   Inodes: 3986.9 Mi   Used Inodes: 7.1%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datafiles/2020.07.14-13.32.08-app/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 400
clients per node    : 40
repetitions         : 1
xfersize            : 16 MiB
blocksize           : 126.95 GiB
aggregate filesize  : 49.59 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
read      134025     8379       0.047716    133120000  16384      0.102313   387.85     0.105277   387.99     0   
Max Read:  134025.44 MiB/sec (140535.86 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
read       134025.44  134025.44  134025.44       0.00    8376.59    8376.59    8376.59       0.00  387.98604         NA            NA     0    400  40    1   1     1        1         0    0      1 136314880000 16777216 52000000.0 POSIX      0
Finished            : Tue Jul 14 14:23:17 2020
ior-easy-write
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Jul 14 13:32:19 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o ./datafiles/2020.07.14-13.32.08-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=./datafiles/2020.07.14-13.32.08-app/ior-easy/stonewall -t 16m -b 130000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX --posix.odirect
Machine             : Linux bss022
TestID              : 0
StartTime           : Tue Jul 14 13:32:19 2020
Path                : /scratch2/leh015/io500-app/datafiles/2020.07.14-13.32.08-app/ior-easy
FS                  : 457.2 TiB   Used FS: 0.0%   Inodes: 3779.5 Mi   Used Inodes: 2.0%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datafiles/2020.07.14-13.32.08-app/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 400
clients per node    : 40
repetitions         : 1
xfersize            : 16 MiB
blocksize           : 126.95 GiB
aggregate filesize  : 49.59 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 4566 max: 8125 -- min data: 71.3 GiB mean data: 118.9 GiB time: 300.1s
write     139582     8728       0.040890    133120000  16384      0.080381   372.37     0.089181   372.54     0   
Max Write: 139582.32 MiB/sec (146362.67 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
write      139582.32  139582.32  139582.32       0.00    8723.90    8723.90    8723.90       0.00  372.54002     300.06     162242.23     0    400  40    1   1     1        1         0    0      1 136314880000 16777216 52000000.0 POSIX      0
Finished            : Tue Jul 14 13:38:32 2020
ior-hard-read
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Jul 14 14:28:14 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o ./datafiles/2020.07.14-13.32.08-app/ior-hard/file -O stoneWallingStatusFile=./datafiles/2020.07.14-13.32.08-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 200000 -r -R -a POSIX
Machine             : Linux bss022
TestID              : 0
StartTime           : Tue Jul 14 14:28:14 2020
Path                : /scratch2/leh015/io500-app/datafiles/2020.07.14-13.32.08-app/ior-hard
FS                  : 457.2 TiB   Used FS: 11.6%   Inodes: 3986.9 Mi   Used Inodes: 7.1%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datafiles/2020.07.14-13.32.08-app/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 200000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 400
clients per node    : 40
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 3.42 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 200000 max: 200000 -- min data: 8.8 GiB mean data: 8.8 GiB time: 259.2s
read      13816      308374     259.29      45.91      45.91      0.142799   259.42     0.188501   259.59     0   
Max Read:  13815.85 MiB/sec (14486.97 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
read        13815.85   13815.85   13815.85       0.00  308180.93  308180.93  308180.93       0.00  259.58777     259.20      13836.41     0    400  40    1   0     1        1         0    0 200000    47008    47008 3586425.8 POSIX      0
Finished            : Tue Jul 14 14:32:35 2020
ior-hard-write
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Jul 14 13:51:44 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o ./datafiles/2020.07.14-13.32.08-app/ior-hard/file -O stoneWallingStatusFile=./datafiles/2020.07.14-13.32.08-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 200000 -w -D 300 -a POSIX
Machine             : Linux bss022
TestID              : 0
StartTime           : Tue Jul 14 13:51:44 2020
Path                : /scratch2/leh015/io500-app/datafiles/2020.07.14-13.32.08-app/ior-hard
FS                  : 457.2 TiB   Used FS: 10.8%   Inodes: 3909.2 Mi   Used Inodes: 5.3%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : ./datafiles/2020.07.14-13.32.08-app/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 200000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 400
clients per node    : 40
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 3.42 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 170587 max: 200000 -- min data: 7.5 GiB mean data: 8.5 GiB time: 300.1s
write     10751      239906     300.31      45.91      45.91      0.110289   333.46     0.078936   333.59     0   
Max Write: 10751.12 MiB/sec (11273.36 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
write       10751.12   10751.12   10751.12       0.00  239817.98  239817.98  239817.98       0.00  333.58634     300.07      11588.32     0    400  40    1   0     1        1         0    0 200000    47008    47008 3586425.8 POSIX      0
Finished            : Tue Jul 14 13:57:18 2020
mdtest-easy-delete
-- started at 07/14/2020 14:36:40 --

mdtest-3.3.0+dev was launched with 400 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '340000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datafiles/2020.07.14-13.32.08-app/mdtest-easy' '-x' './datafiles/2020.07.14-13.32.08-app/mdtest-easy-stonewall' '-r' '-a' 'POSIX'
Path: /scratch2/leh015/io500-app/datafiles/2020.07.14-13.32.08-app
FS: 457.2 TiB   Used FS: 11.6%   Inodes: 3986.9 Mi   Used Inodes: 7.1%

Nodemap: 1111111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 40 for each phase.
400 tasks, 136000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :     181276.577     181264.386     181270.653          2.068
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          1.044          1.044          1.044          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        750.285        750.235        750.259          0.009
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.957          0.957          0.957          0.000
-- finished at 07/14/2020 14:49:12 --

mdtest-easy-stat
-- started at 07/14/2020 14:23:17 --

mdtest-3.3.0+dev was launched with 400 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '340000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datafiles/2020.07.14-13.32.08-app/mdtest-easy' '-x' './datafiles/2020.07.14-13.32.08-app/mdtest-easy-stonewall' '-T' '-a' 'POSIX'
Path: /scratch2/leh015/io500-app/datafiles/2020.07.14-13.32.08-app
FS: 457.2 TiB   Used FS: 11.6%   Inodes: 3986.9 Mi   Used Inodes: 7.1%

Nodemap: 1111111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 40 for each phase.
400 tasks, 136000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :     460257.239     460180.939     460223.177         14.138
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :        295.536        295.487        295.509          0.009
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/14/2020 14:28:13 --

mdtest-easy-write
-- started at 07/14/2020 13:38:33 --

mdtest-3.3.0+dev was launched with 400 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '340000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datafiles/2020.07.14-13.32.08-app/mdtest-easy' '-x' './datafiles/2020.07.14-13.32.08-app/mdtest-easy-stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /scratch2/leh015/io500-app/datafiles/2020.07.14-13.32.08-app
FS: 457.2 TiB   Used FS: 10.8%   Inodes: 3779.5 Mi   Used Inodes: 2.0%

Nodemap: 1111111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 40 for each phase.
400 tasks, 136000000 files
Continue stonewall hit min: 244018 max: 340000 avg: 323885.3 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :     226982.195     226969.437     226976.016          2.894
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA     432644.046             NA
   Tree creation             :          1.923          1.923          1.923          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        599.200        599.166        599.182          0.008
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        299.447             NA
   Tree creation             :          0.520          0.520          0.520          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/14/2020 13:51:43 --

mdtest-hard-delete
-- started at 07/14/2020 14:55:50 --

mdtest-3.3.0+dev was launched with 400 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '800000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datafiles/2020.07.14-13.32.08-app/mdtest-hard' '-x' './datafiles/2020.07.14-13.32.08-app/mdtest-hard-stonewall' '-r' '-a' 'POSIX'
Path: /scratch2/leh015/io500-app/datafiles/2020.07.14-13.32.08-app
FS: 457.2 TiB   Used FS: 11.6%   Inodes: 3857.2 Mi   Used Inodes: 4.0%

Nodemap: 1111111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 40 for each phase.
400 tasks, 320000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :     104034.684     104030.884     104033.083          0.834
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.180          0.180          0.180          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        782.602        782.574        782.586          0.006
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          5.562          5.562          5.562          0.000
-- finished at 07/14/2020 15:08:59 --

mdtest-hard-read
-- started at 07/14/2020 14:49:13 --

mdtest-3.3.0+dev was launched with 400 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '800000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datafiles/2020.07.14-13.32.08-app/mdtest-hard' '-x' './datafiles/2020.07.14-13.32.08-app/mdtest-hard-stonewall' '-E' '-X' '-a' 'POSIX'
Path: /scratch2/leh015/io500-app/datafiles/2020.07.14-13.32.08-app
FS: 457.2 TiB   Used FS: 11.6%   Inodes: 3857.2 Mi   Used Inodes: 4.0%

Nodemap: 1111111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 40 for each phase.
400 tasks, 320000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :     205729.084     205702.888     205723.248          3.229
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :        395.788        395.738        395.749          0.006
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/14/2020 14:55:49 --

mdtest-hard-stat
-- started at 07/14/2020 14:32:35 --

mdtest-3.3.0+dev was launched with 400 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '800000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datafiles/2020.07.14-13.32.08-app/mdtest-hard' '-x' './datafiles/2020.07.14-13.32.08-app/mdtest-hard-stonewall' '-T' '-a' 'POSIX'
Path: /scratch2/leh015/io500-app/datafiles/2020.07.14-13.32.08-app
FS: 457.2 TiB   Used FS: 11.6%   Inodes: 3986.9 Mi   Used Inodes: 7.1%

Nodemap: 1111111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 40 for each phase.
400 tasks, 320000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :     334975.240     334868.193     334936.367         15.088
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :        243.125        243.047        243.075          0.011
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/14/2020 14:36:39 --

mdtest-hard-write
-- started at 07/14/2020 13:57:19 --

mdtest-3.3.0+dev was launched with 400 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '800000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datafiles/2020.07.14-13.32.08-app/mdtest-hard' '-x' './datafiles/2020.07.14-13.32.08-app/mdtest-hard-stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /scratch2/leh015/io500-app/datafiles/2020.07.14-13.32.08-app
FS: 457.2 TiB   Used FS: 11.6%   Inodes: 3909.2 Mi   Used Inodes: 5.3%

Nodemap: 1111111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 40 for each phase.
400 tasks, 320000000 files
Continue stonewall hit min: 148684 max: 203537 avg: 169990.8 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :     157856.839     157851.889     157854.225          1.439
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA     227302.556             NA
   Tree creation             :          1.185          1.185          1.185          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        515.767        515.751        515.759          0.005
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        299.145             NA
   Tree creation             :          0.844          0.844          0.844          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/14/2020 14:09:33 --

result
version         = io500-isc20_v4
config-hash     = 350928F6
result-dir      = ./results/2020.07.14-13.32.08-app
; START 2020-07-14 13:32:19


[ior-easy-write]
t_start         = 2020-07-14 13:32:19
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o ./datafiles/2020.07.14-13.32.08-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=./datafiles/2020.07.14-13.32.08-app/ior-easy/stonewall -t 16m -b 130000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX --posix.odirect
throughput-stonewall = 51046.35
score           = 136.310860
t_delta         = 373.6344
t_end           = 2020-07-14 13:38:32

[mdtest-easy-write]
t_start         = 2020-07-14 13:38:32
exe             = ./mdtest -n 340000 -u -L -F -P -N 1 -d ./datafiles/2020.07.14-13.32.08-app/mdtest-easy -x ./datafiles/2020.07.14-13.32.08-app/mdtest-easy-stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 432.644046
score           = 226.972531
t_delta         = 790.5915
t_end           = 2020-07-14 13:51:43

[timestamp]
t_start         = 2020-07-14 13:51:43
t_delta         = 0.0037
t_end           = 2020-07-14 13:51:43

[ior-hard-write]
t_start         = 2020-07-14 13:51:43
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o ./datafiles/2020.07.14-13.32.08-app/ior-hard/file -O stoneWallingStatusFile=./datafiles/2020.07.14-13.32.08-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 200000 -w -D 300 -a POSIX
throughput-stonewall = 4071.84
score           = 10.499138
t_delta         = 335.3438
t_end           = 2020-07-14 13:57:18

[mdtest-hard-write]
t_start         = 2020-07-14 13:57:18
exe             = ./mdtest -n 800000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datafiles/2020.07.14-13.32.08-app/mdtest-hard -x ./datafiles/2020.07.14-13.32.08-app/mdtest-hard-stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 227.302556
score           = 157.853084
t_delta         = 734.9323
t_end           = 2020-07-14 14:09:33

[find]
t_start         = 2020-07-14 14:09:33
exe             = ./pfind ./datafiles/2020.07.14-13.32.08-app -newer ./datafiles/2020.07.14-13.32.08-app/timestampfile -size 3901c -name *01* -C -q 10000
found           = 3788596
total-files     = 217415206
score           = 501.265724
t_delta         = 433.9271
t_end           = 2020-07-14 14:16:47

[ior-easy-read]
t_start         = 2020-07-14 14:16:47
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o ./datafiles/2020.07.14-13.32.08-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=./datafiles/2020.07.14-13.32.08-app/ior-easy/stonewall -t 16m -b 130000m -F -r -R -a POSIX --posix.odirect
score           = 130.884220
t_delta         = 389.3887
t_end           = 2020-07-14 14:23:17

[mdtest-easy-stat]
t_start         = 2020-07-14 14:23:17
exe             = ./mdtest -n 340000 -u -L -F -P -N 1 -d ./datafiles/2020.07.14-13.32.08-app/mdtest-easy -x ./datafiles/2020.07.14-13.32.08-app/mdtest-easy-stonewall -T -a POSIX
score           = 460.215007
t_delta         = 296.8981
t_end           = 2020-07-14 14:28:13

[ior-hard-read]
t_start         = 2020-07-14 14:28:14
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o ./datafiles/2020.07.14-13.32.08-app/ior-hard/file -O stoneWallingStatusFile=./datafiles/2020.07.14-13.32.08-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 200000 -r -R -a POSIX
score           = 13.492041
t_delta         = 261.0501
t_end           = 2020-07-14 14:32:35

[mdtest-hard-stat]
t_start         = 2020-07-14 14:32:35
exe             = ./mdtest -n 800000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datafiles/2020.07.14-13.32.08-app/mdtest-hard -x ./datafiles/2020.07.14-13.32.08-app/mdtest-hard-stonewall -T -a POSIX
score           = 334.916194
t_delta         = 244.5316
t_end           = 2020-07-14 14:36:39

[mdtest-easy-delete]
t_start         = 2020-07-14 14:36:39
exe             = ./mdtest -n 340000 -u -L -F -P -N 1 -d ./datafiles/2020.07.14-13.32.08-app/mdtest-easy -x ./datafiles/2020.07.14-13.32.08-app/mdtest-easy-stonewall -r -a POSIX
score           = 181.269607
t_delta         = 752.7082
t_end           = 2020-07-14 14:49:12

[mdtest-hard-read]
t_start         = 2020-07-14 14:49:12
exe             = ./mdtest -n 800000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datafiles/2020.07.14-13.32.08-app/mdtest-hard -x ./datafiles/2020.07.14-13.32.08-app/mdtest-hard-stonewall -E -X -a POSIX
score           = 205.722017
t_delta         = 397.1382
t_end           = 2020-07-14 14:55:49

[mdtest-hard-delete]
t_start         = 2020-07-14 14:55:49
exe             = ./mdtest -n 800000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datafiles/2020.07.14-13.32.08-app/mdtest-hard -x ./datafiles/2020.07.14-13.32.08-app/mdtest-hard-stonewall -r -a POSIX
score           = 104.033406
t_delta         = 789.7554
t_end           = 2020-07-14 15:08:59

[SCORE]
MD              = 239.260627
BW              = 39.871471
SCORE           = 97.671251 
hash            = 12BEAD0C
; END 2020-07-14 15:09:00
result_summary
IO500 version io500-isc20_v4
[RESULT]       ior-easy-write      136.310860 GiB/s  : time 373.634 seconds
[RESULT]    mdtest-easy-write      226.972531 kIOPS : time 790.592 seconds
[RESULT]       ior-hard-write       10.499138 GiB/s  : time 335.344 seconds
[RESULT]    mdtest-hard-write      157.853084 kIOPS : time 734.932 seconds
[RESULT]                 find      501.265724 kIOPS : time 433.927 seconds
[RESULT]        ior-easy-read      130.884220 GiB/s  : time 389.389 seconds
[RESULT]     mdtest-easy-stat      460.215007 kIOPS : time 296.898 seconds
[RESULT]        ior-hard-read       13.492041 GiB/s  : time 261.050 seconds
[RESULT]     mdtest-hard-stat      334.916194 kIOPS : time 244.532 seconds
[RESULT]   mdtest-easy-delete      181.269607 kIOPS : time 752.708 seconds
[RESULT]     mdtest-hard-read      205.722017 kIOPS : time 397.138 seconds
[RESULT]   mdtest-hard-delete      104.033406 kIOPS : time 789.755 seconds
[SCORE] Bandwidth 39.871471 GB/s : IOPS 239.260627 kiops : TOTAL 97.671251