DIME

Institution DDN
Client Procs Per Node
Client Operating System CentOS
Client Operating System Version 7.7
Client Kernel Version 3.10

DATA SERVER

Storage Type NVMe
Volatile Memory 216TB
Storage Interface EDR
Network IB-EDR
Software Version IME 1.4
OS Version Centos 7.7

INFORMATION

Client Nodes 10
Client Total Procs 110
Metadata Nodes 2
Metadata Storage Devices 13
Data Nodes 6
Data Storage Devices 108

METADATA

Easy Write 159.50 kIOP/s
Easy Stat 354.26 kIOP/s
Easy Delete 110.08 kIOP/s
Hard Write 24.48 kIOP/s
Hard Read 195.46 kIOP/s
Hard Stat 55.68 kIOP/s
Hard Delete 21.60 kIOP/s

Submitted Files

ior-easy-read
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Wed Jul  8 20:49:01 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-easy/stonewall -t 1M -b 9920000m -r -R -a POSIX
Machine             : Linux dcn-01
TestID              : 0
StartTime           : Wed Jul  8 20:49:01 2020
Path                : /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-easy
FS                  : 7.7 TiB   Used FS: 4.4%   Inodes: 177.5 Mi   Used Inodes: 64.8%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-easy/ior_file_easy
access              : single-shared-file
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 110
clients per node    : 11
repetitions         : 1
xfersize            : 1 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 1040.65 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
WARNING: Expected aggregate file size       = 1144206131200000.
WARNING: Stat() of aggregate file size      = 1134118383386624.
WARNING: Using actual aggregate bytes moved = 34553871728640.
read      100101     100102     0.000033    10158080000 1024.00    0.001196   329.20     0.001674   329.20     0   
Max Read:  100101.05 MiB/sec (104963.55 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
read       100101.05  100101.05  100101.05       0.00  100101.05  100101.05  100101.05       0.00  329.19876         NA            NA     0    110  11    1   0     1        1         0    0      1 10401873920000  1048576 32953140.0 POSIX      0
Finished            : Wed Jul  8 20:54:31 2020
ior-easy-write
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Wed Jul  8 20:26:24 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-easy/stonewall -t 1M -b 9920000m -w -D 300 -O stoneWallingWearOut=1 -a POSIX
Machine             : Linux dcn-01
TestID              : 0
StartTime           : Wed Jul  8 20:26:24 2020
Path                : /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-easy
FS                  : 7.7 TiB   Used FS: 4.4%   Inodes: 119.9 Mi   Used Inodes: 47.9%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-easy/ior_file_easy
access              : single-shared-file
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 110
clients per node    : 11
repetitions         : 1
xfersize            : 1 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 1040.65 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 287413 max: 299574 -- min data: 280.7 GiB mean data: 286.2 GiB time: 300.0s
WARNING: Expected aggregate file size       = 1144206131200000.
WARNING: Stat() of aggregate file size      = 1134118383386624.
WARNING: Using actual aggregate bytes moved = 34553871728640.
WARNING: maybe caused by deadlineForStonewalling
write     105611     105615     0.000031    10158080000 1024.00    0.010360   312.01     0.000084   312.02     0   
Max Write: 105611.44 MiB/sec (110741.62 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
write      105611.44  105611.44  105611.44       0.00  105611.44  105611.44  105611.44       0.00  312.02245     300.01     107467.00     0    110  11    1   0     1        1         0    0      1 10401873920000  1048576 32953140.0 POSIX      0
Finished            : Wed Jul  8 20:31:36 2020
ior-hard-read
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Wed Jul  8 20:57:07 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-hard/file -O stoneWallingStatusFile=/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a POSIX
Machine             : Linux dcn-01
TestID              : 0
StartTime           : Wed Jul  8 20:57:07 2020
Path                : /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-hard
FS                  : 7.7 TiB   Used FS: 4.4%   Inodes: 177.5 Mi   Used Inodes: 64.8%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 110
clients per node    : 11
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 47.03 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 6787520 max: 6787520 -- min data: 297.2 GiB mean data: 297.2 GiB time: 342.0s
WARNING: Expected aggregate file size       = 51708800000000.
WARNING: Stat() of aggregate file size      = 35097451417600.
WARNING: Using actual aggregate bytes moved = 35097451417600.
read      97785      2181269    342.29      45.91      45.91      0.001195   342.29     0.007504   342.30     0   
Max Read:  97784.54 MiB/sec (102534.52 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
read        97784.54   97784.54   97784.54       0.00 2181214.23 2181214.23 2181214.23       0.00  342.29889     342.01      97867.71     0    110  11    1   0     1        1         0    0 10000000    47008    47008 33471538.0 POSIX      0
Finished            : Wed Jul  8 21:02:50 2020
ior-hard-write
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Wed Jul  8 20:37:46 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-hard/file -O stoneWallingStatusFile=/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a POSIX
Machine             : Linux dcn-01
TestID              : 0
StartTime           : Wed Jul  8 20:37:46 2020
Path                : /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-hard
FS                  : 7.7 TiB   Used FS: 4.4%   Inodes: 170.5 Mi   Used Inodes: 63.4%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 110
clients per node    : 11
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 47.03 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 6554980 max: 6787520 -- min data: 287.0 GiB mean data: 292.5 GiB time: 300.0s
WARNING: Expected aggregate file size       = 51708800000000.
WARNING: Stat() of aggregate file size      = 35097451417600.
WARNING: Using actual aggregate bytes moved = 35097451417600.
WARNING: maybe caused by deadlineForStonewalling
write     107929     2407579    310.06      45.91      45.91      0.009409   310.12     0.000095   310.12     0   
Max Write: 107929.24 MiB/sec (113172.01 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
write      107929.24  107929.24  107929.24       0.00 2407505.32 2407505.32 2407505.32       0.00  310.12484     300.01     109822.03     0    110  11    1   0     1        1         0    0 10000000    47008    47008 33471538.0 POSIX      0
Finished            : Wed Jul  8 20:42:57 2020
mdtest-easy-delete
-- started at 07/08/2020 21:05:11 --

mdtest-3.3.0+dev was launched with 110 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-easy' '-x' '/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-easy-stonewall' '-r' '-a' 'POSIX'
Path: /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app
FS: 7.7 TiB   Used FS: 4.4%   Inodes: 182.8 Mi   Used Inodes: 65.9%

Nodemap: 11111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 11 for each phase.
110 tasks, 110000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :     110081.187     110081.181     110081.184          0.001
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          1.295          1.295          1.295          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        504.885        504.885        504.885          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.772          0.772          0.772          0.000
-- finished at 07/08/2020 21:13:36 --

mdtest-easy-stat
-- started at 07/08/2020 20:54:31 --

mdtest-3.3.0+dev was launched with 110 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-easy' '-x' '/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-easy-stonewall' '-T' '-a' 'POSIX'
Path: /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app
FS: 7.7 TiB   Used FS: 4.4%   Inodes: 182.8 Mi   Used Inodes: 65.9%

Nodemap: 11111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 11 for each phase.
110 tasks, 110000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :     354262.101     354262.039     354262.065          0.012
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :        156.885        156.885        156.885          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/08/2020 20:57:07 --

mdtest-easy-write
-- started at 07/08/2020 20:31:36 --

mdtest-3.3.0+dev was launched with 110 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-easy' '-x' '/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-easy-stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app
FS: 7.7 TiB   Used FS: 4.4%   Inodes: 122.3 Mi   Used Inodes: 49.0%

Nodemap: 11111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 11 for each phase.
110 tasks, 110000000 files
Continue stonewall hit min: 442382 max: 505258 avg: 467893.5 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :     159504.386     159504.349     159504.369          0.009
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA     171567.121             NA
   Tree creation             :         12.025         12.025         12.025          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        348.444        348.444        348.444          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        299.989             NA
   Tree creation             :          0.083          0.083          0.083          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/08/2020 20:37:46 --

mdtest-hard-delete
-- started at 07/08/2020 21:14:16 --

mdtest-3.3.0+dev was launched with 110 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-hard' '-x' '/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-hard-stonewall' '-r' '-a' 'POSIX'
Path: /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app
FS: 7.7 TiB   Used FS: 4.4%   Inodes: 129.8 Mi   Used Inodes: 51.9%

Nodemap: 11111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 11 for each phase.
110 tasks, 110000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :      21597.392      21597.385      21597.391          0.001
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          8.457          8.457          8.457          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        363.059        363.059        363.059          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.118          0.118          0.118          0.000
-- finished at 07/08/2020 21:20:20 --

mdtest-hard-read
-- started at 07/08/2020 21:13:36 --

mdtest-3.3.0+dev was launched with 110 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-hard' '-x' '/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-hard-stonewall' '-E' '-X' '-a' 'POSIX'
Path: /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app
FS: 7.7 TiB   Used FS: 4.4%   Inodes: 129.8 Mi   Used Inodes: 51.9%

Nodemap: 11111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 11 for each phase.
110 tasks, 110000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :     195455.836     195455.630     195455.722          0.044
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :         40.117         40.117         40.117          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/08/2020 21:14:16 --

mdtest-hard-stat
-- started at 07/08/2020 21:02:50 --

mdtest-3.3.0+dev was launched with 110 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-hard' '-x' '/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-hard-stonewall' '-T' '-a' 'POSIX'
Path: /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app
FS: 7.7 TiB   Used FS: 4.4%   Inodes: 182.8 Mi   Used Inodes: 65.9%

Nodemap: 11111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 11 for each phase.
110 tasks, 110000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :      55678.109      55678.096      55678.101          0.002
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :        140.830        140.830        140.830          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/08/2020 21:05:11 --

mdtest-hard-write
-- started at 07/08/2020 20:42:57 --

mdtest-3.3.0+dev was launched with 110 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-hard' '-x' '/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-hard-stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app
FS: 7.7 TiB   Used FS: 4.4%   Inodes: 175.4 Mi   Used Inodes: 64.4%

Nodemap: 11111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 11 for each phase.
110 tasks, 110000000 files
Continue stonewall hit min: 69348 max: 71283 avg: 70251.6 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :      24479.073      24479.067      24479.071          0.001
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA      25783.451             NA
   Tree creation             :         89.507         89.507         89.507          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        320.320        320.320        320.320          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        299.715             NA
   Tree creation             :          0.011          0.011          0.011          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/08/2020 20:48:42 --

result
version         = io500-isc20_v4-1-g18c6d03b40b0
config-hash     = BB220FFE
result-dir      = ./results/2020.07.08-20.26.08-app
; START 2020-07-08 20:26:24


[ior-easy-write]
t_start         = 2020-07-08 20:26:24
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-easy/stonewall -t 1M -b 9920000m -w -D 300 -O stoneWallingWearOut=1 -a POSIX
throughput-stonewall = 11099.92
score           = 103.136170
t_delta         = 312.1314
t_end           = 2020-07-08 20:31:36

[mdtest-easy-write]
t_start         = 2020-07-08 20:31:36
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-easy -x /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-easy-stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 171.567121
score           = 159.504365
t_delta         = 370.5706
t_end           = 2020-07-08 20:37:46

[timestamp]
t_start         = 2020-07-08 20:37:46
t_delta         = 0.0319
t_end           = 2020-07-08 20:37:46

[ior-hard-write]
t_start         = 2020-07-08 20:37:46
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-hard/file -O stoneWallingStatusFile=/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a POSIX
throughput-stonewall = 11412.51
score           = 105.399648
t_delta         = 310.2007
t_end           = 2020-07-08 20:42:57

[mdtest-hard-write]
t_start         = 2020-07-08 20:42:57
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-hard -x /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-hard-stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 25.783451
score           = 24.479070
t_delta         = 345.6044
t_end           = 2020-07-08 20:48:42

[find]
t_start         = 2020-07-08 20:48:42
exe             = ./pfind /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app -newer /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/timestampfile -size 3901c -name *01* -C -q 10000
found           = 301818
total-files     = 63419517
score           = 3322.340544
t_delta         = 19.1167
t_end           = 2020-07-08 20:49:01

[ior-easy-read]
t_start         = 2020-07-08 20:49:01
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-easy/stonewall -t 1M -b 9920000m -r -R -a POSIX
score           = 97.754928
t_delta         = 329.2150
t_end           = 2020-07-08 20:54:31

[mdtest-easy-stat]
t_start         = 2020-07-08 20:54:31
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-easy -x /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-easy-stonewall -T -a POSIX
score           = 354.262069
t_delta         = 156.8949
t_end           = 2020-07-08 20:57:07

[ior-hard-read]
t_start         = 2020-07-08 20:57:07
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-hard/file -O stoneWallingStatusFile=/tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a POSIX
score           = 95.492712
t_delta         = 342.3499
t_end           = 2020-07-08 21:02:50

[mdtest-hard-stat]
t_start         = 2020-07-08 21:02:50
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-hard -x /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-hard-stonewall -T -a POSIX
score           = 55.678102
t_delta         = 140.8419
t_end           = 2020-07-08 21:05:11

[mdtest-easy-delete]
t_start         = 2020-07-08 21:05:11
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-easy -x /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-easy-stonewall -r -a POSIX
score           = 110.081185
t_delta         = 505.6893
t_end           = 2020-07-08 21:13:36

[mdtest-hard-read]
t_start         = 2020-07-08 21:13:36
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-hard -x /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-hard-stonewall -E -X -a POSIX
score           = 195.455761
t_delta         = 40.1261
t_end           = 2020-07-08 21:14:16

[mdtest-hard-delete]
t_start         = 2020-07-08 21:14:16
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-hard -x /tmp/fuseMNT/datafiles/2020.07.08-20.26.08-app/mdtest-hard-stonewall -r -a POSIX
score           = 21.597391
t_delta         = 363.2190
t_end           = 2020-07-08 21:20:20

[SCORE]
MD              = 136.269275
BW              = 100.366727
SCORE           = 116.948284 
hash            = DE7B384E
; END 2020-07-08 21:20:20
result_summary
IO500 version io500-isc20_v4-1-g18c6d03b40b0
[RESULT]       ior-easy-write      103.136170 GiB/s  : time 312.131 seconds
[RESULT]    mdtest-easy-write      159.504365 kIOPS : time 370.571 seconds
[RESULT]       ior-hard-write      105.399648 GiB/s  : time 310.201 seconds
[RESULT]    mdtest-hard-write       24.479070 kIOPS : time 345.604 seconds
[RESULT]                 find     3322.340544 kIOPS : time 19.117 seconds
[RESULT]        ior-easy-read       97.754928 GiB/s  : time 329.215 seconds
[RESULT]     mdtest-easy-stat      354.262069 kIOPS : time 156.895 seconds
[RESULT]        ior-hard-read       95.492712 GiB/s  : time 342.350 seconds
[RESULT]     mdtest-hard-stat       55.678102 kIOPS : time 140.842 seconds
[RESULT]   mdtest-easy-delete      110.081185 kIOPS : time 505.689 seconds
[RESULT]     mdtest-hard-read      195.455761 kIOPS : time 40.126 seconds
[RESULT]   mdtest-hard-delete       21.597391 kIOPS : time 363.219 seconds
[SCORE] Bandwidth 100.366727 GB/s : IOPS 136.269275 kiops : TOTAL 116.948284