NURION

Institution Korea Institute of Science and Technology Information (KISTI)
Client Procs Per Node
Client Operating System CentOS
Client Operating System Version 7.7
Client Kernel Version 3.10.0-1062.el7.x86_64

DATA SERVER

Storage Type
Volatile Memory
Storage Interface
Network
Software Version
OS Version

INFORMATION

Client Nodes 2,048
Client Total Procs 2,048
Metadata Nodes 48
Metadata Storage Devices 768
Data Nodes 0
Data Storage Devices 0

METADATA

Easy Write 22.62 kIOP/s
Easy Stat 1,667.46 kIOP/s
Easy Delete 28.24 kIOP/s
Hard Write 15.67 kIOP/s
Hard Read 1,320.59 kIOP/s
Hard Stat 1,659.24 kIOP/s
Hard Delete 18.38 kIOP/s

Submitted Files

ior-easy-read
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Thu Jul  9 00:27:59 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/stonewall -t 512k -b 9920000m -r -R -a POSIX --posix.odirect
Machine             : Linux node0026
TestID              : 0
StartTime           : Thu Jul  9 00:27:59 2020
Path                : /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy
FS                  : 18679.1 TiB   Used FS: 41.1%   Inodes: 3993.6 Mi   Used Inodes: 12.3%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/ior_file_easy
access              : single-shared-file
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 2048
tasks               : 2048
clients per node    : 1
repetitions         : 1
xfersize            : 524288 bytes
blocksize           : 9.46 TiB
aggregate filesize  : 19375 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
WARNING: Expected aggregate file size       = 21303037788160000.
WARNING: Stat() of aggregate file size      = 21292808373010432.
WARNING: Using actual aggregate bytes moved = 353195561844736.
read      707361     1414794    0.000024    10158080000 512.00     0.024373   476.16     0.016056   476.18     0   
Max Read:  707360.57 MiB/sec (741721.31 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
read       707360.57  707360.57  707360.57       0.00 1414721.13 1414721.13 1414721.13       0.00  476.18365         NA            NA     0   2048   1    1   0     1        1         0    0      1 10401873920000   524288 336833536.0 POSIX      0
Finished            : Thu Jul  9 00:35:56 2020
ior-easy-write
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Thu Jul  9 00:01:06 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/stonewall -t 512k -b 9920000m -w -D 300 -O stoneWallingWearOut=1 -a POSIX --posix.odirect
Machine             : Linux node0026
TestID              : 0
StartTime           : Thu Jul  9 00:01:06 2020
Path                : /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy
FS                  : 18679.1 TiB   Used FS: 41.1%   Inodes: 3993.6 Mi   Used Inodes: 12.1%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/ior_file_easy
access              : single-shared-file
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 2048
tasks               : 2048
clients per node    : 1
repetitions         : 1
xfersize            : 524288 bytes
blocksize           : 9.46 TiB
aggregate filesize  : 19375 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 187995 max: 328939 -- min data: 91.8 GiB mean data: 118.2 GiB time: 300.4s
WARNING: Expected aggregate file size       = 21303037788160000.
WARNING: Stat() of aggregate file size      = 21292808373010432.
WARNING: Using actual aggregate bytes moved = 353195561844736.
WARNING: maybe caused by deadlineForStonewalling
write     806622     1613345    0.000017    10158080000 512.00     0.024491   417.56     0.003990   417.59     0   
Max Write: 806621.75 MiB/sec (845804.21 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
write      806621.75  806621.75  806621.75       0.00 1613243.50 1613243.50 1613243.50       0.00  417.58549     300.40     824842.43     0   2048   1    1   0     1        1         0    0      1 10401873920000   524288 336833536.0 POSIX      0
Finished            : Thu Jul  9 00:08:04 2020
ior-hard-read
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Thu Jul  9 00:36:00 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/file -O stoneWallingStatusFile=/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 4800000 -r -R -a POSIX --posix.odirect
Machine             : Linux node0026
TestID              : 0
StartTime           : Thu Jul  9 00:36:00 2020
Path                : /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard
FS                  : 18679.1 TiB   Used FS: 41.1%   Inodes: 3993.6 Mi   Used Inodes: 12.2%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 4800000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 2048
tasks               : 2048
clients per node    : 1
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 420.28 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 3005027 max: 3005027 -- min data: 131.6 GiB mean data: 131.6 GiB time: 1535.7s
WARNING: Expected aggregate file size       = 462107443200000.
WARNING: Stat() of aggregate file size      = 289301113274368.
WARNING: Using actual aggregate bytes moved = 289301113274368.
read      179605     4006473    1536.07     45.91      45.91      0.027609   1536.09    0.060491   1536.14    0   
Max Read:  179605.40 MiB/sec (188329.92 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
read       179605.40  179605.40  179605.40       0.00 4006337.57 4006337.57 4006337.57       0.00 1536.13998    1535.66     179662.01     0   2048   1    1   0     1        1         0    0 4800000    47008    47008 275899040.0 POSIX      0
Finished            : Thu Jul  9 01:01:37 2020
ior-hard-write
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Thu Jul  9 00:14:43 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/file -O stoneWallingStatusFile=/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 4800000 -w -D 300 -a POSIX --posix.odirect
Machine             : Linux node0026
TestID              : 0
StartTime           : Thu Jul  9 00:14:43 2020
Path                : /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard
FS                  : 18679.1 TiB   Used FS: 41.1%   Inodes: 3993.6 Mi   Used Inodes: 12.2%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 4800000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 2048
tasks               : 2048
clients per node    : 1
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 420.28 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 2065597 max: 3005027 -- min data: 90.4 GiB mean data: 115.3 GiB time: 300.2s
WARNING: Expected aggregate file size       = 462107443200000.
WARNING: Stat() of aggregate file size      = 289301113274368.
WARNING: Using actual aggregate bytes moved = 289301113274368.
WARNING: maybe caused by deadlineForStonewalling
write     758186     16929052   319.06      45.91      45.91      0.359954   363.53     0.007313   363.89     0   
Max Write: 758185.91 MiB/sec (795015.55 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
write      758185.91  758185.91  758185.91       0.00 16912345.77 16912345.77 16912345.77       0.00  363.89365     300.21     805217.92     0   2048   1    1   0     1        1         0    0 4800000    47008    47008 275899040.0 POSIX      0
Finished            : Thu Jul  9 00:20:47 2020
mdtest-easy-delete
-- started at 07/09/2020 01:01:40 --

mdtest-3.3.0+dev was launched with 2048 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '250000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy' '-x' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy-stonewall' '-r' '-a' 'POSIX' '--posix.odirect'
Path: /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app
FS: 18679.1 TiB   Used FS: 41.1%   Inodes: 3993.6 Mi   Used Inodes: 12.2%

Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 1 for each phase.
2048 tasks, 512000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :      28238.310      28238.047      28238.170          0.048
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          1.107          1.107          1.107          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        276.107        276.105        276.106          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.903          0.903          0.903          0.000
-- finished at 07/09/2020 01:06:17 --

mdtest-easy-stat
-- started at 07/09/2020 00:35:56 --

mdtest-3.3.0+dev was launched with 2048 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '250000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy' '-x' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy-stonewall' '-T' '-a' 'POSIX' '--posix.odirect'
Path: /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app
FS: 18679.1 TiB   Used FS: 41.1%   Inodes: 3993.6 Mi   Used Inodes: 12.2%

Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 1 for each phase.
2048 tasks, 512000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :    1667478.808    1667442.656    1667459.547          4.810
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          4.676          4.676          4.676          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/09/2020 00:36:00 --

mdtest-easy-write
-- started at 07/09/2020 00:08:04 --

mdtest-3.3.0+dev was launched with 2048 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '250000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy' '-x' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy-stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX' '--posix.odirect'
Path: /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app
FS: 18679.1 TiB   Used FS: 41.1%   Inodes: 3993.6 Mi   Used Inodes: 12.0%

Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 1 for each phase.
2048 tasks, 512000000 files
Continue stonewall hit min: 3210 max: 3807 avg: 3477.7 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :      22622.252      22622.044      22622.140          0.038
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA      23689.373             NA
   Tree creation             :         17.977         17.977         17.977          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        344.652        344.649        344.651          0.001
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.659             NA
   Tree creation             :          0.056          0.056          0.056          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/09/2020 00:14:43 --

mdtest-hard-delete
-- started at 07/09/2020 01:06:21 --

mdtest-3.3.0+dev was launched with 2048 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '20000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard' '-x' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard-stonewall' '-r' '-a' 'POSIX' '--posix.odirect'
Path: /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app
FS: 18679.1 TiB   Used FS: 41.1%   Inodes: 3993.6 Mi   Used Inodes: 12.0%

Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 1 for each phase.
2048 tasks, 40960000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :      18377.692      18377.523      18377.601          0.031
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          1.712          1.712          1.712          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        297.435        297.432        297.433          0.001
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.584          0.584          0.584          0.000
-- finished at 07/09/2020 01:11:20 --

mdtest-hard-read
-- started at 07/09/2020 01:06:17 --

mdtest-3.3.0+dev was launched with 2048 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '20000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard' '-x' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard-stonewall' '-E' '-X' '-a' 'POSIX' '--posix.odirect'
Path: /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app
FS: 18679.1 TiB   Used FS: 41.1%   Inodes: 3993.6 Mi   Used Inodes: 12.0%

Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 1 for each phase.
2048 tasks, 40960000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :    1320610.907    1320585.418    1320598.426          4.431
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          4.139          4.139          4.139          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/09/2020 01:06:21 --

mdtest-hard-stat
-- started at 07/09/2020 01:01:37 --

mdtest-3.3.0+dev was launched with 2048 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '20000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard' '-x' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard-stonewall' '-T' '-a' 'POSIX' '--posix.odirect'
Path: /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app
FS: 18679.1 TiB   Used FS: 41.1%   Inodes: 3993.6 Mi   Used Inodes: 12.2%

Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 1 for each phase.
2048 tasks, 40960000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :    1659266.805    1659221.029    1659243.584          8.225
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          3.294          3.294          3.294          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/09/2020 01:01:40 --

mdtest-hard-write
-- started at 07/09/2020 00:20:47 --

mdtest-3.3.0+dev was launched with 2048 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '20000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard' '-x' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard-stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX' '--posix.odirect'
Path: /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app
FS: 18679.1 TiB   Used FS: 41.1%   Inodes: 3993.6 Mi   Used Inodes: 12.2%

Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 1 for each phase.
2048 tasks, 40960000 files
Continue stonewall hit min: 2213 max: 2669 avg: 2433.1 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :      15667.155      15667.011      15667.077          0.027
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA      16589.971             NA
   Tree creation             :        452.017        452.017        452.017          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        348.893        348.890        348.892          0.001
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.366             NA
   Tree creation             :          0.002          0.002          0.002          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/09/2020 00:27:32 --

result
version         = io500-isc20_v4-2-g20efd247106c
config-hash     = 14F6BBD8
result-dir      = /scratch/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/results/2020.07.08-23.59.24-app
; START 2020-07-09 00:01:06


[ior-easy-write]
t_start         = 2020-07-09 00:01:06
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/stonewall -t 512k -b 9920000m -w -D 300 -O stoneWallingWearOut=1 -a POSIX --posix.odirect
throughput-stonewall = 1186728.23
score           = 787.716552
t_delta         = 417.7037
t_end           = 2020-07-09 00:08:04

[mdtest-easy-write]
t_start         = 2020-07-09 00:08:04
exe             = ./mdtest -n 250000 -u -L -F -P -N 1 -d /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy -x /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy-stonewall -C -Y -W 300 -a POSIX --posix.odirect
rate-stonewall  = 23.689373
score           = 22.622094
t_delta         = 398.6679
t_end           = 2020-07-09 00:14:43

[timestamp]
t_start         = 2020-07-09 00:14:43
t_delta         = 0.0978
t_end           = 2020-07-09 00:14:43

[ior-hard-write]
t_start         = 2020-07-09 00:14:43
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/file -O stoneWallingStatusFile=/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 4800000 -w -D 300 -a POSIX --posix.odirect
throughput-stonewall = 1328595.19
score           = 740.415929
t_delta         = 364.0428
t_end           = 2020-07-09 00:20:47

[mdtest-hard-write]
t_start         = 2020-07-09 00:20:47
exe             = ./mdtest -n 20000 -t -w 3901 -e 3901 -P -N 1 -F -d /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard -x /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard-stonewall -C -Y -W 300 -a POSIX --posix.odirect
rate-stonewall  = 16.589971
score           = 15.667045
t_delta         = 405.2319
t_end           = 2020-07-09 00:27:32

[find]
t_start         = 2020-07-09 00:27:32
exe             = ./pfind /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app -newer /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/timestampfile -size 3901c -name *01* -C -q 10000
found           = 199128
total-files     = 13262855
score           = 489.039063
t_delta         = 27.3482
t_end           = 2020-07-09 00:27:59

[ior-easy-read]
t_start         = 2020-07-09 00:27:59
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/stonewall -t 512k -b 9920000m -r -R -a POSIX --posix.odirect
score           = 690.781803
t_delta         = 476.3229
t_end           = 2020-07-09 00:35:56

[mdtest-easy-stat]
t_start         = 2020-07-09 00:35:56
exe             = ./mdtest -n 250000 -u -L -F -P -N 1 -d /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy -x /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy-stonewall -T -a POSIX --posix.odirect
score           = 1667.462554
t_delta         = 4.7633
t_end           = 2020-07-09 00:36:00

[ior-hard-read]
t_start         = 2020-07-09 00:36:00
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/file -O stoneWallingStatusFile=/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 4800000 -r -R -a POSIX --posix.odirect
score           = 175.395903
t_delta         = 1536.2529
t_end           = 2020-07-09 01:01:37

[mdtest-hard-stat]
t_start         = 2020-07-09 01:01:37
exe             = ./mdtest -n 20000 -t -w 3901 -e 3901 -P -N 1 -F -d /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard -x /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard-stonewall -T -a POSIX --posix.odirect
score           = 1659.243152
t_delta         = 3.3771
t_end           = 2020-07-09 01:01:40

[mdtest-easy-delete]
t_start         = 2020-07-09 01:01:40
exe             = ./mdtest -n 250000 -u -L -F -P -N 1 -d /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy -x /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy-stonewall -r -a POSIX --posix.odirect
score           = 28.238112
t_delta         = 277.1142
t_end           = 2020-07-09 01:06:17

[mdtest-hard-read]
t_start         = 2020-07-09 01:06:17
exe             = ./mdtest -n 20000 -t -w 3901 -e 3901 -P -N 1 -F -d /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard -x /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard-stonewall -E -X -a POSIX --posix.odirect
score           = 1320.590582
t_delta         = 4.2257
t_end           = 2020-07-09 01:06:21

[mdtest-hard-delete]
t_start         = 2020-07-09 01:06:21
exe             = ./mdtest -n 20000 -t -w 3901 -e 3901 -P -N 1 -F -d /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard -x /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard-stonewall -r -a POSIX --posix.odirect
score           = 18.377565
t_delta         = 298.2373
t_end           = 2020-07-09 01:11:20

[SCORE]
MD              = 154.735576
BW              = 515.586473
SCORE           = 282.452774 
hash            = 237BEA02
; END 2020-07-09 01:11:21
result_summary
IO500 version io500-isc20_v4-2-g20efd247106c
[RESULT]       ior-easy-write      787.716552 GiB/s  : time 417.704 seconds
[RESULT]    mdtest-easy-write       22.622094 kIOPS : time 398.668 seconds
[RESULT]       ior-hard-write      740.415929 GiB/s  : time 364.043 seconds
[RESULT]    mdtest-hard-write       15.667045 kIOPS : time 405.232 seconds
[RESULT]                 find      489.039063 kIOPS : time 27.348 seconds
[RESULT]        ior-easy-read      690.781803 GiB/s  : time 476.323 seconds
[RESULT]     mdtest-easy-stat     1667.462554 kIOPS : time 4.763 seconds
[RESULT]        ior-hard-read      175.395903 GiB/s  : time 1536.253 seconds
[RESULT]     mdtest-hard-stat     1659.243152 kIOPS : time 3.377 seconds
[RESULT]   mdtest-easy-delete       28.238112 kIOPS : time 277.114 seconds
[RESULT]     mdtest-hard-read     1320.590582 kIOPS : time 4.226 seconds
[RESULT]   mdtest-hard-delete       18.377565 kIOPS : time 298.237 seconds
[SCORE] Bandwidth 515.586473 GB/s : IOPS 154.735576 kiops : TOTAL 282.452774