Govorun

Institution JINR
Client Procs Per Node
Client Operating System CentOS
Client Operating System Version 7.7.1908
Client Kernel Version 3.10.0-1062.1.1.el7.x86_64

DATA SERVER

Storage Type NVMe
Volatile Memory 187GB
Storage Interface NVMe over Fabric
Network OPA
Software Version 2.13.0
OS Version 7.7.1908

INFORMATION

Client Nodes 50
Client Total Procs 800
Metadata Nodes 12
Metadata Storage Devices 2
Data Nodes 12
Data Storage Devices 2

METADATA

Easy Write 471.97 kIOP/s
Easy Stat 816.14 kIOP/s
Easy Delete 354.22 kIOP/s
Hard Write 72.41 kIOP/s
Hard Read 170.79 kIOP/s
Hard Stat 205.48 kIOP/s
Hard Delete 66.86 kIOP/s

Submitted Files

ior-easy-read
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Jul  7 16:55:46 2020
Command line        : ./ior -v -C -Q 1 -g -G 271 -k -e -o /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/stonewall -t 2m -b 9920000m -F -r -R -a POSIX --posix.odirect
Machine             : Linux n02p001.gvr.local
Start time skew across all tasks: 260452.10 sec
TestID              : 0
StartTime           : Tue Jul  7 16:55:46 2020
Path                : /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy
FS                  : 259.3 TiB   Used FS: 55.1%   Inodes: 1202.1 Mi   Used Inodes: 17.8%
Participating tasks: 800
Using reorderTasks '-C' (useful to avoid read cache in client)

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 50
tasks               : 800
clients per node    : 16
repetitions         : 1
xfersize            : 2 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 7568.36 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
Commencing read performance test: Tue Jul  7 16:55:48 2020

WARNING: Expected aggregate file size       = 8321499136000000.
WARNING: Stat() of aggregate file size      = 79440117760000.
WARNING: Using actual aggregate bytes moved = 79440117760000.
read      82525      41300      0.000185    10158080000 2048.00    0.853655   917.20     0.754555   918.03     0   
Max Read:  82524.93 MiB/sec (86533.66 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
read        82524.93   82524.93   82524.93       0.00   41262.46   41262.46   41262.46       0.00  918.02565         NA            NA     0    800  16    1   1     1        1         0    0      1 10401873920000  2097152 75760000.0 POSIX      0
Finished            : Tue Jul  7 17:11:07 2020
ior-easy-write
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Jul  7 16:09:46 2020
Command line        : ./ior -v -C -Q 1 -g -G 271 -k -e -o /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/stonewall -t 2m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX --posix.odirect
Machine             : Linux n02p001.gvr.local
Start time skew across all tasks: 260452.01 sec
TestID              : 0
StartTime           : Tue Jul  7 16:09:46 2020
Path                : /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy
FS                  : 259.3 TiB   Used FS: 25.7%   Inodes: 1047.6 Mi   Used Inodes: 2.7%
Participating tasks: 800
Using reorderTasks '-C' (useful to avoid read cache in client)

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 50
tasks               : 800
clients per node    : 16
repetitions         : 1
xfersize            : 2 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 7568.36 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
Commencing write performance test: Tue Jul  7 16:09:48 2020
0: stonewalling pairs accessed: 41748
stonewalling pairs accessed min: 21781 max: 47350 -- min data: 42.5 GiB mean data: 69.1 GiB time: 300.5s
WARNING: Expected aggregate file size       = 8321499136000000.
WARNING: Stat() of aggregate file size      = 79440117760000.
WARNING: Using actual aggregate bytes moved = 79440117760000.
WARNING: maybe caused by deadlineForStonewalling
write     139879     70049      0.000068    10158080000 2048.00    0.491230   540.76     0.438729   541.61     0   
Max Write: 139879.45 MiB/sec (146674.24 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
write      139879.45  139879.45  139879.45       0.00   69939.73   69939.73   69939.73       0.00  541.60921     300.53     188461.65     0    800  16    1   1     1        1         0    0      1 10401873920000  2097152 75760000.0 POSIX      0
Finished            : Tue Jul  7 16:18:51 2020
ior-hard-read
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Jul  7 17:14:51 2020
Command line        : ./ior -v -C -Q 1 -g -G 27 -k -e -o /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/file -O stoneWallingStatusFile=/lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 110000 -r -R -a MPIIO
Machine             : Linux n02p001.gvr.local
Start time skew across all tasks: 260452.29 sec
TestID              : 0
StartTime           : Tue Jul  7 17:14:51 2020
Path                : /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard
FS                  : 259.3 TiB   Used FS: 55.1%   Inodes: 1202.1 Mi   Used Inodes: 17.8%
Participating tasks: 800
Using reorderTasks '-C' (useful to avoid read cache in client)

Options: 
api                 : MPIIO
apiVersion          : (3.1)
test filename       : /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 110000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 50
tasks               : 800
clients per node    : 16
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 3.76 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
Commencing read performance test: Tue Jul  7 17:14:58 2020

0: stonewalling pairs accessed: 110000
stonewalling pairs accessed min: 110000 max: 110000 -- min data: 4.8 GiB mean data: 4.8 GiB time: 260.3s
read      14740      336201     260.84      45.91      45.91      5.78       261.75     1.21       267.65     0   
Max Read:  14739.58 MiB/sec (15455.57 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
read        14739.58   14739.58   14739.58       0.00  328785.95  328785.95  328785.95       0.00  267.65134     260.28      15156.91     0    800  16    1   0     1        1         0    0 110000    47008    47008 3945068.2 MPIIO      0
Finished            : Tue Jul  7 17:19:27 2020
ior-hard-write
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Tue Jul  7 16:25:20 2020
Command line        : ./ior -v -C -Q 1 -g -G 27 -k -e -o /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/file -O stoneWallingStatusFile=/lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 110000 -w -D 300 -a MPIIO
Machine             : Linux n02p001.gvr.local
Start time skew across all tasks: 260451.91 sec
TestID              : 0
StartTime           : Tue Jul  7 16:25:20 2020
Path                : /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard
FS                  : 259.3 TiB   Used FS: 53.6%   Inodes: 1202.1 Mi   Used Inodes: 15.2%
Participating tasks: 800
Using reorderTasks '-C' (useful to avoid read cache in client)

Options: 
api                 : MPIIO
apiVersion          : (3.1)
test filename       : /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 110000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 50
tasks               : 800
clients per node    : 16
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 3.76 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
Commencing write performance test: Tue Jul  7 16:26:20 2020
0: stonewalling pairs accessed: 110000
stonewalling pairs accessed min: 110000 max: 110000 -- min data: 4.8 GiB mean data: 4.8 GiB time: 292.8s
write     11190      299428     293.43      45.91      45.91      58.30      293.89     1.13       352.56     0   
Max Write: 11189.74 MiB/sec (11733.29 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
write       11189.74   11189.74   11189.74       0.00  249602.07  249602.07  249602.07       0.00  352.56118     292.76      13475.35     0    800  16    1   0     1        1         0    0 110000    47008    47008 3945068.2 MPIIO      0
Finished            : Tue Jul  7 16:31:23 2020
mdtest-easy-delete
-- started at 07/07/2020 17:22:27 --

mdtest-3.3.0+dev was launched with 800 total task(s) on 50 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy' '-x' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy-stonewall' '-r' '-a' 'POSIX'
Path: /lustre/stor1/io500/2020.07.07-16.09.21-app
FS: 259.3 TiB   Used FS: 55.1%   Inodes: 1202.1 Mi   Used Inodes: 17.8%

Nodemap: 11111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 16 for each phase.
800 tasks, 800000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :     354373.813     354064.083     354208.387         55.782
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.781          0.781          0.781          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        457.260        456.860        457.073          0.072
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          1.281          1.281          1.281          0.000
-- finished at 07/07/2020 17:30:52 --

mdtest-easy-stat
-- started at 07/07/2020 17:11:17 --

mdtest-3.3.0+dev was launched with 800 total task(s) on 50 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy' '-x' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy-stonewall' '-T' '-a' 'POSIX'
Path: /lustre/stor1/io500/2020.07.07-16.09.21-app
FS: 259.3 TiB   Used FS: 55.1%   Inodes: 1202.1 Mi   Used Inodes: 17.8%

Nodemap: 11111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 16 for each phase.
800 tasks, 800000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :     816836.658     815367.031     816070.154        278.066
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :        198.560        198.203        198.389          0.068
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/07/2020 17:14:41 --

mdtest-easy-write
-- started at 07/07/2020 16:19:02 --

mdtest-3.3.0+dev was launched with 800 total task(s) on 50 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy' '-x' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy-stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /lustre/stor1/io500/2020.07.07-16.09.21-app
FS: 259.3 TiB   Used FS: 53.6%   Inodes: 1047.6 Mi   Used Inodes: 2.7%

Nodemap: 11111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 16 for each phase.
800 tasks, 800000000 files
Continue stonewall hit min: 181525 max: 202374 avg: 192275.9 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :     472273.244     469846.052     472016.626        225.831
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA     515556.233             NA
   Tree creation             :          1.083          1.083          1.083          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        344.579        342.808        342.995          0.165
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        298.359             NA
   Tree creation             :          0.923          0.923          0.923          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/07/2020 16:25:08 --

mdtest-hard-delete
-- started at 07/07/2020 17:34:25 --

mdtest-3.3.0+dev was launched with 800 total task(s) on 50 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard' '-x' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard-stonewall' '-r' '-a' 'POSIX'
Path: /lustre/stor1/io500/2020.07.07-16.09.21-app
FS: 259.3 TiB   Used FS: 55.1%   Inodes: 1047.6 Mi   Used Inodes: 5.7%

Nodemap: 11111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 16 for each phase.
800 tasks, 800000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :      66885.062      66837.808      66859.725          7.889
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.108          0.108          0.108          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        480.949        480.610        480.792          0.057
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          9.289          9.289          9.289          0.000
-- finished at 07/07/2020 17:42:40 --

mdtest-hard-read
-- started at 07/07/2020 17:31:02 --

mdtest-3.3.0+dev was launched with 800 total task(s) on 50 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard' '-x' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard-stonewall' '-E' '-X' '-a' 'POSIX'
Path: /lustre/stor1/io500/2020.07.07-16.09.21-app
FS: 259.3 TiB   Used FS: 55.1%   Inodes: 1047.6 Mi   Used Inodes: 5.7%

Nodemap: 11111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 16 for each phase.
800 tasks, 800000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :     170876.264     170600.992     170756.194         54.466
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :        188.426        188.122        188.254          0.060
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/07/2020 17:34:15 --

mdtest-hard-stat
-- started at 07/07/2020 17:19:36 --

mdtest-3.3.0+dev was launched with 800 total task(s) on 50 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard' '-x' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard-stonewall' '-T' '-a' 'POSIX'
Path: /lustre/stor1/io500/2020.07.07-16.09.21-app
FS: 259.3 TiB   Used FS: 55.1%   Inodes: 1202.1 Mi   Used Inodes: 17.8%

Nodemap: 11111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 16 for each phase.
800 tasks, 800000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :     205554.347     205015.935     205365.077         92.612
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :        156.796        156.385        156.529          0.071
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/07/2020 17:22:18 --

mdtest-hard-write
-- started at 07/07/2020 16:31:34 --

mdtest-3.3.0+dev was launched with 800 total task(s) on 50 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard' '-x' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard-stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /lustre/stor1/io500/2020.07.07-16.09.21-app
FS: 259.3 TiB   Used FS: 55.0%   Inodes: 1202.1 Mi   Used Inodes: 15.2%

Nodemap: 11111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 16 for each phase.
800 tasks, 800000000 files
Continue stonewall hit min: 37381 max: 40182 avg: 38548.1 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :      72431.256      72383.478      72409.336         11.832
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA     103319.515             NA
   Tree creation             :          1.332          1.332          1.332          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        444.101        443.808        443.943          0.073
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        298.476             NA
   Tree creation             :          0.751          0.751          0.751          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/07/2020 16:39:44 --

result
version         = io500-isc20_v4-1-g18c6d03b40b0
config-hash     = 9141A89C
result-dir      = ./results/2020.07.07-16.09.21-app
; START 2020-07-07 16:09:32


[ior-easy-write]
t_start         = 2020-07-07 16:09:33
exe             = ./ior -v -C -Q 1 -g -G 271 -k -e -o /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/stonewall -t 2m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX --posix.odirect
throughput-stonewall = 81697.76
score           = 136.601028
t_delta         = 558.3757
t_end           = 2020-07-07 16:18:51

[mdtest-easy-write]
t_start         = 2020-07-07 16:18:51
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy -x /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy-stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 515.556233
score           = 471.965676
t_delta         = 377.0627
t_end           = 2020-07-07 16:25:08

[timestamp]
t_start         = 2020-07-07 16:25:09
t_delta         = 0.0180
t_end           = 2020-07-07 16:25:09

[ior-hard-write]
t_start         = 2020-07-07 16:25:09
exe             = ./ior -v -C -Q 1 -g -G 27 -k -e -o /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/file -O stoneWallingStatusFile=/lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 110000 -w -D 300 -a MPIIO
throughput-stonewall = 8741.99
score           = 10.927482
t_delta         = 373.6386
t_end           = 2020-07-07 16:31:23

[mdtest-hard-write]
t_start         = 2020-07-07 16:31:23
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard -x /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard-stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 103.319515
score           = 72.405949
t_delta         = 500.7195
t_end           = 2020-07-07 16:39:44

[find]
t_start         = 2020-07-07 16:39:44
exe             = ./pfind /lustre/stor1/io500/2020.07.07-16.09.21-app -newer /lustre/stor1/io500/2020.07.07-16.09.21-app/timestampfile -size 3901c -name *01* -C -q 10000
found           = 1216221
total-files     = 194045606
score           = 204.177825
t_delta         = 951.0903
t_end           = 2020-07-07 16:55:35

[ior-easy-read]
t_start         = 2020-07-07 16:55:35
exe             = ./ior -v -C -Q 1 -g -G 271 -k -e -o /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/stonewall -t 2m -b 9920000m -F -r -R -a POSIX --posix.odirect
score           = 80.590750
t_delta         = 931.6662
t_end           = 2020-07-07 17:11:07

[mdtest-easy-stat]
t_start         = 2020-07-07 17:11:07
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy -x /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy-stonewall -T -a POSIX
score           = 816.135984
t_delta         = 213.3078
t_end           = 2020-07-07 17:14:41

[ior-hard-read]
t_start         = 2020-07-07 17:14:41
exe             = ./ior -v -C -Q 1 -g -G 27 -k -e -o /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/file -O stoneWallingStatusFile=/lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 110000 -r -R -a MPIIO
score           = 14.394121
t_delta         = 285.9153
t_end           = 2020-07-07 17:19:27

[mdtest-hard-stat]
t_start         = 2020-07-07 17:19:27
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard -x /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard-stonewall -T -a POSIX
score           = 205.475790
t_delta         = 171.1807
t_end           = 2020-07-07 17:22:18

[mdtest-easy-delete]
t_start         = 2020-07-07 17:22:18
exe             = ./mdtest -n 1000000 -u -L -F -P -N 1 -d /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy -x /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy-stonewall -r -a POSIX
score           = 354.219982
t_delta         = 514.1532
t_end           = 2020-07-07 17:30:52

[mdtest-hard-read]
t_start         = 2020-07-07 17:30:52
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard -x /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard-stonewall -E -X -a POSIX
score           = 170.793932
t_delta         = 202.5748
t_end           = 2020-07-07 17:34:15

[mdtest-hard-delete]
t_start         = 2020-07-07 17:34:15
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard -x /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard-stonewall -r -a POSIX
score           = 66.860460
t_delta         = 504.8564
t_end           = 2020-07-07 17:42:40

[SCORE]
MD              = 215.968997
BW              = 36.275307
SCORE           = 88.511817 
hash            = 1623C448
; END 2020-07-07 17:42:44
result_summary
IO500 version io500-isc20_v4-1-g18c6d03b40b0
[RESULT]       ior-easy-write      136.601028 GiB/s  : time 558.376 seconds
[RESULT]    mdtest-easy-write      471.965676 kIOPS : time 377.063 seconds
[RESULT]       ior-hard-write       10.927482 GiB/s  : time 373.639 seconds
[RESULT]    mdtest-hard-write       72.405949 kIOPS : time 500.719 seconds
[RESULT]                 find      204.177825 kIOPS : time 951.090 seconds
[RESULT]        ior-easy-read       80.590750 GiB/s  : time 931.666 seconds
[RESULT]     mdtest-easy-stat      816.135984 kIOPS : time 213.308 seconds
[RESULT]        ior-hard-read       14.394121 GiB/s  : time 285.915 seconds
[RESULT]     mdtest-hard-stat      205.475790 kIOPS : time 171.181 seconds
[RESULT]   mdtest-easy-delete      354.219982 kIOPS : time 514.153 seconds
[RESULT]     mdtest-hard-read      170.793932 kIOPS : time 202.575 seconds
[RESULT]   mdtest-hard-delete       66.860460 kIOPS : time 504.856 seconds
[SCORE] Bandwidth 36.275307 GB/s : IOPS 215.968997 kiops : TOTAL 88.511817