Venus2

Institution National Supercomputer Center in Guangzhou
Client Procs Per Node
Client Operating System CentOS
Client Operating System Version 7.3
Client Kernel Version 3.10.0-514.el7.x86_64

DATA SERVER

Storage Type NVMe
Volatile Memory 64G
Storage Interface NVMe
Network Ethernet
Software Version 1.0
OS Version 7.3

INFORMATION

Client Nodes 10
Client Total Procs 260
Metadata Nodes 20
Metadata Storage Devices 1
Data Nodes 38
Data Storage Devices 38

METADATA

Easy Write 277.09 kIOP/s
Easy Stat 1,355.07 kIOP/s
Easy Delete 647.30 kIOP/s
Hard Write 209.10 kIOP/s
Hard Read 438.04 kIOP/s
Hard Stat 1,245.93 kIOP/s
Hard Delete 532.29 kIOP/s

Submitted Files

ior-easy-read
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Fri Jul 10 17:43:09 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o /mnt/llfs//2020.07.10-17.14.40-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/mnt/llfs//2020.07.10-17.14.40-app/ior-easy/stonewall -t 1m -b 59392m -F -r -R -a POSIX --posix.odirect
Machine             : Linux cn16609-gn0
TestID              : 0
StartTime           : Fri Jul 10 17:43:09 2020
Path                : /mnt/llfs/2020.07.10-17.14.40-app/ior-easy
FS                  : 51.1 TiB   Used FS: 28.6%   Inodes: 0.0 Mi   Used Inodes: -nan%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /mnt/llfs//2020.07.10-17.14.40-app/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 260
clients per node    : 26
repetitions         : 1
xfersize            : 1 MiB
blocksize           : 58 GiB
aggregate filesize  : 14.73 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
WARNING: Expected aggregate file size       = 16192026705920.
WARNING: Stat() of aggregate file size      = 0.
WARNING: Using actual aggregate bytes moved = 12760435916800.
read      65641      65747      0.003116    60817408   1024.00    0.066083   185.09     0.232194   185.39     0   
Max Read:  65641.49 MiB/sec (68830.09 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
read        65641.49   65641.49   65641.49       0.00   65641.49   65641.49   65641.49       0.00  185.39038         NA            NA     0    260  26    1   1     1        1         0    0      1 62277025792  1048576 12169300.0 POSIX      0
Finished            : Fri Jul 10 17:46:14 2020
ior-easy-write
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Fri Jul 10 17:14:50 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o /mnt/llfs//2020.07.10-17.14.40-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/mnt/llfs//2020.07.10-17.14.40-app/ior-easy/stonewall -t 1m -b 59392m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX --posix.odirect
Machine             : Linux cn16609-gn0
TestID              : 0
StartTime           : Fri Jul 10 17:14:50 2020
Path                : /mnt/llfs/2020.07.10-17.14.40-app/ior-easy
FS                  : 51.1 TiB   Used FS: 0.0%   Inodes: 0.0 Mi   Used Inodes: -nan%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /mnt/llfs//2020.07.10-17.14.40-app/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 260
clients per node    : 26
repetitions         : 1
xfersize            : 1 MiB
blocksize           : 58 GiB
aggregate filesize  : 14.73 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 45315 max: 46805 -- min data: 44.3 GiB mean data: 45.0 GiB time: 300.2s
WARNING: Expected aggregate file size       = 16192026705920.
WARNING: Stat() of aggregate file size      = 0.
WARNING: Using actual aggregate bytes moved = 12760435916800.
WARNING: maybe caused by deadlineForStonewalling
write     39634      39638      0.005103    60817408   1024.00    0.028299   307.01     0.007316   307.04     0   
Max Write: 39633.68 MiB/sec (41558.92 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
write       39633.68   39633.68   39633.68       0.00   39633.68   39633.68   39633.68       0.00  307.04442     300.22      39936.50     0    260  26    1   1     1        1         0    0      1 62277025792  1048576 12169300.0 POSIX      0
Finished            : Fri Jul 10 17:19:58 2020
ior-hard-read
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Fri Jul 10 17:48:30 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o /mnt/llfs//2020.07.10-17.14.40-app/ior-hard/file -O stoneWallingStatusFile=/mnt/llfs//2020.07.10-17.14.40-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 420000 -r -R -a POSIX
Machine             : Linux cn16609-gn0
TestID              : 0
StartTime           : Fri Jul 10 17:48:30 2020
Path                : /mnt/llfs/2020.07.10-17.14.40-app/ior-hard
FS                  : 51.1 TiB   Used FS: 28.6%   Inodes: 0.0 Mi   Used Inodes: -nan%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /mnt/llfs//2020.07.10-17.14.40-app/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 420000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 260
clients per node    : 26
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 4.67 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 269575 max: 269575 -- min data: 11.8 GiB mean data: 11.8 GiB time: 85.3s
WARNING: Expected aggregate file size       = 5133273600000.
WARNING: Stat() of aggregate file size      = 3294767216000.
WARNING: Using actual aggregate bytes moved = 3294767216000.
read      36583      821413     85.31       45.91      45.91      0.512897   85.33      0.052060   85.89      0   
Max Read:  36582.60 MiB/sec (38359.63 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
read        36582.60   36582.60   36582.60       0.00  816023.54  816023.54  816023.54       0.00   85.89152      85.30      36836.91     0    260  26    1   0     1        1         0    0 420000    47008    47008 3142134.8 POSIX      0
Finished            : Fri Jul 10 17:49:56 2020
ior-hard-write
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Fri Jul 10 17:29:13 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o /mnt/llfs//2020.07.10-17.14.40-app/ior-hard/file -O stoneWallingStatusFile=/mnt/llfs//2020.07.10-17.14.40-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 420000 -w -D 300 -a POSIX
Machine             : Linux cn16609-gn0
TestID              : 0
StartTime           : Fri Jul 10 17:29:13 2020
Path                : /mnt/llfs/2020.07.10-17.14.40-app/ior-hard
FS                  : 51.1 TiB   Used FS: 22.7%   Inodes: 0.0 Mi   Used Inodes: -nan%

Options: 
api                 : POSIX
apiVersion          : 
test filename       : /mnt/llfs//2020.07.10-17.14.40-app/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 420000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 260
clients per node    : 26
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 4.67 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 260078 max: 269575 -- min data: 11.4 GiB mean data: 11.6 GiB time: 300.1s
WARNING: Expected aggregate file size       = 5133273600000.
WARNING: Stat() of aggregate file size      = 0.
WARNING: Using actual aggregate bytes moved = 3294767216000.
WARNING: maybe caused by deadlineForStonewalling
write     10155      226825     306.45      45.91      45.91      0.403566   309.00     0.001162   309.41     0   
Max Write: 10155.36 MiB/sec (10648.66 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
write       10155.36   10155.36   10155.36       0.00  226528.74  226528.74  226528.74       0.00  309.40666     300.11      10275.77     0    260  26    1   0     1        1         0    0 420000    47008    47008 3142134.8 POSIX      0
Finished            : Fri Jul 10 17:34:23 2020
mdtest-easy-delete
-- started at 07/10/2020 17:51:40 --

mdtest-3.3.0+dev was launched with 260 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '550000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/mnt/llfs//2020.07.10-17.14.40-app/mdtest-easy' '-x' '/mnt/llfs//2020.07.10-17.14.40-app/mdtest-easy-stonewall' '-r' '-a' 'POSIX'
Path: /mnt/llfs/2020.07.10-17.14.40-app
FS: 51.1 TiB   Used FS: 28.6%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 26 for each phase.
260 tasks, 143000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :     647296.430     647295.597     647296.057          0.229
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          3.999          3.999          3.999          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        159.294        159.293        159.293          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.250          0.250          0.250          0.000
-- finished at 07/10/2020 17:54:20 --

mdtest-easy-stat
-- started at 07/10/2020 17:46:16 --

mdtest-3.3.0+dev was launched with 260 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '550000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/mnt/llfs//2020.07.10-17.14.40-app/mdtest-easy' '-x' '/mnt/llfs//2020.07.10-17.14.40-app/mdtest-easy-stonewall' '-T' '-a' 'POSIX'
Path: /mnt/llfs/2020.07.10-17.14.40-app
FS: 51.1 TiB   Used FS: 28.6%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 26 for each phase.
260 tasks, 143000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :    1355074.555    1355072.810    1355073.918          0.371
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :         76.092         76.092         76.092          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/10/2020 17:47:32 --

mdtest-easy-write
-- started at 07/10/2020 17:20:00 --

mdtest-3.3.0+dev was launched with 260 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '550000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/mnt/llfs//2020.07.10-17.14.40-app/mdtest-easy' '-x' '/mnt/llfs//2020.07.10-17.14.40-app/mdtest-easy-stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /mnt/llfs/2020.07.10-17.14.40-app
FS: 51.1 TiB   Used FS: 22.7%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 26 for each phase.
260 tasks, 143000000 files
Continue stonewall hit min: 352548 max: 396577 avg: 368860.0 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :     277091.847     277091.502     277091.680          0.074
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA     319765.731             NA
   Tree creation             :         11.179         11.179         11.179          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        372.115        372.115        372.115          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        299.918             NA
   Tree creation             :          0.089          0.089          0.089          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/10/2020 17:28:13 --

mdtest-hard-delete
-- started at 07/10/2020 17:57:53 --

mdtest-3.3.0+dev was launched with 260 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '400000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/mnt/llfs//2020.07.10-17.14.40-app/mdtest-hard' '-x' '/mnt/llfs//2020.07.10-17.14.40-app/mdtest-hard-stonewall' '-r' '-a' 'POSIX'
Path: /mnt/llfs/2020.07.10-17.14.40-app
FS: 51.1 TiB   Used FS: 28.6%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 26 for each phase.
260 tasks, 104000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :     532290.726     532289.880     532290.187          0.186
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.160          0.160          0.160          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        137.855        137.855        137.855          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          6.257          6.257          6.257          0.000
-- finished at 07/10/2020 18:00:17 --

mdtest-hard-read
-- started at 07/10/2020 17:54:21 --

mdtest-3.3.0+dev was launched with 260 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '400000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/mnt/llfs//2020.07.10-17.14.40-app/mdtest-hard' '-x' '/mnt/llfs//2020.07.10-17.14.40-app/mdtest-hard-stonewall' '-E' '-X' '-a' 'POSIX'
Path: /mnt/llfs/2020.07.10-17.14.40-app
FS: 51.1 TiB   Used FS: 28.6%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 26 for each phase.
260 tasks, 104000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :     438037.921     438037.523     438037.801          0.084
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :        167.518        167.518        167.518          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/10/2020 17:57:08 --

mdtest-hard-stat
-- started at 07/10/2020 17:49:56 --

mdtest-3.3.0+dev was launched with 260 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '400000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/mnt/llfs//2020.07.10-17.14.40-app/mdtest-hard' '-x' '/mnt/llfs//2020.07.10-17.14.40-app/mdtest-hard-stonewall' '-T' '-a' 'POSIX'
Path: /mnt/llfs/2020.07.10-17.14.40-app
FS: 51.1 TiB   Used FS: 28.6%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 26 for each phase.
260 tasks, 104000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :    1245930.086    1245658.618    1245915.506         58.918
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :         58.908         58.895         58.896          0.003
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/10/2020 17:50:55 --

mdtest-hard-write
-- started at 07/10/2020 17:34:24 --

mdtest-3.3.0+dev was launched with 260 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '400000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/mnt/llfs//2020.07.10-17.14.40-app/mdtest-hard' '-x' '/mnt/llfs//2020.07.10-17.14.40-app/mdtest-hard-stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /mnt/llfs/2020.07.10-17.14.40-app
FS: 51.1 TiB   Used FS: 28.6%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 11111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 26 for each phase.
260 tasks, 104000000 files
Continue stonewall hit min: 259240 max: 282227 avg: 271470.1 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :     209095.500     209095.393     209095.451          0.025
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA     235271.957             NA
   Tree creation             :        816.966        816.966        816.966          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        350.936        350.935        350.936          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.003             NA
   Tree creation             :          0.001          0.001          0.001          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/10/2020 17:41:40 --

result
version         = io500-isc20_v4-2-g20efd247106c-17
config-hash     = D27E6923
result-dir      = /mnt/llfs/results/2020.07.10-17.14.40-app
; START 2020-07-10 17:14:48


[ior-easy-write]
t_start         = 2020-07-10 17:14:50
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o /mnt/llfs//2020.07.10-17.14.40-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/mnt/llfs//2020.07.10-17.14.40-app/ior-easy/stonewall -t 1m -b 59392m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX --posix.odirect
throughput-stonewall = 9914.91
score           = 38.704764
t_delta         = 308.1774
t_end           = 2020-07-10 17:19:58

[mdtest-easy-write]
t_start         = 2020-07-10 17:20:00
exe             = ./mdtest -n 550000 -u -L -F -P -N 1 -d /mnt/llfs//2020.07.10-17.14.40-app/mdtest-easy -x /mnt/llfs//2020.07.10-17.14.40-app/mdtest-easy-stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 319.765731
score           = 277.091678
t_delta         = 493.6729
t_end           = 2020-07-10 17:28:13

[timestamp]
t_start         = 2020-07-10 17:29:12
t_delta         = 0.0158
t_end           = 2020-07-10 17:29:12

[ior-hard-write]
t_start         = 2020-07-10 17:29:13
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o /mnt/llfs//2020.07.10-17.14.40-app/ior-hard/file -O stoneWallingStatusFile=/mnt/llfs//2020.07.10-17.14.40-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 420000 -w -D 300 -a POSIX
throughput-stonewall = 2530.65
score           = 9.917340
t_delta         = 309.4600
t_end           = 2020-07-10 17:34:23

[mdtest-hard-write]
t_start         = 2020-07-10 17:34:24
exe             = ./mdtest -n 400000 -t -w 3901 -e 3901 -P -N 1 -F -d /mnt/llfs//2020.07.10-17.14.40-app/mdtest-hard -x /mnt/llfs//2020.07.10-17.14.40-app/mdtest-hard-stonewall -C -Y -W 300 -a POSIX
rate-stonewall  = 235.271957
score           = 209.095462
t_delta         = 435.7473
t_end           = 2020-07-10 17:41:40

[find]
t_start         = 2020-07-10 17:42:25
exe             =  /root/io500-app/bin/llfs-pfind.sh  /mnt/llfs//2020.07.10-17.14.40-app -newer /mnt/llfs//2020.07.10-17.14.40-app/timestampfile -size 3901c -name "*01*"
nproc           = 1
last-output     = "MATCHED 3240946/176489306"
total-files     = 176489306
found           = 3240946
score           = 4684.309045
t_delta         = 37.6951
t_end           = 2020-07-10 17:43:02

[ior-easy-read]
t_start         = 2020-07-10 17:43:09
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o /mnt/llfs//2020.07.10-17.14.40-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/mnt/llfs//2020.07.10-17.14.40-app/ior-easy/stonewall -t 1m -b 59392m -F -r -R -a POSIX --posix.odirect
score           = 64.103014
t_delta         = 185.4286
t_end           = 2020-07-10 17:46:14

[mdtest-easy-stat]
t_start         = 2020-07-10 17:46:16
exe             = ./mdtest -n 550000 -u -L -F -P -N 1 -d /mnt/llfs//2020.07.10-17.14.40-app/mdtest-easy -x /mnt/llfs//2020.07.10-17.14.40-app/mdtest-easy-stonewall -T -a POSIX
score           = 1355.074126
t_delta         = 76.1205
t_end           = 2020-07-10 17:47:32

[ior-hard-read]
t_start         = 2020-07-10 17:48:30
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o /mnt/llfs//2020.07.10-17.14.40-app/ior-hard/file -O stoneWallingStatusFile=/mnt/llfs//2020.07.10-17.14.40-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 420000 -r -R -a POSIX
score           = 35.725194
t_delta         = 85.9862
t_end           = 2020-07-10 17:49:56

[mdtest-hard-stat]
t_start         = 2020-07-10 17:49:56
exe             = ./mdtest -n 400000 -t -w 3901 -e 3901 -P -N 1 -F -d /mnt/llfs//2020.07.10-17.14.40-app/mdtest-hard -x /mnt/llfs//2020.07.10-17.14.40-app/mdtest-hard-stonewall -T -a POSIX
score           = 1245.929561
t_delta         = 58.9459
t_end           = 2020-07-10 17:50:55

[mdtest-easy-delete]
t_start         = 2020-07-10 17:51:40
exe             = ./mdtest -n 550000 -u -L -F -P -N 1 -d /mnt/llfs//2020.07.10-17.14.40-app/mdtest-easy -x /mnt/llfs//2020.07.10-17.14.40-app/mdtest-easy-stonewall -r -a POSIX
score           = 647.296271
t_delta         = 159.6266
t_end           = 2020-07-10 17:54:20

[mdtest-hard-read]
t_start         = 2020-07-10 17:54:21
exe             = ./mdtest -n 400000 -t -w 3901 -e 3901 -P -N 1 -F -d /mnt/llfs//2020.07.10-17.14.40-app/mdtest-hard -x /mnt/llfs//2020.07.10-17.14.40-app/mdtest-hard-stonewall -E -X -a POSIX
score           = 438.037918
t_delta         = 167.5357
t_end           = 2020-07-10 17:57:08

[mdtest-hard-delete]
t_start         = 2020-07-10 17:57:53
exe             = ./mdtest -n 400000 -t -w 3901 -e 3901 -P -N 1 -F -d /mnt/llfs//2020.07.10-17.14.40-app/mdtest-hard -x /mnt/llfs//2020.07.10-17.14.40-app/mdtest-hard-stonewall -r -a POSIX
score           = 532.290078
t_delta         = 144.1534
t_end           = 2020-07-10 18:00:17

[SCORE]
MD              = 716.109062
BW              = 30.619858
SCORE           = 148.078214 
hash            = 1372FD50
; END 2020-07-10 18:00:17
result_summary
IO500 version io500-isc20_v4-2-g20efd247106c-17
[RESULT]       ior-easy-write       38.704764 GiB/s  : time 308.177 seconds
[RESULT]    mdtest-easy-write      277.091678 kIOPS : time 493.673 seconds
[RESULT]       ior-hard-write        9.917340 GiB/s  : time 309.460 seconds
[RESULT]    mdtest-hard-write      209.095462 kIOPS : time 435.747 seconds
[RESULT]                 find     4684.309045 kIOPS : time 37.695 seconds
[RESULT]        ior-easy-read       64.103014 GiB/s  : time 185.429 seconds
[RESULT]     mdtest-easy-stat     1355.074126 kIOPS : time 76.121 seconds
[RESULT]        ior-hard-read       35.725194 GiB/s  : time 85.986 seconds
[RESULT]     mdtest-hard-stat     1245.929561 kIOPS : time 58.946 seconds
[RESULT]   mdtest-easy-delete      647.296271 kIOPS : time 159.627 seconds
[RESULT]     mdtest-hard-read      438.037918 kIOPS : time 167.536 seconds
[RESULT]   mdtest-hard-delete      532.290078 kIOPS : time 144.153 seconds
[SCORE] Bandwidth 30.619858 GB/s : IOPS 716.109062 kiops : TOTAL 148.078214