Cygnus

Institution University of Tsukuba
Client Procs Per Node
Client Operating System CentOS
Client Operating System Version 7.7.1908
Client Kernel Version 3.10.0-1062.18.1.el7.x86_64

DATA SERVER

Storage Type NVMe-SSD
Volatile Memory
Storage Interface NVMe
Network InfiniBand
Software Version
OS Version

INFORMATION

Client Nodes 10
Client Total Procs 20
Metadata Nodes 1
Metadata Storage Devices 1
Data Nodes 10
Data Storage Devices 1

METADATA

Easy Write 30.09 kIOP/s
Easy Stat 40.64 kIOP/s
Easy Delete 19.31 kIOP/s
Hard Write 13.19 kIOP/s
Hard Read 16.40 kIOP/s
Hard Stat 39.23 kIOP/s
Hard Delete 15.91 kIOP/s

Submitted Files

ior-easy-read
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Sat Oct 31 23:07:06 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/ior-easy/ior_file_easy -O stoneWallingStatusFile=/work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/ior-easy.stonewall -t 1m -b 600000m -F -r -R -a Gfarm
Machine             : Linux gnode08
TestID              : 0
StartTime           : Sat Oct 31 23:07:07 2020
Path                : /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/ior-easy/ior_file_easy.00000000
FS                  : 27.8 GiB   Used FS: 57.5%   Inodes: 30.7 Mi   Used Inodes: 50.0%

Options: 
api                 : Gfarm
apiVersion          : 2.7.17
test filename       : /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 20
clients per node    : 2
repetitions         : 1
xfersize            : 1 MiB
blocksize           : 585.94 GiB
aggregate filesize  : 11.44 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
WARNING: Expected aggregate file size       = 12582912000000
WARNING: Stat() of aggregate file size      = 10246202327040
WARNING: Using actual aggregate bytes moved = 10246202327040
read      22264      22265      0.000731    614400000  1024.00    0.006364   438.88     0.003082   438.89     0   
ior-easy-write
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Sat Oct 31 22:40:52 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/ior-easy/ior_file_easy -O stoneWallingStatusFile=/work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/ior-easy.stonewall -t 1m -b 600000m -F -w -D 300 -O stoneWallingWearOut=1 -a Gfarm
Machine             : Linux gnode08
TestID              : 0
StartTime           : Sat Oct 31 22:40:52 2020
Path                : /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/ior-easy/ior_file_easy.00000000
FS                  : 27.8 GiB   Used FS: 0.0%   Inodes: 0.0 Mi   Used Inodes: 50.0%

Options: 
api                 : Gfarm
apiVersion          : 2.7.17
test filename       : /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 20
clients per node    : 2
repetitions         : 1
xfersize            : 1 MiB
blocksize           : 585.94 GiB
aggregate filesize  : 11.44 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 391267 max: 488577 -- min data: 382.1 GiB mean data: 432.3 GiB time: 300.0s
WARNING: Expected aggregate file size       = 12582912000000
WARNING: Stat() of aggregate file size      = 10246202327040
WARNING: Using actual aggregate bytes moved = 10246202327040
WARNING: Maybe caused by deadlineForStonewalling
write     26598      26599      0.000591    614400000  1024.00    0.000778   367.37     0.005025   367.37     0   
ior-hard-read
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Sat Oct 31 23:18:58 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/ior-hard/file -O stoneWallingStatusFile=/work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a MPIIO
Machine             : Linux gnode08
TestID              : 0
StartTime           : Sat Oct 31 23:18:58 2020
Path                : /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/ior-hard/file
FS                  : 27.8 TiB   Used FS: 57.5%   Inodes: 15.4 Mi   Used Inodes: 100.0%

Options: 
api                 : MPIIO
apiVersion          : (3.1)
test filename       : /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 20
clients per node    : 2
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 8.55 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 4479707 max: 4479707 -- min data: 196.1 GiB mean data: 196.1 GiB time: 5879.4s
WARNING: inconsistent file size by different tasks.
WARNING: Expected aggregate file size       = 9401600000000
WARNING: Stat() of aggregate file size      = 4211640439968
WARNING: Using actual aggregate bytes moved = 4211641333120
read      676.17     15238      5880        45.91      45.91      55.07      5880       5.42       5940       0   
ior-hard-write
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Sat Oct 31 22:53:04 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/ior-hard/file -O stoneWallingStatusFile=/work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a MPIIO
Machine             : Linux gnode08
TestID              : 0
StartTime           : Sat Oct 31 22:53:04 2020
Path                : /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/ior-hard/file
FS                  : 27.8 TiB   Used FS: 33.5%   Inodes: 10.4 Mi   Used Inodes: 100.0%

Options: 
api                 : MPIIO
apiVersion          : (3.1)
test filename       : /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 20
clients per node    : 2
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 8.55 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 4038371 max: 4479707 -- min data: 176.8 GiB mean data: 186.1 GiB time: 300.0s
WARNING: inconsistent file size by different tasks.
WARNING: Expected aggregate file size       = 9401600000000
WARNING: Stat() of aggregate file size      = 4211640439968
WARNING: Using actual aggregate bytes moved = 4211641333120
WARNING: Maybe caused by deadlineForStonewalling
write     11836      268733     322.55      45.91      45.91      5.48       333.39     0.466898   339.34     0   
mdtest-easy-delete
-- started at 11/01/2020 01:01:11 --

mdtest-3.4.0+dev was launched with 20 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '600000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/mdtest-easy' '-x' '/work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/mdtest-easy.stonewall' '-r' '-a' 'Gfarm'
Path: /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45
FS: 27.8 GiB   Used FS: 57.5%   Inodes: 30.7 Mi   Used Inodes: 50.0%

Nodemap: 10000000001000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
20 tasks, 12000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :      19305.836      19305.819      19305.827          0.005
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :        702.799        702.799        702.799          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        565.340        565.340        565.340          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.001          0.001          0.001          0.000
-- finished at 11/01/2020 01:10:37 --

mdtest-easy-stat
-- started at 10/31/2020 23:14:29 --

mdtest-3.4.0+dev was launched with 20 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '600000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/mdtest-easy' '-x' '/work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/mdtest-easy.stonewall' '-T' '-a' 'Gfarm'
Path: /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45
FS: 27.8 GiB   Used FS: 57.5%   Inodes: 30.7 Mi   Used Inodes: 50.0%

Nodemap: 10000000001000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
20 tasks, 12000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :      40636.414      40636.350      40636.385          0.019
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :        268.586        268.586        268.586          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/31/2020 23:18:58 --

mdtest-easy-write
-- started at 10/31/2020 22:47:01 --

mdtest-3.4.0+dev was launched with 20 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '600000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/mdtest-easy' '-x' '/work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/mdtest-easy.stonewall' '-C' '-Y' '-W' '300' '-a' 'Gfarm'
Path: /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45
FS: 27.8 GiB   Used FS: 29.3%   Inodes: 0.0 Mi   Used Inodes: 50.0%

Nodemap: 10000000001000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
20 tasks, 12000000 files
Continue stonewall hit min: 429459 max: 545718 avg: 456453.8 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :      30085.923      30085.843      30085.868          0.023
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA      30430.188             NA
   Tree creation             :        208.154        208.154        208.154          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        362.774        362.773        362.774          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.001             NA
   Tree creation             :          0.005          0.005          0.005          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/31/2020 22:53:04 --

mdtest-hard-delete
-- started at 11/01/2020 01:15:58 --

mdtest-3.4.0+dev was launched with 20 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '260000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/mdtest-hard' '-x' '/work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/mdtest-hard.stonewall' '-r' '-a' 'Gfarm'
Path: /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45
FS: 27.8 GiB   Used FS: 57.5%   Inodes: 9.9 Mi   Used Inodes: 50.0%

Nodemap: 10000000001000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
20 tasks, 5200000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :      15908.956      15908.944      15908.949          0.004
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :       3134.756       3134.756       3134.756          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        326.860        326.860        326.860          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 11/01/2020 01:21:25 --

mdtest-hard-read
-- started at 11/01/2020 01:10:39 --

mdtest-3.4.0+dev was launched with 20 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '260000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/mdtest-hard' '-x' '/work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/mdtest-hard.stonewall' '-E' '-X' '-a' 'Gfarm'
Path: /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45
FS: 27.8 GiB   Used FS: 57.5%   Inodes: 9.9 Mi   Used Inodes: 50.0%

Nodemap: 10000000001000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
20 tasks, 5200000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :      16399.611      16399.593      16399.603          0.006
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :        317.081        317.081        317.081          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 11/01/2020 01:15:56 --

mdtest-hard-stat
-- started at 11/01/2020 00:58:58 --

mdtest-3.4.0+dev was launched with 20 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '260000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/mdtest-hard' '-x' '/work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/mdtest-hard.stonewall' '-T' '-a' 'Gfarm'
Path: /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45
FS: 27.8 GiB   Used FS: 57.5%   Inodes: 30.7 Mi   Used Inodes: 50.0%

Nodemap: 10000000001000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
20 tasks, 5200000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :      39234.537      39234.430      39234.481          0.039
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :        132.537        132.536        132.536          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 11/01/2020 01:01:11 --

mdtest-hard-write
-- started at 10/31/2020 22:59:46 --

mdtest-3.4.0+dev was launched with 20 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '260000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/mdtest-hard' '-x' '/work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/mdtest-hard.stonewall' '-C' '-Y' '-W' '300' '-a' 'Gfarm'
Path: /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45
FS: 27.8 GiB   Used FS: 54.5%   Inodes: 20.8 Mi   Used Inodes: 50.0%

Nodemap: 10000000001000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
20 tasks, 5200000 files
Continue stonewall hit min: 186215 max: 260000 avg: 201356.1 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :      13190.816      13190.792      13190.804          0.008
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA      13423.690             NA
   Tree creation             :       2878.726       2878.726       2878.726          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        394.214        394.214        394.214          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.001             NA
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 10/31/2020 23:06:20 --

result
version         = io500-sc20_v3-5-gee79fd4bcd36
config-hash     = 584608B2
result-dir      = /work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45
; START 2020-10-31 22:40:52


[ior-easy-write]
t_start         = 2020-10-31 22:40:52
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/ior-easy/ior_file_easy -O stoneWallingStatusFile=/work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/ior-easy.stonewall -t 1m -b 600000m -F -w -D 300 -O stoneWallingWearOut=1 -a Gfarm
throughput-stonewall = 470.67
score           = 25.974926
t_delta         = 367.4814
t_end           = 2020-10-31 22:46:59

[mdtest-easy-write]
t_start         = 2020-10-31 22:46:59
exe             = ./mdtest -n 600000 -u -L -F -P -N 1 -d /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/mdtest-easy -x /work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/mdtest-easy.stonewall -C -Y -W 300 -a Gfarm
rate-stonewall  = 30.430188
score           = 30.085873
t_delta         = 364.7815
t_end           = 2020-10-31 22:53:04

[timestamp]
t_start         = 2020-10-31 22:53:04
t_delta         = 0.0072
t_end           = 2020-10-31 22:53:04

[ior-hard-write]
t_start         = 2020-10-31 22:53:04
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/ior-hard/file -O stoneWallingStatusFile=/work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a MPIIO
throughput-stonewall = 219.41
score           = 11.558757
t_delta         = 401.0785
t_end           = 2020-10-31 22:59:45

[mdtest-hard-write]
t_start         = 2020-10-31 22:59:45
exe             = ./mdtest -n 260000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/mdtest-hard -x /work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/mdtest-hard.stonewall -C -Y -W 300 -a Gfarm
rate-stonewall  = 13.423690
score           = 13.190808
t_delta         = 394.8013
t_end           = 2020-10-31 23:06:20

[find]
t_start         = 2020-10-31 23:06:20
exe             = ./pfind /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45 -newer /work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/timestampfile -size 3901c -name *01* -C -q 10000
found           = 192480
total-files     = 16114420
score           = 352.423767
t_delta         = 45.8006
t_end           = 2020-10-31 23:07:06

[ior-easy-read]
t_start         = 2020-10-31 23:07:06
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/ior-easy/ior_file_easy -O stoneWallingStatusFile=/work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/ior-easy.stonewall -t 1m -b 600000m -F -r -R -a Gfarm
score           = 21.742418
t_delta         = 440.8893
t_end           = 2020-10-31 23:14:27

[mdtest-easy-stat]
t_start         = 2020-10-31 23:14:27
exe             = ./mdtest -n 600000 -u -L -F -P -N 1 -d /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/mdtest-easy -x /work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/mdtest-easy.stonewall -T -a Gfarm
score           = 40.636410
t_delta         = 270.9705
t_end           = 2020-10-31 23:18:58

[ior-hard-read]
t_start         = 2020-10-31 23:18:58
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/ior-hard/file -O stoneWallingStatusFile=/work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a MPIIO
score           = 0.660321
t_delta         = 6000.2903
t_end           = 2020-11-01 00:58:58

[mdtest-hard-stat]
t_start         = 2020-11-01 00:58:58
exe             = ./mdtest -n 260000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/mdtest-hard -x /work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/mdtest-hard.stonewall -T -a Gfarm
score           = 39.234431
t_delta         = 132.6768
t_end           = 2020-11-01 01:01:11

[mdtest-easy-delete]
t_start         = 2020-11-01 01:01:11
exe             = ./mdtest -n 600000 -u -L -F -P -N 1 -d /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/mdtest-easy -x /work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/mdtest-easy.stonewall -r -a Gfarm
score           = 19.305836
t_delta         = 566.0639
t_end           = 2020-11-01 01:10:37

[mdtest-hard-read]
t_start         = 2020-11-01 01:10:37
exe             = ./mdtest -n 260000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/mdtest-hard -x /work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/mdtest-hard.stonewall -E -X -a Gfarm
score           = 16.399593
t_delta         = 319.0832
t_end           = 2020-11-01 01:15:56

[mdtest-hard-delete]
t_start         = 2020-11-01 01:15:56
exe             = ./mdtest -n 260000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/gfarm-sugiharak/datafiles/2020.10.31-22.40.45/mdtest-hard -x /work/NBB/sugiharak/io500-sc20/results/2020.10.31-22.40.45/mdtest-hard.stonewall -r -a Gfarm
score           = 15.908947
t_delta         = 330.1939
t_end           = 2020-11-01 01:21:26

[SCORE]
MD              = 32.085265
BW              = 8.102743
SCORE           = 16.123854 
hash            = F00CAB33
result_summary
IO500 version io500-sc20_v3-5-gee79fd4bcd36
[RESULT]       ior-easy-write       25.974926 GiB/s : time 367.481 seconds
[RESULT]    mdtest-easy-write       30.085873 kIOPS : time 364.781 seconds
[RESULT]       ior-hard-write       11.558757 GiB/s : time 401.078 seconds
[RESULT]    mdtest-hard-write       13.190808 kIOPS : time 394.801 seconds
[RESULT]                 find      352.423767 kIOPS : time 45.801 seconds
[RESULT]        ior-easy-read       21.742418 GiB/s : time 440.889 seconds
[RESULT]     mdtest-easy-stat       40.636410 kIOPS : time 270.971 seconds
[RESULT]        ior-hard-read        0.660321 GiB/s : time 6000.290 seconds
[RESULT]     mdtest-hard-stat       39.234431 kIOPS : time 132.677 seconds
[RESULT]   mdtest-easy-delete       19.305836 kIOPS : time 566.064 seconds
[RESULT]     mdtest-hard-read       16.399593 kIOPS : time 319.083 seconds
[RESULT]   mdtest-hard-delete       15.908947 kIOPS : time 330.194 seconds
[SCORE] Bandwidth 8.102743 GiB/s : IOPS 32.085265 kiops : TOTAL 16.123854