Cygnus

Institution University of Tsukuba
Client Procs Per Node
Client Operating System CentOS
Client Operating System Version 7.9
Client Kernel Version 3.1

DATA SERVER

Storage Type
Volatile Memory
Storage Interface
Network
Software Version
OS Version

INFORMATION

Client Nodes 10
Client Total Procs 240

METADATA

Easy Write 1,582.62 kIOP/s
Easy Stat 1,955.42 kIOP/s
Easy Delete 656.24 kIOP/s
Hard Write 258.63 kIOP/s
Hard Read 841.25 kIOP/s
Hard Stat 602.41 kIOP/s
Hard Delete 279.39 kIOP/s

Submitted Files

ior-easy-read
WARNING: fsync() not supported in selected backend.  Using value of 0.
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Mon May 24 00:36:51 2021
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/ior-easy/ior_file_easy -O stoneWallingStatusFile=/work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/ior-easy.stonewall -t 1048576 -b 9920000m -F -r -R -a CHFS --chfs.chunk_size=1048576
Machine             : Linux gnode01
TestID              : 0
StartTime           : Mon May 24 00:36:51 2021
Path                : /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/ior-easy/ior_file_easy.00000000
FS                  : 43.3 TiB   Used FS: 395.3%   Inodes: -134202007.2 Mi   Used Inodes: 100.0%

Options: 
api                 : CHFS
apiVersion          : 1.0.0
test filename       : /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 240
clients per node    : 24
repetitions         : 1
xfersize            : 1 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 2270.51 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
WARNING: Expected aggregate file size       = 2496449740800000
WARNING: Stat() of aggregate file size      = 23740683386880
WARNING: Using actual aggregate bytes moved = 23740683386880
read      73561      73574      0.000031    10158080000 1024.00    0.053849   307.73     0.001816   307.78     0   
ior-easy-write
WARNING: fsync() not supported in selected backend.  Using value of 0.
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Mon May 24 00:05:24 2021
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/ior-easy/ior_file_easy -O stoneWallingStatusFile=/work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/ior-easy.stonewall -t 1048576 -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a CHFS --chfs.chunk_size=1048576
Machine             : Linux gnode01
TestID              : 0
StartTime           : Mon May 24 00:05:25 2021
Path                : /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/ior-easy/ior_file_easy.00000000
FS                  : -4999713313.5 GiB   Used FS: 227.3%   Inodes: 0.0 Mi   Used Inodes: -7036050192654.0%

Options: 
api                 : CHFS
apiVersion          : 1.0.0
test filename       : /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 240
clients per node    : 24
repetitions         : 1
xfersize            : 1 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 2270.51 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 89541 max: 94337 -- min data: 87.4 GiB mean data: 90.1 GiB time: 300.0s
WARNING: Expected aggregate file size       = 2496449740800000
WARNING: Stat() of aggregate file size      = 23740683386880
WARNING: Using actual aggregate bytes moved = 23740683386880
WARNING: Maybe caused by deadlineForStonewalling
write     73414      73423      0.000031    10158080000 1024.00    0.037591   308.36     0.000305   308.40     0   
ior-hard-read
WARNING: fsync() not supported in selected backend.  Using value of 0.
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Mon May 24 00:46:41 2021
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/ior-hard/file -O stoneWallingStatusFile=/work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a CHFS --chfs.chunk_size=47008
Machine             : Linux gnode01
TestID              : 0
StartTime           : Mon May 24 00:46:42 2021
Path                : /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/ior-hard/file
FS                  : 43.3 TiB   Used FS: 395.3%   Inodes: -134202007.2 Mi   Used Inodes: 100.0%

Options: 
api                 : CHFS
apiVersion          : 1.0.0
test filename       : /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 240
clients per node    : 24
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 102.61 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 358904 max: 358904 -- min data: 15.7 GiB mean data: 15.7 GiB time: 276.2s
WARNING: Expected aggregate file size       = 112819200000000
WARNING: Stat() of aggregate file size      = 4049126215680
WARNING: Using actual aggregate bytes moved = 4049126215680
read      13955      311871     276.19      45.91      45.91      0.511545   276.19     0.002317   276.71     0   
ior-hard-write
WARNING: fsync() not supported in selected backend.  Using value of 0.
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Mon May 24 00:16:31 2021
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/ior-hard/file -O stoneWallingStatusFile=/work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a CHFS --chfs.chunk_size=47008
Machine             : Linux gnode01
TestID              : 0
StartTime           : Mon May 24 00:16:32 2021
Path                : /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/ior-hard/file
FS                  : 43.3 TiB   Used FS: 395.3%   Inodes: -134202007.2 Mi   Used Inodes: 100.0%

Options: 
api                 : CHFS
apiVersion          : 1.0.0
test filename       : /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 10000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 10
tasks               : 240
clients per node    : 24
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 102.61 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 347460 max: 358904 -- min data: 15.2 GiB mean data: 15.5 GiB time: 300.0s
WARNING: Expected aggregate file size       = 112819200000000
WARNING: Stat() of aggregate file size      = 4049126215680
WARNING: Using actual aggregate bytes moved = 4049126215680
WARNING: Maybe caused by deadlineForStonewalling
write     12436      277861     305.80      45.91      45.91      0.512516   310.00     0.000718   310.51     0   
mdtest-easy-delete
-- started at 05/24/2021 00:53:35 --

mdtest-3.4.0+dev was launched with 240 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '3000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/mdtest-easy' '-x' '/work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/mdtest-easy.stonewall' '-r' '-a' 'CHFS' '--chfs.chunk_size=1'
Path: /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20
FS: 0.0 GiB   Used FS: -nan%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
240 tasks, 720000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :     656241.835     656182.121     656205.788         25.324
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.859          0.859          0.859          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        835.676        835.600        835.646          0.032
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          1.165          1.165          1.165          0.000
-- finished at 05/24/2021 01:07:45 --

mdtest-easy-stat
-- started at 05/24/2021 00:42:00 --

mdtest-3.4.0+dev was launched with 240 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '3000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/mdtest-easy' '-x' '/work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/mdtest-easy.stonewall' '-T' '-a' 'CHFS' '--chfs.chunk_size=1'
Path: /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20
FS: 0.0 GiB   Used FS: -nan%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
240 tasks, 720000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :    1955424.400    1955420.142    1955422.982          0.951
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :        280.429        280.428        280.428          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 05/24/2021 00:46:41 --

mdtest-easy-write
-- started at 05/24/2021 00:10:44 --

mdtest-3.4.0+dev was launched with 240 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '3000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/mdtest-easy' '-x' '/work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/mdtest-easy.stonewall' '-C' '-Y' '-W' '300' '-a' 'CHFS' '--chfs.chunk_size=1'
Path: /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20
FS: 0.0 GiB   Used FS: -nan%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
240 tasks, 720000000 files
Continue stonewall hit min: 1962262 max: 2284815 avg: 2127763.2 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :    1582616.015    1582614.006    1582615.166          0.451
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA    1701966.389             NA
   Tree creation             :         14.656         14.656         14.656          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        346.487        346.487        346.487          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.043             NA
   Tree creation             :          0.068          0.068          0.068          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 05/24/2021 00:16:31 --

mdtest-hard-delete
-- started at 05/24/2021 01:09:26 --

mdtest-3.4.0+dev was launched with 240 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/mdtest-hard' '-x' '/work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/mdtest-hard.stonewall' '-r' '-a' 'CHFS' '--chfs.chunk_size=3902'
Path: /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20
FS: 0.0 GiB   Used FS: -nan%   Inodes: 4421639116436.7 Mi   Used Inodes: 100.0%

Nodemap: 100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
240 tasks, 240000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :     279392.353     279391.469     279392.163          0.150
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.384          0.384          0.384          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        287.722        287.721        287.721          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          2.605          2.605          2.605          0.000
-- finished at 05/24/2021 01:14:18 --

mdtest-hard-read
-- started at 05/24/2021 01:07:49 --

mdtest-3.4.0+dev was launched with 240 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/mdtest-hard' '-x' '/work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/mdtest-hard.stonewall' '-E' '-X' '-a' 'CHFS' '--chfs.chunk_size=3902'
Path: /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20
FS: 0.0 GiB   Used FS: -nan%   Inodes: 0.0 Mi   Used Inodes: -inf%

Nodemap: 100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
240 tasks, 240000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :     841245.414     841235.927     841244.565          1.468
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :         95.558         95.557         95.557          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 05/24/2021 01:09:25 --

mdtest-hard-stat
-- started at 05/24/2021 00:51:20 --

mdtest-3.4.0+dev was launched with 240 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/mdtest-hard' '-x' '/work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/mdtest-hard.stonewall' '-T' '-a' 'CHFS' '--chfs.chunk_size=3902'
Path: /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20
FS: 0.0 GiB   Used FS: -nan%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
240 tasks, 240000000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :     602416.561     602413.927     602414.876          0.375
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :        133.442        133.441        133.441          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 05/24/2021 00:53:33 --

mdtest-hard-write
-- started at 05/24/2021 00:21:44 --

mdtest-3.4.0+dev was launched with 240 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/mdtest-hard' '-x' '/work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/mdtest-hard.stonewall' '-C' '-Y' '-W' '300' '-a' 'CHFS' '--chfs.chunk_size=3902'
Path: /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20
FS: 0.0 GiB   Used FS: -nan%   Inodes: 0.0 Mi   Used Inodes: -nan%

Nodemap: 100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001000000000
V-0: Rank   0 Line  2216 Shifting ranks by 1 for each phase.
240 tasks, 240000000 files
Continue stonewall hit min: 322343 max: 334946 avg: 329504.5 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :     258633.311     258632.684     258633.146          0.152
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA     263594.203             NA
   Tree creation             :         75.593         75.593         75.593          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        310.815        310.815        310.815          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.011             NA
   Tree creation             :          0.013          0.013          0.013          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 05/24/2021 00:26:55 --

result
version         = io500-sc20_v3-2-g26a6fed616ff
config-hash     = 3595B519
result-dir      = /work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20
; START 2021-05-24 00:05:24


[ior-easy-write]
t_start         = 2021-05-24 00:05:24
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/ior-easy/ior_file_easy -O stoneWallingStatusFile=/work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/ior-easy.stonewall -t 1048576 -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a CHFS --chfs.chunk_size=1048576
throughput-stonewall = 16820.71
score           = 71.693064
t_delta         = 319.8163
t_end           = 2021-05-24 00:10:44

[mdtest-easy-write]
t_start         = 2021-05-24 00:10:44
exe             = ./mdtest -n 3000000 -u -L -F -P -N 1 -d /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/mdtest-easy -x /work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/mdtest-easy.stonewall -C -Y -W 300 -a CHFS --chfs.chunk_size=1
rate-stonewall  = 1701.966389
score           = 1582.615900
t_delta         = 347.3781
t_end           = 2021-05-24 00:16:31

[timestamp]
t_start         = 2021-05-24 00:16:31
t_delta         = 0.0003
t_end           = 2021-05-24 00:16:31

[ior-hard-write]
t_start         = 2021-05-24 00:16:31
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/ior-hard/file -O stoneWallingStatusFile=/work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a CHFS --chfs.chunk_size=47008
throughput-stonewall = 2868.56
score           = 12.144590
t_delta         = 311.6980
t_end           = 2021-05-24 00:21:43

[mdtest-hard-write]
t_start         = 2021-05-24 00:21:43
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/mdtest-hard -x /work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/mdtest-hard.stonewall -C -Y -W 300 -a CHFS --chfs.chunk_size=3902
rate-stonewall  = 263.594203
score           = 258.633235
t_delta         = 311.6567
t_end           = 2021-05-24 00:26:55

[find]
t_start         = 2021-05-24 00:26:55
exe             = ./pfind /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20 -newer /work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/timestampfile -size 3901c -name *01* -C -q 10000
nproc           = 10
found           = 3739140
total-files     = 628742881
score           = 1056.268758
t_delta         = 596.0691
t_end           = 2021-05-24 00:36:51

[ior-easy-read]
t_start         = 2021-05-24 00:36:51
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/ior-easy/ior_file_easy -O stoneWallingStatusFile=/work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/ior-easy.stonewall -t 1048576 -b 9920000m -F -r -R -a CHFS --chfs.chunk_size=1048576
score           = 71.836846
t_delta         = 308.7992
t_end           = 2021-05-24 00:42:00

[mdtest-easy-stat]
t_start         = 2021-05-24 00:42:00
exe             = ./mdtest -n 3000000 -u -L -F -P -N 1 -d /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/mdtest-easy -x /work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/mdtest-easy.stonewall -T -a CHFS --chfs.chunk_size=1
score           = 1955.423375
t_delta         = 281.3829
t_end           = 2021-05-24 00:46:41

[ior-hard-read]
t_start         = 2021-05-24 00:46:41
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/ior-hard/file -O stoneWallingStatusFile=/work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a CHFS --chfs.chunk_size=47008
score           = 13.628314
t_delta         = 277.8493
t_end           = 2021-05-24 00:51:19

[mdtest-hard-stat]
t_start         = 2021-05-24 00:51:19
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/mdtest-hard -x /work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/mdtest-hard.stonewall -T -a CHFS --chfs.chunk_size=3902
score           = 602.414994
t_delta         = 134.3402
t_end           = 2021-05-24 00:53:34

[mdtest-easy-delete]
t_start         = 2021-05-24 00:53:34
exe             = ./mdtest -n 3000000 -u -L -F -P -N 1 -d /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/mdtest-easy -x /work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/mdtest-easy.stonewall -r -a CHFS --chfs.chunk_size=1
score           = 656.241600
t_delta         = 852.7919
t_end           = 2021-05-24 01:07:46

[mdtest-hard-read]
t_start         = 2021-05-24 01:07:46
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/mdtest-hard -x /work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/mdtest-hard.stonewall -E -X -a CHFS --chfs.chunk_size=3902
score           = 841.245156
t_delta         = 98.9519
t_end           = 2021-05-24 01:09:25

[mdtest-hard-delete]
t_start         = 2021-05-24 01:09:26
exe             = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/gfarm-tatebe/datafiles/2021.05.24-00.05.20/mdtest-hard -x /work/NBB/tatebe/io500-sc20/results/2021.05.24-00.05.20/mdtest-hard.stonewall -r -a CHFS --chfs.chunk_size=3902
score           = 279.392210
t_delta         = 292.8581
t_end           = 2021-05-24 01:14:18

[SCORE]
MD              = 727.606276
BW              = 30.385217
SCORE           = 148.689189 
hash            = 4B6A7D35
; END 2021-05-24 01:56:27
result_summary
IO500 version io500-sc20_v3-2-g26a6fed616ff
[RESULT]       ior-easy-write       71.693064 GiB/s : time 319.816 seconds
[RESULT]    mdtest-easy-write     1582.615900 kIOPS : time 347.378 seconds
[RESULT]       ior-hard-write       12.144590 GiB/s : time 311.698 seconds
[RESULT]    mdtest-hard-write      258.633235 kIOPS : time 311.657 seconds
[RESULT]                 find     1056.268758 kIOPS : time 596.069 seconds
[RESULT]        ior-easy-read       71.836846 GiB/s : time 308.799 seconds
[RESULT]     mdtest-easy-stat     1955.423375 kIOPS : time 281.383 seconds
[RESULT]        ior-hard-read       13.628314 GiB/s : time 277.849 seconds
[RESULT]     mdtest-hard-stat      602.414994 kIOPS : time 134.340 seconds
[RESULT]   mdtest-easy-delete      656.241600 kIOPS : time 852.792 seconds
[RESULT]     mdtest-hard-read      841.245156 kIOPS : time 98.952 seconds
[RESULT]   mdtest-hard-delete      279.392210 kIOPS : time 292.858 seconds
[SCORE] Bandwidth 30.385217 GiB/s : IOPS 727.606276 kiops : TOTAL 148.689189