Officinalis

Institution
Client Procs Per Node
Client Operating System CentOS
Client Operating System Version CentOS Linux release 8.0.1905 (Core)
Client Kernel Version 4.18.0-147.6.el8.x86_64

DATA SERVER

Storage Type NVMe
Volatile Memory 384GB
Storage Interface NVMe
Network 4x25GbE
Software Version
OS Version CentOS Linux release 8.0.1905 (Core)

INFORMATION

Client Nodes 8
Client Total Procs 256
Metadata Nodes 8
Metadata Storage Devices 8
Data Nodes 8
Data Storage Devices 8

METADATA

Easy Write 108.68 kIOP/s
Easy Stat 501.63 kIOP/s
Easy Delete 116.08 kIOP/s
Hard Write 9.87 kIOP/s
Hard Read 139.25 kIOP/s
Hard Stat 127.89 kIOP/s
Hard Delete 36.33 kIOP/s

Submitted Files

ior-easy-read
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Wed Jul  8 18:16:33 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-easy/stonewall -t 2m -b 9920000m -F -r -R -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
Machine             : Linux o03
TestID              : 0
StartTime           : Wed Jul  8 18:16:33 2020
Path                : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-easy
FS                  : 428.3 TiB   Used FS: 14.0%   Inodes: 75.8 Mi   Used Inodes: 100.0%

Options: 
api                 : CEPHFS
apiVersion          : 
test filename       : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 8
tasks               : 256
clients per node    : 32
repetitions         : 1
xfersize            : 2 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 2421.88 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
WARNING: Expected aggregate file size       = 2662879723520000.
WARNING: Stat() of aggregate file size      = 29182155292672.
WARNING: Using actual aggregate bytes moved = 29182155292672.
read      72594      36330      0.000077    10158080000 2048.00    0.353169   383.02     0.001075   383.37     0   
Max Read:  72593.55 MiB/sec (76119.86 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
read        72593.55   72593.55   72593.55       0.00   36296.78   36296.78   36296.78       0.00  383.37112         NA            NA     0    256  32    1   1     1        1         0    0      1 10401873920000  2097152 27830272.0 CEPHFS      0
Finished            : Wed Jul  8 18:22:57 2020
ior-easy-write
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Wed Jul  8 17:48:16 2020
Command line        : ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-easy/stonewall -t 2m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
Machine             : Linux o03
TestID              : 0
StartTime           : Wed Jul  8 17:48:16 2020
Path                : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-easy
FS                  : 429.8 TiB   Used FS: 7.0%   Inodes: 7.5 Mi   Used Inodes: 100.0%

Options: 
api                 : CEPHFS
apiVersion          : 
test filename       : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-easy/ior_file_easy
access              : file-per-process
type                : independent
segments            : 1
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 8
tasks               : 256
clients per node    : 32
repetitions         : 1
xfersize            : 2 MiB
blocksize           : 9.46 TiB
aggregate filesize  : 2421.88 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 24162 max: 54356 -- min data: 47.2 GiB mean data: 80.2 GiB time: 301.5s
WARNING: Expected aggregate file size       = 2662879723520000.
WARNING: Stat() of aggregate file size      = 29182155292672.
WARNING: Using actual aggregate bytes moved = 29182155292672.
WARNING: maybe caused by deadlineForStonewalling
write     59254      29629      0.000070    10158080000 2048.00    0.028567   469.65     0.000449   469.68     0   
Max Write: 59254.27 MiB/sec (62132.61 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
write       59254.27   59254.27   59254.27       0.00   29627.14   29627.14   29627.14       0.00  469.67535     301.49      69704.21     0    256  32    1   1     1        1         0    0      1 10401873920000  2097152 27830272.0 CEPHFS      0
Finished            : Wed Jul  8 17:56:06 2020
ior-hard-read
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Wed Jul  8 18:25:00 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-hard/file -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 1000000 -r -R -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
Machine             : Linux o03
TestID              : 0
StartTime           : Wed Jul  8 18:25:00 2020
Path                : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-hard
FS                  : 428.3 TiB   Used FS: 14.0%   Inodes: 75.8 Mi   Used Inodes: 100.0%

Options: 
api                 : CEPHFS
apiVersion          : 
test filename       : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 1000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 8
tasks               : 256
clients per node    : 32
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 10.94 TiB

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 304385 max: 304385 -- min data: 13.3 GiB mean data: 13.3 GiB time: 242.1s
WARNING: Expected aggregate file size       = 12034048000000.
WARNING: Stat() of aggregate file size      = 3662983700480.
WARNING: Using actual aggregate bytes moved = 3662983700480.
read      14420      321832     242.12      45.91      45.91      0.139184   242.12     0.001834   242.26     0   
Max Read:  14419.55 MiB/sec (15119.99 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
read        14419.55   14419.55   14419.55       0.00  321647.18  321647.18  321647.18       0.00  242.26098     242.11      14428.42     0    256  32    1   0     1        1         0    0 1000000    47008    47008 3493293.5 CEPHFS      0
Finished            : Wed Jul  8 18:29:03 2020
ior-hard-write
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began               : Wed Jul  8 18:05:32 2020
Command line        : ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-hard/file -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 1000000 -w -D 300 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
Machine             : Linux o03
TestID              : 0
StartTime           : Wed Jul  8 18:05:32 2020
Path                : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-hard
FS                  : 428.5 TiB   Used FS: 13.2%   Inodes: 69.9 Mi   Used Inodes: 100.0%

Options: 
api                 : CEPHFS
apiVersion          : 
test filename       : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-hard/file
access              : single-shared-file
type                : independent
segments            : 1000000
ordering in a file  : sequential
ordering inter file : constant task offset
task offset         : 1
nodes               : 8
tasks               : 256
clients per node    : 32
repetitions         : 1
xfersize            : 47008 bytes
blocksize           : 47008 bytes
aggregate filesize  : 10.94 TiB
stonewallingTime    : 300
stoneWallingWearOut : 1

Results: 

access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
stonewalling pairs accessed min: 303763 max: 304385 -- min data: 13.3 GiB mean data: 13.3 GiB time: 300.2s
WARNING: Expected aggregate file size       = 12034048000000.
WARNING: Stat() of aggregate file size      = 3662983700480.
WARNING: Using actual aggregate bytes moved = 3662983700480.
WARNING: maybe caused by deadlineForStonewalling
write     11595      258953     300.64      45.91      45.91      0.352667   300.91     0.000299   301.27     0   
Max Write: 11595.36 MiB/sec (12158.61 MB/sec)

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
write       11595.36   11595.36   11595.36       0.00  258649.86  258649.86  258649.86       0.00  301.26659     300.21      11616.82     0    256  32    1   0     1        1         0    0 1000000    47008    47008 3493293.5 CEPHFS      0
Finished            : Wed Jul  8 18:10:34 2020
mdtest-easy-delete
-- started at 07/08/2020 18:29:27 --

mdtest-3.3.0+dev was launched with 256 total task(s) on 8 node(s)
Command line used: ./mdtest '-n' '240000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-easy' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-easy-stonewall' '-r' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app
FS: 428.3 TiB   Used FS: 14.0%   Inodes: 0.0 Mi   Used Inodes: 100.0%

Nodemap: 1111111111111111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 32 for each phase.
256 tasks, 61440000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :     116075.472     116075.218     116075.346          0.060
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          7.139          7.139          7.139          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :        529.312        529.311        529.311          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.140          0.140          0.140          0.000
-- finished at 07/08/2020 18:38:19 --

mdtest-easy-stat
-- started at 07/08/2020 18:22:57 --

mdtest-3.3.0+dev was launched with 256 total task(s) on 8 node(s)
Command line used: ./mdtest '-n' '240000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-easy' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-easy-stonewall' '-T' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app
FS: 428.3 TiB   Used FS: 14.0%   Inodes: 0.0 Mi   Used Inodes: 100.0%

Nodemap: 1111111111111111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 32 for each phase.
256 tasks, 61440000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :     501629.721     501623.574     501626.684          1.220
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :        122.482        122.481        122.482          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/08/2020 18:24:59 --

mdtest-easy-write
-- started at 07/08/2020 17:56:06 --

mdtest-3.3.0+dev was launched with 256 total task(s) on 8 node(s)
Command line used: ./mdtest '-n' '240000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-easy' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-easy-stonewall' '-C' '-Y' '-W' '300' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app
FS: 428.6 TiB   Used FS: 13.2%   Inodes: 0.0 Mi   Used Inodes: 100.0%

Nodemap: 1111111111111111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 32 for each phase.
256 tasks, 61440000 files
Continue stonewall hit min: 154805 max: 240000 avg: 234139.8 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :     108678.990     108678.892     108678.951          0.022
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA     199687.399             NA
   Tree creation             :         21.681         21.681         21.681          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        565.335        565.335        565.335          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        300.168             NA
   Tree creation             :          0.046          0.046          0.046          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/08/2020 18:05:32 --

mdtest-hard-delete
-- started at 07/08/2020 18:39:13 --

mdtest-3.3.0+dev was launched with 256 total task(s) on 8 node(s)
Command line used: ./mdtest '-n' '100000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-hard' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-hard-stonewall' '-r' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app
FS: 428.3 TiB   Used FS: 14.0%   Inodes: 54.8 Mi   Used Inodes: 100.0%

Nodemap: 1111111111111111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 32 for each phase.
256 tasks, 25600000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :      36327.352      36326.942      36327.136          0.098
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          6.504          6.504          6.504          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :         84.763         84.762         84.762          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.154          0.154          0.154          0.000
-- finished at 07/08/2020 18:40:38 --

mdtest-hard-read
-- started at 07/08/2020 18:38:49 --

mdtest-3.3.0+dev was launched with 256 total task(s) on 8 node(s)
Command line used: ./mdtest '-n' '100000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-hard' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-hard-stonewall' '-E' '-X' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app
FS: 428.3 TiB   Used FS: 14.0%   Inodes: 54.8 Mi   Used Inodes: 100.0%

Nodemap: 1111111111111111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 32 for each phase.
256 tasks, 25600000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :     139259.902     139252.824     139256.253          1.639
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :         22.112         22.111         22.112          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/08/2020 18:39:12 --

mdtest-hard-stat
-- started at 07/08/2020 18:29:03 --

mdtest-3.3.0+dev was launched with 256 total task(s) on 8 node(s)
Command line used: ./mdtest '-n' '100000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-hard' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-hard-stonewall' '-T' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app
FS: 428.3 TiB   Used FS: 14.0%   Inodes: 0.0 Mi   Used Inodes: 100.0%

Nodemap: 1111111111111111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 32 for each phase.
256 tasks, 25600000 files

SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :     127888.103     127884.534     127886.467          0.789
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :          0.000          0.000          0.000          0.000
   File stat                 :         24.078         24.077         24.077          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   Tree creation             :          0.000          0.000          0.000          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/08/2020 18:29:27 --

mdtest-hard-write
-- started at 07/08/2020 18:10:34 --

mdtest-3.3.0+dev was launched with 256 total task(s) on 8 node(s)
Command line used: ./mdtest '-n' '100000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-hard' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-hard-stonewall' '-C' '-Y' '-W' '300' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app
FS: 428.3 TiB   Used FS: 14.0%   Inodes: 0.0 Mi   Used Inodes: 100.0%

Nodemap: 1111111111111111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank   0 Line  2166 Shifting ranks by 32 for each phase.
256 tasks, 25600000 files
Continue stonewall hit min: 9006 max: 12028 avg: 10452.5 


SUMMARY rate: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :       9873.110       9873.105       9873.107          0.001
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA       8921.469             NA
   Tree creation             :          9.001          9.001          9.001          0.000
   Tree removal              :          0.000          0.000          0.000          0.000

SUMMARY time: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation             :        311.874        311.874        311.874          0.000
   File stat                 :          0.000          0.000          0.000          0.000
   File read                 :          0.000          0.000          0.000          0.000
   File removal              :          0.000          0.000          0.000          0.000
   File create (stonewall)   :             NA             NA        299.932             NA
   Tree creation             :          0.111          0.111          0.111          0.000
   Tree removal              :          0.000          0.000          0.000          0.000
-- finished at 07/08/2020 18:15:46 --

result
version         = io500-isc20_v4
config-hash     = 27ECB143
result-dir      = ./results/2020.07.08-16.53.40-app
; START 2020-07-08 17:48:16


[ior-easy-write]
t_start         = 2020-07-08 17:48:16
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-easy/stonewall -t 2m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
throughput-stonewall = 11185.87
score           = 57.865502
t_delta         = 469.9090
t_end           = 2020-07-08 17:56:06

[mdtest-easy-write]
t_start         = 2020-07-08 17:56:06
exe             = ./mdtest -n 240000 -u -L -F -P -N 1 -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-easy -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-easy-stonewall -C -Y -W 300 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
rate-stonewall  = 199.687399
score           = 108.678956
t_delta         = 565.7802
t_end           = 2020-07-08 18:05:32

[timestamp]
t_start         = 2020-07-08 18:05:32
t_delta         = 0.0053
t_end           = 2020-07-08 18:05:32

[ior-hard-write]
t_start         = 2020-07-08 18:05:32
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-hard/file -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 1000000 -w -D 300 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
throughput-stonewall = 2894.04
score           = 11.323590
t_delta         = 301.7669
t_end           = 2020-07-08 18:10:34

[mdtest-hard-write]
t_start         = 2020-07-08 18:10:34
exe             = ./mdtest -n 100000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-hard -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-hard-stonewall -C -Y -W 300 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
rate-stonewall  = 8.921469
score           = 9.873107
t_delta         = 312.5518
t_end           = 2020-07-08 18:15:46

[find]
t_start         = 2020-07-08 18:15:46
exe             = ./pfind /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app -newer /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/timestampfile -size 3901c -name *01* -C -q 10000
found           = 110162
total-files     = 64519430
score           = 1394.901467
t_delta         = 46.2558
t_end           = 2020-07-08 18:16:33

[ior-easy-read]
t_start         = 2020-07-08 18:16:33
exe             = ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-easy/stonewall -t 2m -b 9920000m -F -r -R -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score           = 70.892143
t_delta         = 384.0279
t_end           = 2020-07-08 18:22:57

[mdtest-easy-stat]
t_start         = 2020-07-08 18:22:57
exe             = ./mdtest -n 240000 -u -L -F -P -N 1 -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-easy -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-easy-stonewall -T -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score           = 501.629537
t_delta         = 122.8602
t_end           = 2020-07-08 18:25:00

[ior-hard-read]
t_start         = 2020-07-08 18:25:00
exe             = ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-hard/file -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 1000000 -r -R -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score           = 14.081589
t_delta         = 242.8458
t_end           = 2020-07-08 18:29:03

[mdtest-hard-stat]
t_start         = 2020-07-08 18:29:03
exe             = ./mdtest -n 100000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-hard -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-hard-stonewall -T -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score           = 127.887913
t_delta         = 24.4290
t_end           = 2020-07-08 18:29:27

[mdtest-easy-delete]
t_start         = 2020-07-08 18:29:27
exe             = ./mdtest -n 240000 -u -L -F -P -N 1 -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-easy -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-easy-stonewall -r -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score           = 116.075434
t_delta         = 562.2122
t_end           = 2020-07-08 18:38:49

[mdtest-hard-read]
t_start         = 2020-07-08 18:38:49
exe             = ./mdtest -n 100000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-hard -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-hard-stonewall -E -X -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score           = 139.253021
t_delta         = 22.5978
t_end           = 2020-07-08 18:39:12

[mdtest-hard-delete]
t_start         = 2020-07-08 18:39:12
exe             = ./mdtest -n 100000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-hard -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-16.53.40-app/mdtest-hard-stonewall -r -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score           = 36.327299
t_delta         = 85.2415
t_end           = 2020-07-08 18:40:38

[SCORE]
MD              = 124.134267
BW              = 28.438940
SCORE           = 59.415882 
hash            = E9426B44
; END 2020-07-08 18:41:18
result_summary
IO500 version io500-isc20_v4
[RESULT]       ior-easy-write       57.865502 GiB/s  : time 469.909 seconds
[RESULT]    mdtest-easy-write      108.678956 kIOPS : time 565.780 seconds
[RESULT]       ior-hard-write       11.323590 GiB/s  : time 301.767 seconds
[RESULT]    mdtest-hard-write        9.873107 kIOPS : time 312.552 seconds
[RESULT]                 find     1394.901467 kIOPS : time 46.256 seconds
[RESULT]        ior-easy-read       70.892143 GiB/s  : time 384.028 seconds
[RESULT]     mdtest-easy-stat      501.629537 kIOPS : time 122.860 seconds
[RESULT]        ior-hard-read       14.081589 GiB/s  : time 242.846 seconds
[RESULT]     mdtest-hard-stat      127.887913 kIOPS : time 24.429 seconds
[RESULT]   mdtest-easy-delete      116.075434 kIOPS : time 562.212 seconds
[RESULT]     mdtest-hard-read      139.253021 kIOPS : time 22.598 seconds
[RESULT]   mdtest-hard-delete       36.327299 kIOPS : time 85.242 seconds
[SCORE] Bandwidth 28.438940 GB/s : IOPS 124.134267 kiops : TOTAL 59.415882