- ior-easy-read
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Tue Jul 7 20:55:25 2020
Command line : ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-easy/stonewall -t 2m -b 9920000m -F -r -R -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
Machine : Linux io500-c1
TestID : 0
StartTime : Tue Jul 7 20:55:25 2020
Path : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-easy
FS : 49.3 TiB Used FS: 35.0% Inodes: 93.1 Mi Used Inodes: 100.0%
Options:
api : CEPHFS
apiVersion :
test filename : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-easy/ior_file_easy
access : file-per-process
type : independent
segments : 1
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 10
tasks : 360
clients per node : 36
repetitions : 1
xfersize : 2 MiB
blocksize : 9.46 TiB
aggregate filesize : 3405.76 TiB
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 3744674611200000.
WARNING: Stat() of aggregate file size = 14241843118080.
WARNING: Using actual aggregate bytes moved = 14241843118080.
read 56326 28180 0.000049 10158080000 2048.00 0.138071 240.99 0.002234 241.13 0
Max Read: 56326.44 MiB/sec (59062.56 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
read 56326.44 56326.44 56326.44 0.00 28163.22 28163.22 28163.22 0.00 241.13150 NA NA 0 360 36 1 1 1 1 0 0 1 10401873920000 2097152 13582080.0 CEPHFS 0
Finished : Tue Jul 7 20:59:27 2020
- ior-easy-write
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Tue Jul 7 20:27:23 2020
Command line : ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-easy/stonewall -t 2m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
Machine : Linux io500-c1
TestID : 0
StartTime : Tue Jul 7 20:27:23 2020
Path : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-easy
FS : 55.2 TiB Used FS: 0.0% Inodes: 0.0 Mi Used Inodes: 100.0%
Options:
api : CEPHFS
apiVersion :
test filename : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-easy/ior_file_easy
access : file-per-process
type : independent
segments : 1
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 10
tasks : 360
clients per node : 36
repetitions : 1
xfersize : 2 MiB
blocksize : 9.46 TiB
aggregate filesize : 3405.76 TiB
stonewallingTime : 300
stoneWallingWearOut : 1
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
stonewalling pairs accessed min: 9772 max: 18864 -- min data: 19.1 GiB mean data: 28.4 GiB time: 305.0s
WARNING: Expected aggregate file size = 3744674611200000.
WARNING: Stat() of aggregate file size = 14241843118080.
WARNING: Using actual aggregate bytes moved = 14241843118080.
WARNING: maybe caused by deadlineForStonewalling
write 31494 15748 0.000077 10158080000 2048.00 0.029155 431.23 0.000900 431.26 0
Max Write: 31494.22 MiB/sec (33024.08 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
write 31494.22 31494.22 31494.22 0.00 15747.11 15747.11 15747.11 0.00 431.25627 304.97 34317.19 0 360 36 1 1 1 1 0 0 1 10401873920000 2097152 13582080.0 CEPHFS 0
Finished : Tue Jul 7 20:34:35 2020
- ior-hard-read
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Tue Jul 7 21:01:02 2020
Command line : ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-hard/file -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 1000000 -r -R -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
Machine : Linux io500-c1
TestID : 0
StartTime : Tue Jul 7 21:01:02 2020
Path : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-hard
FS : 49.3 TiB Used FS: 35.0% Inodes: 93.1 Mi Used Inodes: 100.0%
Options:
api : CEPHFS
apiVersion :
test filename : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-hard/file
access : single-shared-file
type : independent
segments : 1000000
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 10
tasks : 360
clients per node : 36
repetitions : 1
xfersize : 47008 bytes
blocksize : 47008 bytes
aggregate filesize : 15.39 TiB
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
stonewalling pairs accessed min: 277842 max: 277842 -- min data: 12.2 GiB mean data: 12.2 GiB time: 203.2s
WARNING: Expected aggregate file size = 16922880000000.
WARNING: Stat() of aggregate file size = 4701886824960.
WARNING: Using actual aggregate bytes moved = 4701886824960.
read 22065 492288 203.18 45.91 45.91 0.037140 203.18 0.001905 203.22 0
Max Read: 22065.32 MiB/sec (23137.17 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
read 22065.32 22065.32 22065.32 0.00 492196.37 492196.37 492196.37 0.00 203.21791 203.17 22070.59 0 360 36 1 0 1 1 0 0 1000000 47008 47008 4484068.5 CEPHFS 0
Finished : Tue Jul 7 21:04:25 2020
- ior-hard-write
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Tue Jul 7 20:42:11 2020
Command line : ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-hard/file -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 1000000 -w -D 300 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
Machine : Linux io500-c1
TestID : 0
StartTime : Tue Jul 7 20:42:12 2020
Path : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-hard
FS : 50.8 TiB Used FS: 25.5% Inodes: 82.9 Mi Used Inodes: 100.0%
Options:
api : CEPHFS
apiVersion :
test filename : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-hard/file
access : single-shared-file
type : independent
segments : 1000000
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 10
tasks : 360
clients per node : 36
repetitions : 1
xfersize : 47008 bytes
blocksize : 47008 bytes
aggregate filesize : 15.39 TiB
stonewallingTime : 300
stoneWallingWearOut : 1
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
stonewalling pairs accessed min: 215011 max: 277842 -- min data: 9.4 GiB mean data: 11.6 GiB time: 300.2s
WARNING: Expected aggregate file size = 16922880000000.
WARNING: Stat() of aggregate file size = 4701886824960.
WARNING: Using actual aggregate bytes moved = 4701886824960.
WARNING: maybe caused by deadlineForStonewalling
write 12197 272176 300.36 45.91 45.91 0.139823 367.49 0.001263 367.64 0
Max Write: 12197.06 MiB/sec (12789.54 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
write 12197.06 12197.06 12197.06 0.00 272071.59 272071.59 272071.59 0.00 367.63530 300.21 14185.42 0 360 36 1 0 1 1 0 0 1000000 47008 47008 4484068.5 CEPHFS 0
Finished : Tue Jul 7 20:48:20 2020
- mdtest-easy-delete
-
-- started at 07/07/2020 21:05:22 --
mdtest-3.3.0+dev was launched with 360 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '240000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-easy' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-easy-stonewall' '-r' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app
FS: 49.3 TiB Used FS: 35.0% Inodes: 88.8 Mi Used Inodes: 100.0%
Nodemap: 111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 36 for each phase.
360 tasks, 86400000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 121258.221 121257.954 121258.078 0.053
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 3.476 3.476 3.476 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 712.531 712.529 712.530 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.288 0.288 0.288 0.000
-- finished at 07/07/2020 21:17:15 --
- mdtest-easy-stat
-
-- started at 07/07/2020 20:59:27 --
mdtest-3.3.0+dev was launched with 360 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '240000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-easy' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-easy-stonewall' '-T' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app
FS: 49.3 TiB Used FS: 35.0% Inodes: 88.8 Mi Used Inodes: 100.0%
Nodemap: 111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 36 for each phase.
360 tasks, 86400000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 917719.130 917705.407 917710.613 2.398
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 94.148 94.146 94.147 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/07/2020 21:01:01 --
- mdtest-easy-write
-
-- started at 07/07/2020 20:34:35 --
mdtest-3.3.0+dev was launched with 360 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '240000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-easy' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-easy-stonewall' '-C' '-Y' '-W' '300' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app
FS: 50.9 TiB Used FS: 25.4% Inodes: 0.0 Mi Used Inodes: 100.0%
Nodemap: 111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 36 for each phase.
360 tasks, 86400000 files
Continue stonewall hit min: 110585 max: 240000 avg: 222666.1
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 189582.695 189582.285 189582.454 0.080
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 266719.536 NA
Tree creation : 8.243 8.243 8.243 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 455.739 455.738 455.738 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 300.540 NA
Tree creation : 0.121 0.121 0.121 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/07/2020 20:42:11 --
- mdtest-hard-delete
-
-- started at 07/07/2020 21:18:53 --
mdtest-3.3.0+dev was launched with 360 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '100000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-hard' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-hard-stonewall' '-r' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app
FS: 49.3 TiB Used FS: 35.0% Inodes: 88.8 Mi Used Inodes: 100.0%
Nodemap: 111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 36 for each phase.
360 tasks, 36000000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 11519.947 11519.906 11519.930 0.007
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 18.811 18.811 18.811 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 582.067 582.065 582.066 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.053 0.053 0.053 0.000
-- finished at 07/07/2020 21:28:36 --
- mdtest-hard-read
-
-- started at 07/07/2020 21:17:16 --
mdtest-3.3.0+dev was launched with 360 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '100000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-hard' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-hard-stonewall' '-E' '-X' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app
FS: 49.3 TiB Used FS: 35.0% Inodes: 88.8 Mi Used Inodes: 100.0%
Nodemap: 111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 36 for each phase.
360 tasks, 36000000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 68942.853 68941.830 68942.362 0.204
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 97.261 97.260 97.260 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/07/2020 21:18:53 --
- mdtest-hard-stat
-
-- started at 07/07/2020 21:04:25 --
mdtest-3.3.0+dev was launched with 360 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '100000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-hard' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-hard-stonewall' '-T' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app
FS: 49.3 TiB Used FS: 35.0% Inodes: 88.8 Mi Used Inodes: 100.0%
Nodemap: 111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 36 for each phase.
360 tasks, 36000000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 119173.042 119169.891 119171.465 0.476
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 56.267 56.266 56.266 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/07/2020 21:05:22 --
- mdtest-hard-write
-
-- started at 07/07/2020 20:48:20 --
mdtest-3.3.0+dev was launched with 360 total task(s) on 10 node(s)
Command line used: ./mdtest '-n' '100000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-hard' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-hard-stonewall' '-C' '-Y' '-W' '300' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app
FS: 49.3 TiB Used FS: 35.0% Inodes: 82.4 Mi Used Inodes: 100.0%
Nodemap: 111111111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 36 for each phase.
360 tasks, 36000000 files
Continue stonewall hit min: 14912 max: 18626 avg: 16865.4
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 20631.415 20631.364 20631.387 0.009
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 20110.220 NA
Tree creation : 547.490 547.490 547.490 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 325.008 325.007 325.008 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 301.914 NA
Tree creation : 0.002 0.002 0.002 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/07/2020 20:53:45 --
- result
-
version = io500-isc20_v4
config-hash = CE64F3BB
result-dir = /home/fedora/io500-app/results/2020.07.07-20.27.19-app
; START 2020-07-07 20:27:23
[ior-easy-write]
t_start = 2020-07-07 20:27:23
exe = ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-easy/stonewall -t 2m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
throughput-stonewall = 8531.70
score = 30.756075
t_delta = 431.6146
t_end = 2020-07-07 20:34:35
[mdtest-easy-write]
t_start = 2020-07-07 20:34:35
exe = ./mdtest -n 240000 -u -L -F -P -N 1 -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-easy -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-easy-stonewall -C -Y -W 300 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
rate-stonewall = 266.719536
score = 189.582331
t_delta = 456.6563
t_end = 2020-07-07 20:42:11
[timestamp]
t_start = 2020-07-07 20:42:11
t_delta = 0.0005
t_end = 2020-07-07 20:42:11
[ior-hard-write]
t_start = 2020-07-07 20:42:11
exe = ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-hard/file -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 1000000 -w -D 300 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
throughput-stonewall = 4072.38
score = 11.911189
t_delta = 368.3363
t_end = 2020-07-07 20:48:20
[mdtest-hard-write]
t_start = 2020-07-07 20:48:20
exe = ./mdtest -n 100000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-hard -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-hard-stonewall -C -Y -W 300 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
rate-stonewall = 20.110220
score = 20.631366
t_delta = 325.8275
t_end = 2020-07-07 20:53:46
[find]
t_start = 2020-07-07 20:53:46
exe = ./pfind /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app -newer /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/timestampfile -size 3901c -name *01* -C -q 10000
found = 221883
total-files = 93105726
score = 938.666920
t_delta = 99.1907
t_end = 2020-07-07 20:55:25
[ior-easy-read]
t_start = 2020-07-07 20:55:25
exe = ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-easy/stonewall -t 2m -b 9920000m -F -r -R -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 55.006292
t_delta = 241.9220
t_end = 2020-07-07 20:59:27
[mdtest-easy-stat]
t_start = 2020-07-07 20:59:27
exe = ./mdtest -n 240000 -u -L -F -P -N 1 -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-easy -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-easy-stonewall -T -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 917.709973
t_delta = 94.7418
t_end = 2020-07-07 21:01:02
[ior-hard-read]
t_start = 2020-07-07 21:01:02
exe = ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-hard/file -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 1000000 -r -R -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 21.548166
t_delta = 203.4786
t_end = 2020-07-07 21:04:25
[mdtest-hard-stat]
t_start = 2020-07-07 21:04:25
exe = ./mdtest -n 100000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-hard -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-hard-stonewall -T -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 119.171335
t_delta = 56.8966
t_end = 2020-07-07 21:05:22
[mdtest-easy-delete]
t_start = 2020-07-07 21:05:22
exe = ./mdtest -n 240000 -u -L -F -P -N 1 -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-easy -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-easy-stonewall -r -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 121.258101
t_delta = 713.0408
t_end = 2020-07-07 21:17:15
[mdtest-hard-read]
t_start = 2020-07-07 21:17:15
exe = ./mdtest -n 100000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-hard -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-hard-stonewall -E -X -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 68.942548
t_delta = 97.6661
t_end = 2020-07-07 21:18:53
[mdtest-hard-delete]
t_start = 2020-07-07 21:18:53
exe = ./mdtest -n 100000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-hard -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.07-20.27.19-app/mdtest-hard-stonewall -r -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 11.519936
t_delta = 582.5288
t_end = 2020-07-07 21:28:36
[SCORE]
MD = 118.418715
BW = 25.670077
SCORE = 55.134541
hash = F9CE3A4
; END 2020-07-07 21:29:50
- result_summary
-
IO500 version io500-isc20_v4
[RESULT] ior-easy-write 30.756075 GiB/s : time 431.615 seconds
[RESULT] mdtest-easy-write 189.582331 kIOPS : time 456.656 seconds
[RESULT] ior-hard-write 11.911189 GiB/s : time 368.336 seconds
[RESULT] mdtest-hard-write 20.631366 kIOPS : time 325.828 seconds
[RESULT] find 938.666920 kIOPS : time 99.191 seconds
[RESULT] ior-easy-read 55.006292 GiB/s : time 241.922 seconds
[RESULT] mdtest-easy-stat 917.709973 kIOPS : time 94.742 seconds
[RESULT] ior-hard-read 21.548166 GiB/s : time 203.479 seconds
[RESULT] mdtest-hard-stat 119.171335 kIOPS : time 56.897 seconds
[RESULT] mdtest-easy-delete 121.258101 kIOPS : time 713.041 seconds
[RESULT] mdtest-hard-read 68.942548 kIOPS : time 97.666 seconds
[RESULT] mdtest-hard-delete 11.519936 kIOPS : time 582.529 seconds
[SCORE] Bandwidth 25.670077 GB/s : IOPS 118.418715 kiops : TOTAL 55.134541