- ior-easy-read
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Mon Jul 13 19:22:23 2020
Command line : ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-easy/stonewall -t 2m -b 9920000m -F -r -R -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
Machine : Linux o01
TestID : 0
StartTime : Mon Jul 13 19:22:23 2020
Path : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-easy
FS : 430.5 TiB Used FS: 3.7% Inodes: 52.1 Mi Used Inodes: 100.0%
Options:
api : CEPHFS
apiVersion :
test filename : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-easy/ior_file_easy
access : file-per-process
type : independent
segments : 1
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 1
tasks : 112
clients per node : 112
repetitions : 1
xfersize : 2 MiB
blocksize : 9.46 TiB
aggregate filesize : 1059.57 TiB
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1165009879040000.
WARNING: Stat() of aggregate file size = 6119120437248.
WARNING: Using actual aggregate bytes moved = 6119120437248.
read 9932 4966 0.000118 10158080000 2048.00 0.064065 587.52 0.039420 587.56 0
Max Read: 9931.93 MiB/sec (10414.38 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
read 9931.93 9931.93 9931.93 0.00 4965.97 4965.97 4965.97 0.00 587.56430 NA NA 0 112 112 1 1 1 1 0 0 1 10401873920000 2097152 5835648.0 CEPHFS 0
Finished : Mon Jul 13 19:32:13 2020
- ior-easy-write
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Mon Jul 13 18:36:11 2020
Command line : ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-easy/stonewall -t 2m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
Machine : Linux o01
TestID : 0
StartTime : Mon Jul 13 18:36:11 2020
Path : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-easy
FS : 430.8 TiB Used FS: 1.9% Inodes: 2.1 Mi Used Inodes: 100.0%
Options:
api : CEPHFS
apiVersion :
test filename : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-easy/ior_file_easy
access : file-per-process
type : independent
segments : 1
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 1
tasks : 112
clients per node : 112
repetitions : 1
xfersize : 2 MiB
blocksize : 9.46 TiB
aggregate filesize : 1059.57 TiB
stonewallingTime : 300
stoneWallingWearOut : 1
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
stonewalling pairs accessed min: 5984 max: 26052 -- min data: 11.7 GiB mean data: 27.8 GiB time: 301.2s
WARNING: Expected aggregate file size = 1165009879040000.
WARNING: Stat() of aggregate file size = 6119120437248.
WARNING: Using actual aggregate bytes moved = 6119120437248.
WARNING: maybe caused by deadlineForStonewalling
write 10458 5230 0.000111 10158080000 2048.00 0.040895 557.94 0.025561 557.98 0
Max Write: 10458.48 MiB/sec (10966.52 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
write 10458.48 10458.48 10458.48 0.00 5229.24 5229.24 5229.24 0.00 557.98216 301.15 10591.74 0 112 112 1 1 1 1 0 0 1 10401873920000 2097152 5835648.0 CEPHFS 0
Finished : Mon Jul 13 18:45:30 2020
- ior-hard-read
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Mon Jul 13 19:37:14 2020
Command line : ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-hard/file -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 1000000 -r -R -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
Machine : Linux o01
TestID : 0
StartTime : Mon Jul 13 19:37:14 2020
Path : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-hard
FS : 430.5 TiB Used FS: 3.7% Inodes: 52.1 Mi Used Inodes: 100.0%
Options:
api : CEPHFS
apiVersion :
test filename : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-hard/file
access : single-shared-file
type : independent
segments : 1000000
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 1
tasks : 112
clients per node : 112
repetitions : 1
xfersize : 47008 bytes
blocksize : 47008 bytes
aggregate filesize : 4.79 TiB
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
stonewalling pairs accessed min: 436245 max: 436245 -- min data: 19.1 GiB mean data: 19.1 GiB time: 303.1s
WARNING: Expected aggregate file size = 5264896000000.
WARNING: Stat() of aggregate file size = 2296784555520.
WARNING: Using actual aggregate bytes moved = 2296784555520.
read 7224 161161 303.14 45.91 45.91 0.061223 303.17 0.038975 303.21 0
Max Read: 7223.91 MiB/sec (7574.82 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
read 7223.91 7223.91 7223.91 0.00 161138.95 161138.95 161138.95 0.00 303.21309 303.13 7225.91 0 112 112 1 0 1 1 0 0 1000000 47008 47008 2190384.5 CEPHFS 0
Finished : Mon Jul 13 19:42:18 2020
- ior-hard-write
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Mon Jul 13 18:53:00 2020
Command line : ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-hard/file -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 1000000 -w -D 300 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
Machine : Linux o01
TestID : 0
StartTime : Mon Jul 13 18:53:00 2020
Path : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-hard
FS : 430.6 TiB Used FS: 3.2% Inodes: 43.4 Mi Used Inodes: 100.0%
Options:
api : CEPHFS
apiVersion :
test filename : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-hard/file
access : single-shared-file
type : independent
segments : 1000000
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 1
tasks : 112
clients per node : 112
repetitions : 1
xfersize : 47008 bytes
blocksize : 47008 bytes
aggregate filesize : 4.79 TiB
stonewallingTime : 300
stoneWallingWearOut : 1
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
stonewalling pairs accessed min: 327291 max: 436245 -- min data: 14.3 GiB mean data: 17.8 GiB time: 300.0s
WARNING: Expected aggregate file size = 5264896000000.
WARNING: Stat() of aggregate file size = 2296784555520.
WARNING: Using actual aggregate bytes moved = 2296784555520.
WARNING: maybe caused by deadlineForStonewalling
write 5613 125257 300.20 45.91 45.91 0.147916 390.07 0.038964 390.24 0
Max Write: 5612.96 MiB/sec (5885.62 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
write 5612.96 5612.96 5612.96 0.00 125204.64 125204.64 125204.64 0.00 390.23665 300.03 6817.85 0 112 112 1 0 1 1 0 0 1000000 47008 47008 2190384.5 CEPHFS 0
Finished : Mon Jul 13 18:59:31 2020
- mdtest-easy-delete
-
-- started at 07/13/2020 19:44:34 --
mdtest-3.3.0+dev was launched with 112 total task(s) on 1 node(s)
Command line used: ./mdtest '-n' '400000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-easy' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-easy-stonewall' '-r' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app
FS: 430.5 TiB Used FS: 3.7% Inodes: 1.3 Mi Used Inodes: 100.0%
Nodemap: 1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
V-0: Rank 0 Line 2166 Shifting ranks by 1 for each phase.
112 tasks, 44800000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 61975.341 61975.312 61975.331 0.006
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 5.297 5.297 5.297 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 722.868 722.868 722.868 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.189 0.189 0.189 0.000
-- finished at 07/13/2020 19:56:37 --
- mdtest-easy-stat
-
-- started at 07/13/2020 19:32:27 --
mdtest-3.3.0+dev was launched with 112 total task(s) on 1 node(s)
Command line used: ./mdtest '-n' '400000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-easy' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-easy-stonewall' '-T' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app
FS: 430.5 TiB Used FS: 3.7% Inodes: 1.3 Mi Used Inodes: 100.0%
Nodemap: 1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
V-0: Rank 0 Line 2166 Shifting ranks by 1 for each phase.
112 tasks, 44800000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 165348.982 165348.828 165348.873 0.027
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 270.942 270.942 270.942 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/13/2020 19:36:59 --
- mdtest-easy-write
-
-- started at 07/13/2020 18:45:44 --
mdtest-3.3.0+dev was launched with 112 total task(s) on 1 node(s)
Command line used: ./mdtest '-n' '400000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-easy' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-easy-stonewall' '-C' '-Y' '-W' '300' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app
FS: 430.6 TiB Used FS: 3.2% Inodes: 1.3 Mi Used Inodes: 100.0%
Nodemap: 1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
V-0: Rank 0 Line 2166 Shifting ranks by 1 for each phase.
112 tasks, 44800000 files
Continue stonewall hit min: 257523 max: 400000 avg: 379868.2
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 110705.063 110704.856 110704.957 0.048
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 141773.388 NA
Tree creation : 21.849 21.849 21.849 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 404.680 404.679 404.679 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 300.093 NA
Tree creation : 0.046 0.046 0.046 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/13/2020 18:52:28 --
- mdtest-hard-delete
-
-- started at 07/13/2020 19:59:37 --
mdtest-3.3.0+dev was launched with 112 total task(s) on 1 node(s)
Command line used: ./mdtest '-n' '100000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-hard' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-hard-stonewall' '-r' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app
FS: 430.5 TiB Used FS: 3.7% Inodes: 1.3 Mi Used Inodes: 100.0%
Nodemap: 1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
V-0: Rank 0 Line 2166 Shifting ranks by 1 for each phase.
112 tasks, 11200000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 9862.129 9862.126 9862.128 0.001
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 2.574 2.574 2.574 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 577.630 577.629 577.629 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.389 0.389 0.389 0.000
-- finished at 07/13/2020 20:09:16 --
- mdtest-hard-read
-
-- started at 07/13/2020 19:57:09 --
mdtest-3.3.0+dev was launched with 112 total task(s) on 1 node(s)
Command line used: ./mdtest '-n' '100000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-hard' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-hard-stonewall' '-E' '-X' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app
FS: 430.5 TiB Used FS: 3.7% Inodes: 1.3 Mi Used Inodes: 100.0%
Nodemap: 1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
V-0: Rank 0 Line 2166 Shifting ranks by 1 for each phase.
112 tasks, 11200000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 43050.590 43050.195 43050.565 0.038
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 132.326 132.325 132.325 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/13/2020 19:59:21 --
- mdtest-hard-stat
-
-- started at 07/13/2020 19:42:33 --
mdtest-3.3.0+dev was launched with 112 total task(s) on 1 node(s)
Command line used: ./mdtest '-n' '100000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-hard' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-hard-stonewall' '-T' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app
FS: 430.5 TiB Used FS: 3.7% Inodes: 1.3 Mi Used Inodes: 100.0%
Nodemap: 1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
V-0: Rank 0 Line 2166 Shifting ranks by 1 for each phase.
112 tasks, 11200000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 53904.366 53904.325 53904.348 0.009
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 105.681 105.681 105.681 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/13/2020 19:44:19 --
- mdtest-hard-write
-
-- started at 07/13/2020 18:59:47 --
mdtest-3.3.0+dev was launched with 112 total task(s) on 1 node(s)
Command line used: ./mdtest '-n' '100000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-hard' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-hard-stonewall' '-C' '-Y' '-W' '300' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app
FS: 430.5 TiB Used FS: 3.7% Inodes: 1.3 Mi Used Inodes: 100.0%
Nodemap: 1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
V-0: Rank 0 Line 2166 Shifting ranks by 1 for each phase.
112 tasks, 11200000 files
Continue stonewall hit min: 42887 max: 50863 avg: 46445.6
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 16861.041 16860.798 16861.037 0.023
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 17343.488 NA
Tree creation : 12.777 12.777 12.777 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 337.864 337.859 337.859 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 299.935 NA
Tree creation : 0.078 0.078 0.078 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/13/2020 19:05:25 --
- result
-
version = io500-isc20_v4
config-hash = EF4BAF31
result-dir = ./results/2020.07.13-17.30.55-app
; START 2020-07-13 18:35:56
[ior-easy-write]
t_start = 2020-07-13 18:36:10
exe = ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-easy/stonewall -t 2m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
throughput-stonewall = 625.25
score = 10.213364
t_delta = 559.0477
t_end = 2020-07-13 18:45:30
[mdtest-easy-write]
t_start = 2020-07-13 18:45:43
exe = ./mdtest -n 400000 -u -L -F -P -N 1 -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-easy -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-easy-stonewall -C -Y -W 300 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
rate-stonewall = 141.773388
score = 110.704940
t_delta = 405.4338
t_end = 2020-07-13 18:52:29
[timestamp]
t_start = 2020-07-13 18:52:44
t_delta = 0.0026
t_end = 2020-07-13 18:52:44
[ior-hard-write]
t_start = 2020-07-13 18:53:00
exe = ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-hard/file -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 1000000 -w -D 300 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
throughput-stonewall = 573.33
score = 5.481411
t_delta = 390.6569
t_end = 2020-07-13 18:59:31
[mdtest-hard-write]
t_start = 2020-07-13 18:59:47
exe = ./mdtest -n 100000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-hard -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-hard-stonewall -C -Y -W 300 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
rate-stonewall = 17.343488
score = 16.861038
t_delta = 338.5948
t_end = 2020-07-13 19:05:26
[find]
t_start = 2020-07-13 19:05:40
exe = ./pfind /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app -newer /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/timestampfile -size 3901c -name *01* -C -q 10000
found = 217696
total-files = 50496774
score = 266.313303
t_delta = 189.7446
t_end = 2020-07-13 19:08:50
[ior-easy-read]
t_start = 2020-07-13 19:22:23
exe = ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-easy/stonewall -t 2m -b 9920000m -F -r -R -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 9.699151
t_delta = 590.2061
t_end = 2020-07-13 19:32:13
[mdtest-easy-stat]
t_start = 2020-07-13 19:32:27
exe = ./mdtest -n 400000 -u -L -F -P -N 1 -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-easy -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-easy-stonewall -T -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 165.348980
t_delta = 271.7093
t_end = 2020-07-13 19:36:59
[ior-hard-read]
t_start = 2020-07-13 19:37:14
exe = ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-hard/file -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 1000000 -r -R -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 7.054601
t_delta = 304.0222
t_end = 2020-07-13 19:42:18
[mdtest-hard-stat]
t_start = 2020-07-13 19:42:33
exe = ./mdtest -n 100000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-hard -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-hard-stonewall -T -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 53.904333
t_delta = 106.1832
t_end = 2020-07-13 19:44:19
[mdtest-easy-delete]
t_start = 2020-07-13 19:44:34
exe = ./mdtest -n 400000 -u -L -F -P -N 1 -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-easy -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-easy-stonewall -r -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 61.975327
t_delta = 723.5306
t_end = 2020-07-13 19:56:37
[mdtest-hard-read]
t_start = 2020-07-13 19:57:09
exe = ./mdtest -n 100000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-hard -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-hard-stonewall -E -X -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 43.050559
t_delta = 132.9221
t_end = 2020-07-13 19:59:21
[mdtest-hard-delete]
t_start = 2020-07-13 19:59:37
exe = ./mdtest -n 100000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-hard -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.13-17.30.55-app/mdtest-hard-stonewall -r -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 9.862129
t_delta = 578.7045
t_end = 2020-07-13 20:09:16
[SCORE]
MD = 57.323105
BW = 7.867139
SCORE = 21.236027
hash = 4968A7E0
; END 2020-07-13 20:09:46
- result_summary
-
IO500 version io500-isc20_v4
[RESULT] ior-easy-write 10.213364 GiB/s : time 559.048 seconds
[RESULT] mdtest-easy-write 110.704940 kIOPS : time 405.434 seconds
[RESULT] ior-hard-write 5.481411 GiB/s : time 390.657 seconds
[RESULT] mdtest-hard-write 16.861038 kIOPS : time 338.595 seconds
[RESULT] find 266.313303 kIOPS : time 189.745 seconds
[RESULT] ior-easy-read 9.699151 GiB/s : time 590.206 seconds
[RESULT] mdtest-easy-stat 165.348980 kIOPS : time 271.709 seconds
[RESULT] ior-hard-read 7.054601 GiB/s : time 304.022 seconds
[RESULT] mdtest-hard-stat 53.904333 kIOPS : time 106.183 seconds
[RESULT] mdtest-easy-delete 61.975327 kIOPS : time 723.531 seconds
[RESULT] mdtest-hard-read 43.050559 kIOPS : time 132.922 seconds
[RESULT] mdtest-hard-delete 9.862129 kIOPS : time 578.704 seconds
[SCORE] Bandwidth 7.867139 GB/s : IOPS 57.323105 kiops : TOTAL 21.236027