- ior-easy-read
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Wed Jul 8 13:58:06 2020
Command line : ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-easy/stonewall -t 2m -b 9920000m -F -r -R -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
Machine : Linux o03
TestID : 0
StartTime : Wed Jul 8 13:58:06 2020
Path : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-easy
FS : 429.8 TiB Used FS: 7.0% Inodes: 68.6 Mi Used Inodes: 100.0%
Options:
api : CEPHFS
apiVersion :
test filename : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-easy/ior_file_easy
access : file-per-process
type : independent
segments : 1
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 8
tasks : 256
clients per node : 32
repetitions : 1
xfersize : 2 MiB
blocksize : 9.46 TiB
aggregate filesize : 2421.88 TiB
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2662879723520000.
WARNING: Stat() of aggregate file size = 29430189654016.
WARNING: Using actual aggregate bytes moved = 29430189654016.
read 72503 36289 0.000078 10158080000 2048.00 0.396472 386.71 0.001542 387.11 0
Max Read: 72503.44 MiB/sec (76025.36 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
read 72503.44 72503.44 72503.44 0.00 36251.72 36251.72 36251.72 0.00 387.11015 NA NA 0 256 32 1 1 1 1 0 0 1 10401873920000 2097152 28066816.0 CEPHFS 0
Finished : Wed Jul 8 14:04:34 2020
- ior-easy-write
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Wed Jul 8 13:33:03 2020
Command line : ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-easy/stonewall -t 2m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
Machine : Linux o03
TestID : 0
StartTime : Wed Jul 8 13:33:03 2020
Path : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-easy
FS : 430.7 TiB Used FS: 0.0% Inodes: 0.0 Mi Used Inodes: 100.0%
Options:
api : CEPHFS
apiVersion :
test filename : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-easy/ior_file_easy
access : file-per-process
type : independent
segments : 1
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 8
tasks : 256
clients per node : 32
repetitions : 1
xfersize : 2 MiB
blocksize : 9.46 TiB
aggregate filesize : 2421.88 TiB
stonewallingTime : 300
stoneWallingWearOut : 1
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
stonewalling pairs accessed min: 22244 max: 54818 -- min data: 43.4 GiB mean data: 81.2 GiB time: 302.4s
WARNING: Expected aggregate file size = 2662879723520000.
WARNING: Stat() of aggregate file size = 29430189654016.
WARNING: Using actual aggregate bytes moved = 29430189654016.
WARNING: maybe caused by deadlineForStonewalling
write 58850 29427 0.000069 10158080000 2048.00 0.024980 476.89 0.000780 476.92 0
Max Write: 58850.22 MiB/sec (61708.93 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
write 58850.22 58850.22 58850.22 0.00 29425.11 29425.11 29425.11 0.00 476.91945 302.37 70426.29 0 256 32 1 1 1 1 0 0 1 10401873920000 2097152 28066816.0 CEPHFS 0
Finished : Wed Jul 8 13:41:02 2020
- ior-hard-read
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Wed Jul 8 14:06:00 2020
Command line : ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-hard/file -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 1000000 -r -R -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
Machine : Linux o03
TestID : 0
StartTime : Wed Jul 8 14:06:00 2020
Path : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-hard
FS : 429.8 TiB Used FS: 7.0% Inodes: 68.6 Mi Used Inodes: 100.0%
Options:
api : CEPHFS
apiVersion :
test filename : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-hard/file
access : single-shared-file
type : independent
segments : 1000000
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 8
tasks : 256
clients per node : 32
repetitions : 1
xfersize : 47008 bytes
blocksize : 47008 bytes
aggregate filesize : 10.94 TiB
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
stonewalling pairs accessed min: 312030 max: 312030 -- min data: 13.7 GiB mean data: 13.7 GiB time: 247.4s
WARNING: Expected aggregate file size = 12034048000000.
WARNING: Stat() of aggregate file size = 3754983997440.
WARNING: Using actual aggregate bytes moved = 3754983997440.
read 14471 322845 247.42 45.91 45.91 0.031179 247.42 0.001643 247.45 0
Max Read: 14471.49 MiB/sec (15174.46 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
read 14471.49 14471.49 14471.49 0.00 322805.91 322805.91 322805.91 0.00 247.45420 247.42 14473.73 0 256 32 1 0 1 1 0 0 1000000 47008 47008 3581031.8 CEPHFS 0
Finished : Wed Jul 8 14:10:07 2020
- ior-hard-write
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Wed Jul 8 13:46:55 2020
Command line : ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-hard/file -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 1000000 -w -D 300 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
Machine : Linux o03
TestID : 0
StartTime : Wed Jul 8 13:46:55 2020
Path : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-hard
FS : 430.0 TiB Used FS: 6.2% Inodes: 62.4 Mi Used Inodes: 100.0%
Options:
api : CEPHFS
apiVersion :
test filename : /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-hard/file
access : single-shared-file
type : independent
segments : 1000000
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 8
tasks : 256
clients per node : 32
repetitions : 1
xfersize : 47008 bytes
blocksize : 47008 bytes
aggregate filesize : 10.94 TiB
stonewallingTime : 300
stoneWallingWearOut : 1
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
stonewalling pairs accessed min: 311376 max: 312030 -- min data: 13.6 GiB mean data: 13.6 GiB time: 300.1s
WARNING: Expected aggregate file size = 12034048000000.
WARNING: Stat() of aggregate file size = 3754983997440.
WARNING: Using actual aggregate bytes moved = 3754983997440.
WARNING: maybe caused by deadlineForStonewalling
write 11888 265496 300.75 45.91 45.91 0.357277 300.87 0.000769 301.23 0
Max Write: 11888.15 MiB/sec (12465.63 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
write 11888.15 11888.15 11888.15 0.00 265181.07 265181.07 265181.07 0.00 301.22693 300.14 11913.76 0 256 32 1 0 1 1 0 0 1000000 47008 47008 3581031.8 CEPHFS 0
Finished : Wed Jul 8 13:51:58 2020
- mdtest-easy-delete
-
-- started at 07/08/2020 14:10:16 --
mdtest-3.3.0+dev was launched with 256 total task(s) on 8 node(s)
Command line used: ./mdtest '-n' '240000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-easy' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-easy-stonewall' '-r' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app
FS: 429.8 TiB Used FS: 7.0% Inodes: 61.7 Mi Used Inodes: 100.0%
Nodemap: 1111111111111111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 32 for each phase.
256 tasks, 61440000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 131732.881 131732.510 131732.715 0.083
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 1.809 1.809 1.809 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 466.400 466.398 466.399 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.553 0.553 0.553 0.000
-- finished at 07/08/2020 14:18:03 --
- mdtest-easy-stat
-
-- started at 07/08/2020 14:04:34 --
mdtest-3.3.0+dev was launched with 256 total task(s) on 8 node(s)
Command line used: ./mdtest '-n' '240000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-easy' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-easy-stonewall' '-T' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app
FS: 429.8 TiB Used FS: 7.0% Inodes: 61.7 Mi Used Inodes: 100.0%
Nodemap: 1111111111111111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 32 for each phase.
256 tasks, 61440000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 719134.802 719126.468 719131.411 1.842
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 85.437 85.436 85.436 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/08/2020 14:06:00 --
- mdtest-easy-write
-
-- started at 07/08/2020 13:41:03 --
mdtest-3.3.0+dev was launched with 256 total task(s) on 8 node(s)
Command line used: ./mdtest '-n' '240000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-easy' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-easy-stonewall' '-C' '-Y' '-W' '300' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app
FS: 429.8 TiB Used FS: 6.2% Inodes: 0.0 Mi Used Inodes: 100.0%
Nodemap: 1111111111111111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 32 for each phase.
256 tasks, 61440000 files
Continue stonewall hit min: 193400 max: 240000 avg: 238766.0
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 174645.131 174644.907 174645.027 0.052
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 203725.838 NA
Tree creation : 17.297 17.297 17.297 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 351.800 351.799 351.799 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 300.031 NA
Tree creation : 0.058 0.058 0.058 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/08/2020 13:46:55 --
- mdtest-hard-delete
-
-- started at 07/08/2020 14:18:59 --
mdtest-3.3.0+dev was launched with 256 total task(s) on 8 node(s)
Command line used: ./mdtest '-n' '100000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-hard' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-hard-stonewall' '-r' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app
FS: 429.8 TiB Used FS: 7.0% Inodes: 61.5 Mi Used Inodes: 100.0%
Nodemap: 1111111111111111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 32 for each phase.
256 tasks, 25600000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 28698.678 28698.262 28698.492 0.075
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 7.614 7.614 7.614 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 114.841 114.840 114.840 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.131 0.131 0.131 0.000
-- finished at 07/08/2020 14:20:54 --
- mdtest-hard-read
-
-- started at 07/08/2020 14:18:33 --
mdtest-3.3.0+dev was launched with 256 total task(s) on 8 node(s)
Command line used: ./mdtest '-n' '100000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-hard' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-hard-stonewall' '-E' '-X' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app
FS: 429.8 TiB Used FS: 7.0% Inodes: 61.5 Mi Used Inodes: 100.0%
Nodemap: 1111111111111111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 32 for each phase.
256 tasks, 25600000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 132627.667 132619.958 132623.645 1.523
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 24.851 24.850 24.850 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/08/2020 14:18:58 --
- mdtest-hard-stat
-
-- started at 07/08/2020 14:10:08 --
mdtest-3.3.0+dev was launched with 256 total task(s) on 8 node(s)
Command line used: ./mdtest '-n' '100000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-hard' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-hard-stonewall' '-T' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app
FS: 429.8 TiB Used FS: 7.0% Inodes: 61.7 Mi Used Inodes: 100.0%
Nodemap: 1111111111111111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 32 for each phase.
256 tasks, 25600000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 417706.209 417622.547 417663.831 15.247
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 7.892 7.890 7.891 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/08/2020 14:10:15 --
- mdtest-hard-write
-
-- started at 07/08/2020 13:51:59 --
mdtest-3.3.0+dev was launched with 256 total task(s) on 8 node(s)
Command line used: ./mdtest '-n' '100000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-hard' '-x' '/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-hard-stonewall' '-C' '-Y' '-W' '300' '-a' 'CEPHFS' '--cephfs.user=admin' '--cephfs.conf=/etc/ceph/ceph.conf' '--cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0'
Path: /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app
FS: 429.8 TiB Used FS: 7.0% Inodes: 58.6 Mi Used Inodes: 100.0%
Nodemap: 1111111111111111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 32 for each phase.
256 tasks, 25600000 files
Continue stonewall hit min: 9649 max: 12874 avg: 11163.3
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 10403.550 10403.540 10403.545 0.003
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 9508.011 NA
Tree creation : 7.388 7.388 7.388 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 316.791 316.790 316.790 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 300.569 NA
Tree creation : 0.135 0.135 0.135 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/08/2020 13:57:16 --
- result
-
version = io500-isc20_v4
config-hash = 27ECB143
result-dir = ./results/2020.07.08-13.32.59-app
; START 2020-07-08 13:33:03
[ior-easy-write]
t_start = 2020-07-08 13:33:03
exe = ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-easy/stonewall -t 2m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
throughput-stonewall = 11162.73
score = 57.470920
t_delta = 479.2972
t_end = 2020-07-08 13:41:02
[mdtest-easy-write]
t_start = 2020-07-08 13:41:02
exe = ./mdtest -n 240000 -u -L -F -P -N 1 -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-easy -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-easy-stonewall -C -Y -W 300 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
rate-stonewall = 203.725838
score = 174.644992
t_delta = 353.4387
t_end = 2020-07-08 13:46:55
[timestamp]
t_start = 2020-07-08 13:46:55
t_delta = 0.0003
t_end = 2020-07-08 13:46:55
[ior-hard-write]
t_start = 2020-07-08 13:46:55
exe = ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-hard/file -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 1000000 -w -D 300 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
throughput-stonewall = 2967.67
score = 11.609524
t_delta = 302.6935
t_end = 2020-07-08 13:51:58
[mdtest-hard-write]
t_start = 2020-07-08 13:51:58
exe = ./mdtest -n 100000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-hard -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-hard-stonewall -C -Y -W 300 -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
rate-stonewall = 9.508011
score = 10.403542
t_delta = 318.5080
t_end = 2020-07-08 13:57:17
[find]
t_start = 2020-07-08 13:57:17
exe = ./pfind /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app -newer /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/timestampfile -size 3901c -name *01* -C -q 10000
found = 113886
total-files = 64736006
score = 1313.621999
t_delta = 49.2826
t_end = 2020-07-08 13:58:06
[ior-easy-read]
t_start = 2020-07-08 13:58:06
exe = ./ior -C -Q 1 -g -G 271 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-easy/stonewall -t 2m -b 9920000m -F -r -R -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 70.804136
t_delta = 387.7586
t_end = 2020-07-08 14:04:34
[mdtest-easy-stat]
t_start = 2020-07-08 14:04:34
exe = ./mdtest -n 240000 -u -L -F -P -N 1 -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-easy -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-easy-stonewall -T -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 719.132715
t_delta = 85.9831
t_end = 2020-07-08 14:06:00
[ior-hard-read]
t_start = 2020-07-08 14:06:00
exe = ./ior -C -Q 1 -g -G 27 -k -e -o /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-hard/file -O stoneWallingStatusFile=/tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 1000000 -r -R -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 14.132317
t_delta = 247.6251
t_end = 2020-07-08 14:10:07
[mdtest-hard-stat]
t_start = 2020-07-08 14:10:07
exe = ./mdtest -n 100000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-hard -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-hard-stonewall -T -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 417.704885
t_delta = 8.2903
t_end = 2020-07-08 14:10:16
[mdtest-easy-delete]
t_start = 2020-07-08 14:10:16
exe = ./mdtest -n 240000 -u -L -F -P -N 1 -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-easy -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-easy-stonewall -r -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 131.732831
t_delta = 467.2992
t_end = 2020-07-08 14:18:03
[mdtest-hard-read]
t_start = 2020-07-08 14:18:33
exe = ./mdtest -n 100000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-hard -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-hard-stonewall -E -X -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 132.620182
t_delta = 25.2984
t_end = 2020-07-08 14:18:58
[mdtest-hard-delete]
t_start = 2020-07-08 14:18:58
exe = ./mdtest -n 100000 -t -w 3901 -e 3901 -P -N 1 -F -d /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-hard -x /tmp/cbt/mnt/cbt-cephfs-kernel/0/io500/io500-app/datafiles/2020.07.08-13.32.59-app/mdtest-hard-stonewall -r -a CEPHFS --cephfs.user=admin --cephfs.conf=/etc/ceph/ceph.conf --cephfs.prefix=/tmp/cbt/mnt/cbt-cephfs-kernel/0
score = 28.698628
t_delta = 115.3509
t_end = 2020-07-08 14:20:54
[SCORE]
MD = 156.479119
BW = 28.584701
SCORE = 66.879808
hash = BA058651
; END 2020-07-08 14:20:54
- result_summary
-
IO500 version io500-isc20_v4
[RESULT] ior-easy-write 57.470920 GiB/s : time 479.297 seconds
[RESULT] mdtest-easy-write 174.644992 kIOPS : time 353.439 seconds
[RESULT] ior-hard-write 11.609524 GiB/s : time 302.693 seconds
[RESULT] mdtest-hard-write 10.403542 kIOPS : time 318.508 seconds
[RESULT] find 1313.621999 kIOPS : time 49.283 seconds
[RESULT] ior-easy-read 70.804136 GiB/s : time 387.759 seconds
[RESULT] mdtest-easy-stat 719.132715 kIOPS : time 85.983 seconds
[RESULT] ior-hard-read 14.132317 GiB/s : time 247.625 seconds
[RESULT] mdtest-hard-stat 417.704885 kIOPS : time 8.290 seconds
[RESULT] mdtest-easy-delete 131.732831 kIOPS : time 467.299 seconds
[RESULT] mdtest-hard-read 132.620182 kIOPS : time 25.298 seconds
[RESULT] mdtest-hard-delete 28.698628 kIOPS : time 115.351 seconds
[SCORE] Bandwidth 28.584701 GB/s : IOPS 156.479119 kiops : TOTAL 66.879808