- io500
-
#!/bin/bash
export I_MPI_FABRICS=shm:dapl
export I_MPI_FALLBACK=tcp
export I_MPI_DEBUG=0
export I_MPI_ROOT=/misc/local/INTEL-2018/compilers_and_libraries_2018.0.128/linux/mpi
export DAT_OVERRIDE=/misc/local/admin/cx5_dat.conf
mpirun -np 1248 -ppn 26 -f hostfile-48-mix ./io500 config-hhmi.ini
- ior-easy-read
-
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began : Sun Oct 18 14:56:34 2020
Command line : ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.18-14.18.58/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.18-14.18.58/ior-easy.stonewall -t 32m -b 9920000m -F -r -R -a POSIX
Machine : Linux h09u02.int.janelia.org
TestID : 0
StartTime : Sun Oct 18 14:56:34 2020
Path : ./datadir/2020.10.18-14.18.58/ior-easy/ior_file_easy.00000000
FS : 150.0 TiB Used FS: 60.7% Inodes: 858.3 Mi Used Inodes: 47.0%
Options:
api : POSIX
apiVersion :
test filename : ./datadir/2020.10.18-14.18.58/ior-easy/ior_file_easy
access : file-per-process
type : independent
segments : 1
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 48
tasks : 1248
clients per node : 26
repetitions : 1
xfersize : 32 MiB
blocksize : 9.46 TiB
aggregate filesize : 11806.64 TiB
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 12981538652160000
WARNING: Stat() of aggregate file size = 39949638303744
WARNING: Using actual aggregate bytes moved = 39949638303744
read 82607 2581.50 0.001488 10158080000 32768 0.006506 461.20 0.003419 461.21 0
- ior-easy-write
-
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began : Sun Oct 18 14:18:58 2020
Command line : ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.18-14.18.58/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.18-14.18.58/ior-easy.stonewall -t 32m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX --posix.odirect
Machine : Linux h09u02.int.janelia.org
TestID : 0
StartTime : Sun Oct 18 14:18:58 2020
Path : ./datadir/2020.10.18-14.18.58/ior-easy/ior_file_easy.00000000
FS : 150.0 TiB Used FS: 25.4% Inodes: 858.3 Mi Used Inodes: 26.4%
Options:
api : POSIX
apiVersion :
test filename : ./datadir/2020.10.18-14.18.58/ior-easy/ior_file_easy
access : file-per-process
type : independent
segments : 1
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 48
tasks : 1248
clients per node : 26
repetitions : 1
xfersize : 32 MiB
blocksize : 9.46 TiB
aggregate filesize : 11806.64 TiB
stonewallingTime : 300
stoneWallingWearOut : 1
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
stonewalling pairs accessed min: 397 max: 954 -- min data: 12.4 GiB mean data: 19.8 GiB time: 300.4s
WARNING: Expected aggregate file size = 12981538652160000
WARNING: Stat() of aggregate file size = 39949638303744
WARNING: Using actual aggregate bytes moved = 39949638303744
WARNING: Maybe caused by deadlineForStonewalling
write 77155 2411.39 0.001476 10158080000 32768 0.054248 493.74 0.004369 493.79 0
- ior-hard-read
-
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began : Sun Oct 18 15:04:40 2020
Command line : ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.18-14.18.58/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.18-14.18.58/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a POSIX
Machine : Linux h09u02.int.janelia.org
TestID : 0
StartTime : Sun Oct 18 15:04:40 2020
Path : ./datadir/2020.10.18-14.18.58/ior-hard/file
FS : 150.0 TiB Used FS: 60.7% Inodes: 858.3 Mi Used Inodes: 47.0%
Options:
api : POSIX
apiVersion :
test filename : ./datadir/2020.10.18-14.18.58/ior-hard/file
access : single-shared-file
type : independent
segments : 10000000
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 48
tasks : 1248
clients per node : 26
repetitions : 1
xfersize : 47008 bytes
blocksize : 47008 bytes
aggregate filesize : 533.56 TiB
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
stonewalling pairs accessed min: 298372 max: 298372 -- min data: 13.1 GiB mean data: 13.1 GiB time: 367.3s
WARNING: Expected aggregate file size = 586659840000000
WARNING: Stat() of aggregate file size = 17504286978048
WARNING: Using actual aggregate bytes moved = 17504286978048
read 45417 1013118 367.54 45.91 45.91 0.014385 367.55 0.003304 367.56 0
- ior-hard-write
-
IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
Began : Sun Oct 18 14:33:34 2020
Command line : ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.18-14.18.58/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.18-14.18.58/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a POSIX
Machine : Linux h09u02.int.janelia.org
TestID : 0
StartTime : Sun Oct 18 14:33:34 2020
Path : ./datadir/2020.10.18-14.18.58/ior-hard/file
FS : 150.0 TiB Used FS: 50.0% Inodes: 858.3 Mi Used Inodes: 42.6%
Options:
api : POSIX
apiVersion :
test filename : ./datadir/2020.10.18-14.18.58/ior-hard/file
access : single-shared-file
type : independent
segments : 10000000
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 48
tasks : 1248
clients per node : 26
repetitions : 1
xfersize : 47008 bytes
blocksize : 47008 bytes
aggregate filesize : 533.56 TiB
stonewallingTime : 300
stoneWallingWearOut : 1
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
stonewalling pairs accessed min: 9 max: 298372 -- min data: 0.0 GiB mean data: 0.5 GiB time: 300.1s
WARNING: Expected aggregate file size = 586659840000000
WARNING: Stat() of aggregate file size = 17504286978048
WARNING: Using actual aggregate bytes moved = 17504286978048
WARNING: Maybe caused by deadlineForStonewalling
write 20210 450823 789.27 45.91 45.91 0.010618 825.97 0.003273 825.99 0
- mdtest-easy-delete
-
-- started at 10/18/2020 15:11:33 --
mdtest-3.4.0+dev was launched with 1248 total task(s) on 48 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datadir/2020.10.18-14.18.58/mdtest-easy' '-x' './results/2020.10.18-14.18.58/mdtest-easy.stonewall' '-r' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.18-14.18.58
FS: 150.0 TiB Used FS: 60.7% Inodes: 858.3 Mi Used Inodes: 47.0%
Nodemap: 111111111111111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2216 Shifting ranks by 26 for each phase.
1248 tasks, 1248000000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 451129.556 451128.434 451129.188 0.260
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.933 0.933 0.933 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 235.797 235.796 235.796 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 1.072 1.072 1.072 0.000
-- finished at 10/18/2020 15:15:33 --
- mdtest-easy-stat
-
-- started at 10/18/2020 15:04:16 --
mdtest-3.4.0+dev was launched with 1248 total task(s) on 48 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datadir/2020.10.18-14.18.58/mdtest-easy' '-x' './results/2020.10.18-14.18.58/mdtest-easy.stonewall' '-T' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.18-14.18.58
FS: 150.0 TiB Used FS: 60.7% Inodes: 858.3 Mi Used Inodes: 47.0%
Nodemap: 111111111111111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2216 Shifting ranks by 26 for each phase.
1248 tasks, 1248000000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 4314480.217 4314459.732 4314466.567 3.132
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 24.655 24.655 24.655 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 10/18/2020 15:04:40 --
- mdtest-easy-write
-
-- started at 10/18/2020 14:27:12 --
mdtest-3.4.0+dev was launched with 1248 total task(s) on 48 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './datadir/2020.10.18-14.18.58/mdtest-easy' '-x' './results/2020.10.18-14.18.58/mdtest-easy.stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.18-14.18.58
FS: 150.0 TiB Used FS: 49.7% Inodes: 858.3 Mi Used Inodes: 30.6%
Nodemap: 111111111111111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2216 Shifting ranks by 26 for each phase.
1248 tasks, 1248000000 files
Continue stonewall hit min: 37181 max: 85236 avg: 78958.5
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 301376.122 301374.815 301375.530 0.184
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 328441.802 NA
Tree creation : 16.158 16.158 16.158 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 352.964 352.963 352.963 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 300.023 NA
Tree creation : 0.062 0.062 0.062 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 10/18/2020 14:33:34 --
- mdtest-hard-delete
-
-- started at 10/18/2020 15:16:43 --
mdtest-3.4.0+dev was launched with 1248 total task(s) on 48 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.18-14.18.58/mdtest-hard' '-x' './results/2020.10.18-14.18.58/mdtest-hard.stonewall' '-r' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.18-14.18.58
FS: 150.0 TiB Used FS: 60.5% Inodes: 858.3 Mi Used Inodes: 35.0%
Nodemap: 111111111111111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2216 Shifting ranks by 26 for each phase.
1248 tasks, 1248000000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 81631.309 81631.165 81631.259 0.027
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.313 0.313 0.313 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 279.883 279.882 279.882 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 3.192 3.192 3.192 0.000
-- finished at 10/18/2020 15:21:27 --
- mdtest-hard-read
-
-- started at 10/18/2020 15:15:34 --
mdtest-3.4.0+dev was launched with 1248 total task(s) on 48 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.18-14.18.58/mdtest-hard' '-x' './results/2020.10.18-14.18.58/mdtest-hard.stonewall' '-E' '-X' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.18-14.18.58
FS: 150.0 TiB Used FS: 60.5% Inodes: 858.3 Mi Used Inodes: 35.0%
Nodemap: 111111111111111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2216 Shifting ranks by 26 for each phase.
1248 tasks, 1248000000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 327238.318 327237.752 327238.148 0.108
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 69.818 69.818 69.818 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 10/18/2020 15:16:43 --
- mdtest-hard-stat
-
-- started at 10/18/2020 15:10:48 --
mdtest-3.4.0+dev was launched with 1248 total task(s) on 48 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.18-14.18.58/mdtest-hard' '-x' './results/2020.10.18-14.18.58/mdtest-hard.stonewall' '-T' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.18-14.18.58
FS: 150.0 TiB Used FS: 60.7% Inodes: 858.3 Mi Used Inodes: 47.0%
Nodemap: 111111111111111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2216 Shifting ranks by 26 for each phase.
1248 tasks, 1248000000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 506149.501 506143.927 506145.671 0.690
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 45.140 45.139 45.139 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 10/18/2020 15:11:33 --
- mdtest-hard-write
-
-- started at 10/18/2020 14:47:20 --
mdtest-3.4.0+dev was launched with 1248 total task(s) on 48 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './datadir/2020.10.18-14.18.58/mdtest-hard' '-x' './results/2020.10.18-14.18.58/mdtest-hard.stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /misc/weka/wekaFS_1/io500-sc20/io500/datadir/2020.10.18-14.18.58
FS: 150.0 TiB Used FS: 60.4% Inodes: 858.3 Mi Used Inodes: 44.5%
Nodemap: 111111111111111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2216 Shifting ranks by 26 for each phase.
1248 tasks, 1248000000 files
Continue stonewall hit min: 17009 max: 18307 avg: 17585.2
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 69443.543 69443.324 69443.426 0.031
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 73149.139 NA
Tree creation : 965.317 965.317 965.317 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 329.004 329.003 329.004 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 300.021 NA
Tree creation : 0.001 0.001 0.001 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 10/18/2020 14:53:30 --
- result
-
version = io500-sc20
config-hash = F6D0CA1
result-dir = ./results/2020.10.18-14.18.58
; START 2020-10-18 14:18:58
[ior-easy-write]
t_start = 2020-10-18 14:18:58
exe = ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.18-14.18.58/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.18-14.18.58/ior-easy.stonewall -t 32m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX --posix.odirect
throughput-stonewall = 62319.37
score = 75.347071
t_delta = 493.8916
t_end = 2020-10-18 14:27:12
[mdtest-easy-write]
t_start = 2020-10-18 14:27:12
exe = ./mdtest -n 1000000 -u -L -F -P -N 1 -d ./datadir/2020.10.18-14.18.58/mdtest-easy -x ./results/2020.10.18-14.18.58/mdtest-easy.stonewall -C -Y -W 300 -a POSIX
rate-stonewall = 328.441802
score = 301.375520
t_delta = 382.4359
t_end = 2020-10-18 14:33:34
[timestamp]
t_start = 2020-10-18 14:33:34
t_delta = 0.0012
t_end = 2020-10-18 14:33:34
[ior-hard-write]
t_start = 2020-10-18 14:33:34
exe = ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.18-14.18.58/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.18-14.18.58/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -w -D 300 -a POSIX
throughput-stonewall = 998.16
score = 19.736564
t_delta = 826.0049
t_end = 2020-10-18 14:47:20
[mdtest-hard-write]
t_start = 2020-10-18 14:47:20
exe = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.18-14.18.58/mdtest-hard -x ./results/2020.10.18-14.18.58/mdtest-hard.stonewall -C -Y -W 300 -a POSIX
rate-stonewall = 73.149139
score = 69.443424
t_delta = 369.8390
t_end = 2020-10-18 14:53:30
[find]
t_start = 2020-10-18 14:53:30
exe = ./pfind ./datadir/2020.10.18-14.18.58 -newer ./results/2020.10.18-14.18.58/timestampfile -size 3901c -name *01* -C -q 10000
nproc = 480
found = 969166
total-files = 129222913
score = 700.954508
t_delta = 184.3682
t_end = 2020-10-18 14:56:34
[ior-easy-read]
t_start = 2020-10-18 14:56:34
exe = ./ior -C -Q 1 -g -G 271 -k -e -o ./datadir/2020.10.18-14.18.58/ior-easy/ior_file_easy -O stoneWallingStatusFile=./results/2020.10.18-14.18.58/ior-easy.stonewall -t 32m -b 9920000m -F -r -R -a POSIX
score = 80.670930
t_delta = 461.2849
t_end = 2020-10-18 15:04:16
[mdtest-easy-stat]
t_start = 2020-10-18 15:04:16
exe = ./mdtest -n 1000000 -u -L -F -P -N 1 -d ./datadir/2020.10.18-14.18.58/mdtest-easy -x ./results/2020.10.18-14.18.58/mdtest-easy.stonewall -T -a POSIX
score = 4314.479508
t_delta = 24.7125
t_end = 2020-10-18 15:04:40
[ior-hard-read]
t_start = 2020-10-18 15:04:40
exe = ./ior -C -Q 1 -g -G 27 -k -e -o ./datadir/2020.10.18-14.18.58/ior-hard/file -O stoneWallingStatusFile=./results/2020.10.18-14.18.58/ior-hard.stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 10000000 -r -R -a POSIX
score = 44.352347
t_delta = 367.5758
t_end = 2020-10-18 15:10:48
[mdtest-hard-stat]
t_start = 2020-10-18 15:10:48
exe = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.18-14.18.58/mdtest-hard -x ./results/2020.10.18-14.18.58/mdtest-hard.stonewall -T -a POSIX
score = 506.145766
t_delta = 45.1544
t_end = 2020-10-18 15:11:33
[mdtest-easy-delete]
t_start = 2020-10-18 15:11:33
exe = ./mdtest -n 1000000 -u -L -F -P -N 1 -d ./datadir/2020.10.18-14.18.58/mdtest-easy -x ./results/2020.10.18-14.18.58/mdtest-easy.stonewall -r -a POSIX
score = 451.129284
t_delta = 240.2008
t_end = 2020-10-18 15:15:33
[mdtest-hard-read]
t_start = 2020-10-18 15:15:33
exe = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.18-14.18.58/mdtest-hard -x ./results/2020.10.18-14.18.58/mdtest-hard.stonewall -E -X -a POSIX
score = 327.238233
t_delta = 70.0446
t_end = 2020-10-18 15:16:43
[mdtest-hard-delete]
t_start = 2020-10-18 15:16:43
exe = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d ./datadir/2020.10.18-14.18.58/mdtest-hard -x ./results/2020.10.18-14.18.58/mdtest-hard.stonewall -r -a POSIX
score = 81.631238
t_delta = 283.0916
t_end = 2020-10-18 15:21:27
[SCORE]
MD = 374.396517
BW = 48.027823
SCORE = 134.094927
hash = DF69C0A5
; END 2020-10-18 15:21:32
- result_summary
-
IO500 version io500-sc20
[RESULT] ior-easy-write 75.347071 GiB/s : time 493.892 seconds
[RESULT] mdtest-easy-write 301.375520 kIOPS : time 382.436 seconds
[RESULT] ior-hard-write 19.736564 GiB/s : time 826.005 seconds
[RESULT] mdtest-hard-write 69.443424 kIOPS : time 369.839 seconds
[RESULT] find 700.954508 kIOPS : time 184.368 seconds
[RESULT] ior-easy-read 80.670930 GiB/s : time 461.285 seconds
[RESULT] mdtest-easy-stat 4314.479508 kIOPS : time 24.713 seconds
[RESULT] ior-hard-read 44.352347 GiB/s : time 367.576 seconds
[RESULT] mdtest-hard-stat 506.145766 kIOPS : time 45.154 seconds
[RESULT] mdtest-easy-delete 451.129284 kIOPS : time 240.201 seconds
[RESULT] mdtest-hard-read 327.238233 kIOPS : time 70.045 seconds
[RESULT] mdtest-hard-delete 81.631238 kIOPS : time 283.092 seconds
[SCORE] Bandwidth 48.027823 GB/s : IOPS 374.396517 kiops : TOTAL 134.094927