- ior-easy-read
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Tue Jul 14 04:03:57 2020
Command line : ./ior -C -Q 1 -g -G 271 -k -e -o ./out//2020.07.14-01.17.59-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=./out//2020.07.14-01.17.59-app/ior-easy/stonewall -t 1m -b 200000m -F -r -R -a POSIX
Machine : Linux inst-0aaux-brave-troll
TestID : 0
StartTime : Tue Jul 14 04:03:57 2020
Path : /mnt/beeond/io500-app/out/2020.07.14-01.17.59-app/ior-easy
FS : 1571.4 TiB Used FS: 41.7% Inodes: 0.0 Mi Used Inodes: -nan%
Options:
api : POSIX
apiVersion :
test filename : ./out//2020.07.14-01.17.59-app/ior-easy/ior_file_easy
access : file-per-process
type : independent
segments : 1
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 270
tasks : 3240
clients per node : 12
repetitions : 1
xfersize : 1 MiB
blocksize : 195.31 GiB
aggregate filesize : 617.98 TiB
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 679477248000000.
WARNING: Stat() of aggregate file size = 341050015088640.
WARNING: Using actual aggregate bytes moved = 341050015088640.
read 764657 765838 0.002123 204800000 1024.00 0.270898 424.70 0.387367 425.36 0
Max Read: 764656.54 MiB/sec (801800.49 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
read 764656.54 764656.54 764656.54 0.00 764656.54 764656.54 764656.54 0.00 425.35521 NA NA 0 3240 12 1 1 1 1 0 0 1 209715200000 1048576 325250624.0 POSIX 0
Finished : Tue Jul 14 04:11:02 2020
- ior-easy-write
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Tue Jul 14 02:52:46 2020
Command line : ./ior -C -Q 1 -g -G 271 -k -e -o ./out//2020.07.14-01.17.59-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=./out//2020.07.14-01.17.59-app/ior-easy/stonewall -t 1m -b 200000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX
Machine : Linux inst-0aaux-brave-troll
TestID : 0
StartTime : Tue Jul 14 02:52:46 2020
Path : /mnt/beeond/io500-app/out/2020.07.14-01.17.59-app/ior-easy
FS : 1571.4 TiB Used FS: 21.2% Inodes: 0.0 Mi Used Inodes: -nan%
Options:
api : POSIX
apiVersion :
test filename : ./out//2020.07.14-01.17.59-app/ior-easy/ior_file_easy
access : file-per-process
type : independent
segments : 1
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 270
tasks : 3240
clients per node : 12
repetitions : 1
xfersize : 1 MiB
blocksize : 195.31 GiB
aggregate filesize : 617.98 TiB
stonewallingTime : 300
stoneWallingWearOut : 1
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
stonewalling pairs accessed min: 48079 max: 100386 -- min data: 47.0 GiB mean data: 83.3 GiB time: 300.1s
WARNING: Expected aggregate file size = 679477248000000.
WARNING: Stat() of aggregate file size = 341050015088640.
WARNING: Using actual aggregate bytes moved = 341050015088640.
WARNING: maybe caused by deadlineForStonewalling
write 495329 696807 0.001989 204800000 1024.00 189.46 466.77 0.405525 656.64 0
Max Write: 495329.07 MiB/sec (519390.18 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
write 495329.07 495329.07 495329.07 0.00 495329.07 495329.07 495329.07 0.00 656.63547 300.09 920924.36 0 3240 12 1 1 1 1 0 0 1 209715200000 1048576 325250624.0 POSIX 0
Finished : Tue Jul 14 03:05:56 2020
- ior-hard-read
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Tue Jul 14 04:13:35 2020
Command line : ./ior -C -Q 1 -g -G 27 -k -e -o ./out//2020.07.14-01.17.59-app/ior-hard/file -O stoneWallingStatusFile=./out//2020.07.14-01.17.59-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 2200000 -r -R -a POSIX
Machine : Linux inst-0aaux-brave-troll
TestID : 0
StartTime : Tue Jul 14 04:13:35 2020
Path : /mnt/beeond/io500-app/out/2020.07.14-01.17.59-app/ior-hard
FS : 1571.4 TiB Used FS: 41.7% Inodes: 0.0 Mi Used Inodes: -nan%
Options:
api : POSIX
apiVersion :
test filename : ./out//2020.07.14-01.17.59-app/ior-hard/file
access : single-shared-file
type : independent
segments : 2200000
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 270
tasks : 3240
clients per node : 12
repetitions : 1
xfersize : 47008 bytes
blocksize : 47008 bytes
aggregate filesize : 304.75 TiB
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
stonewalling pairs accessed min: 16793 max: 16793 -- min data: 0.7 GiB mean data: 0.7 GiB time: 223.4s
WARNING: Expected aggregate file size = 335073024000000.
WARNING: Stat() of aggregate file size = 2557673314560.
WARNING: Using actual aggregate bytes moved = 2557673314560.
read 10897 243397 223.54 45.91 45.91 0.222213 223.54 0.070848 223.83 0
Max Read: 10897.38 MiB/sec (11426.73 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
read 10897.38 10897.38 10897.38 0.00 243080.64 243080.64 243080.64 0.00 223.83239 223.41 10918.12 0 3240 12 1 0 1 1 0 0 2200000 47008 47008 2439187.2 POSIX 0
Finished : Tue Jul 14 04:17:19 2020
- ior-hard-write
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Tue Jul 14 03:18:48 2020
Command line : ./ior -C -Q 1 -g -G 27 -k -e -o ./out//2020.07.14-01.17.59-app/ior-hard/file -O stoneWallingStatusFile=./out//2020.07.14-01.17.59-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 2200000 -w -D 300 -a POSIX
Machine : Linux inst-0aaux-brave-troll
TestID : 0
StartTime : Tue Jul 14 03:18:48 2020
Path : /mnt/beeond/io500-app/out/2020.07.14-01.17.59-app/ior-hard
FS : 1571.4 TiB Used FS: 41.5% Inodes: 0.0 Mi Used Inodes: -nan%
Options:
api : POSIX
apiVersion :
test filename : ./out//2020.07.14-01.17.59-app/ior-hard/file
access : single-shared-file
type : independent
segments : 2200000
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 270
tasks : 3240
clients per node : 12
repetitions : 1
xfersize : 47008 bytes
blocksize : 47008 bytes
aggregate filesize : 304.75 TiB
stonewallingTime : 300
stoneWallingWearOut : 1
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
stonewalling pairs accessed min: 1710 max: 16793 -- min data: 0.1 GiB mean data: 0.2 GiB time: 321.4s
WARNING: Expected aggregate file size = 335073024000000.
WARNING: Stat() of aggregate file size = 2557673314560.
WARNING: Using actual aggregate bytes moved = 2557673314560.
WARNING: maybe caused by deadlineForStonewalling
write 1349.26 30102 1722.66 45.91 45.91 0.274388 1807.51 0.013749 1807.79 0
Max Write: 1349.26 MiB/sec (1414.80 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
write 1349.26 1349.26 1349.26 0.00 30097.07 30097.07 30097.07 0.00 1807.79484 321.40 1952.46 0 3240 12 1 0 1 1 0 0 2200000 47008 47008 2439187.2 POSIX 0
Finished : Tue Jul 14 03:52:06 2020
- mdtest-easy-delete
-
-- started at 07/14/2020 04:18:18 --
mdtest-3.3.0+dev was launched with 3240 total task(s) on 270 node(s)
Command line used: ./mdtest '-n' '5000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './out//2020.07.14-01.17.59-app/mdtest-easy' '-x' './out//2020.07.14-01.17.59-app/mdtest-easy-stonewall' '-r' '-a' 'POSIX'
Path: /mnt/beeond/io500-app/out/2020.07.14-01.17.59-app
FS: 1571.4 TiB Used FS: 41.7% Inodes: 0.0 Mi Used Inodes: -nan%
Nodemap: 111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 12 for each phase.
3240 tasks, 16200000000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 3147976.309 3147971.802 3147974.377 0.697
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.027 0.027 0.027 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 695.611 695.610 695.610 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 36.528 36.528 36.528 0.000
-- finished at 07/14/2020 04:30:34 --
- mdtest-easy-stat
-
-- started at 07/14/2020 04:11:02 --
mdtest-3.3.0+dev was launched with 3240 total task(s) on 270 node(s)
Command line used: ./mdtest '-n' '5000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './out//2020.07.14-01.17.59-app/mdtest-easy' '-x' './out//2020.07.14-01.17.59-app/mdtest-easy-stonewall' '-T' '-a' 'POSIX'
Path: /mnt/beeond/io500-app/out/2020.07.14-01.17.59-app
FS: 1571.4 TiB Used FS: 41.7% Inodes: 0.0 Mi Used Inodes: -nan%
Nodemap: 111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 12 for each phase.
3240 tasks, 16200000000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 14312128.656 14312038.566 14312099.437 5.506
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 153.002 153.001 153.001 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/14/2020 04:13:35 --
- mdtest-easy-write
-
-- started at 07/14/2020 03:05:56 --
mdtest-3.3.0+dev was launched with 3240 total task(s) on 270 node(s)
Command line used: ./mdtest '-n' '5000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' './out//2020.07.14-01.17.59-app/mdtest-easy' '-x' './out//2020.07.14-01.17.59-app/mdtest-easy-stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /mnt/beeond/io500-app/out/2020.07.14-01.17.59-app
FS: 1571.4 TiB Used FS: 40.9% Inodes: 0.0 Mi Used Inodes: -nan%
Nodemap: 111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 12 for each phase.
3240 tasks, 16200000000 files
Continue stonewall hit min: 51730 max: 675853 avg: 316338.7
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 2908536.665 2908525.261 2908530.064 2.509
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 3623543.965 NA
Tree creation : 0.058 0.058 0.058 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 752.878 752.875 752.876 0.001
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 282.855 NA
Tree creation : 17.189 17.189 17.189 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/14/2020 03:18:48 --
- mdtest-hard-delete
-
-- started at 07/14/2020 04:31:42 --
mdtest-3.3.0+dev was launched with 3240 total task(s) on 270 node(s)
Command line used: ./mdtest '-n' '160000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './out//2020.07.14-01.17.59-app/mdtest-hard' '-x' './out//2020.07.14-01.17.59-app/mdtest-hard-stonewall' '-r' '-a' 'POSIX'
Path: /mnt/beeond/io500-app/out/2020.07.14-01.17.59-app
FS: 1571.4 TiB Used FS: 41.1% Inodes: 0.0 Mi Used Inodes: -nan%
Nodemap: 111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 12 for each phase.
3240 tasks, 518400000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 7035.877 7035.847 7035.867 0.003
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.769 0.769 0.769 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 162.556 162.555 162.556 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 1.301 1.301 1.301 0.000
-- finished at 07/14/2020 04:34:25 --
- mdtest-hard-read
-
-- started at 07/14/2020 04:30:34 --
mdtest-3.3.0+dev was launched with 3240 total task(s) on 270 node(s)
Command line used: ./mdtest '-n' '160000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './out//2020.07.14-01.17.59-app/mdtest-hard' '-x' './out//2020.07.14-01.17.59-app/mdtest-hard-stonewall' '-E' '-X' '-a' 'POSIX'
Path: /mnt/beeond/io500-app/out/2020.07.14-01.17.59-app
FS: 1571.4 TiB Used FS: 41.1% Inodes: 0.0 Mi Used Inodes: -nan%
Nodemap: 111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 12 for each phase.
3240 tasks, 518400000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 17027.467 17027.353 17027.420 0.009
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 67.170 67.169 67.169 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/14/2020 04:31:42 --
- mdtest-hard-stat
-
-- started at 07/14/2020 04:17:19 --
mdtest-3.3.0+dev was launched with 3240 total task(s) on 270 node(s)
Command line used: ./mdtest '-n' '160000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './out//2020.07.14-01.17.59-app/mdtest-hard' '-x' './out//2020.07.14-01.17.59-app/mdtest-hard-stonewall' '-T' '-a' 'POSIX'
Path: /mnt/beeond/io500-app/out/2020.07.14-01.17.59-app
FS: 1571.4 TiB Used FS: 41.7% Inodes: 0.0 Mi Used Inodes: -nan%
Nodemap: 111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 12 for each phase.
3240 tasks, 518400000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 19554.766 19554.621 19554.675 0.012
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 58.488 58.488 58.488 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/14/2020 04:18:18 --
- mdtest-hard-write
-
-- started at 07/14/2020 03:52:06 --
mdtest-3.3.0+dev was launched with 3240 total task(s) on 270 node(s)
Command line used: ./mdtest '-n' '160000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' './out//2020.07.14-01.17.59-app/mdtest-hard' '-x' './out//2020.07.14-01.17.59-app/mdtest-hard-stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /mnt/beeond/io500-app/out/2020.07.14-01.17.59-app
FS: 1571.4 TiB Used FS: 41.7% Inodes: 0.0 Mi Used Inodes: -nan%
Nodemap: 111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 12 for each phase.
3240 tasks, 518400000 files
Continue stonewall hit min: 238 max: 353 avg: 273.1
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 2221.466 2221.464 2221.465 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 2936.237 NA
Tree creation : 142.131 142.131 142.131 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 514.850 514.849 514.850 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 301.330 NA
Tree creation : 0.007 0.007 0.007 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/14/2020 04:00:42 --
- result
-
version = io500-isc20_v4
config-hash = 65073174
result-dir = ./results//2020.07.14-01.17.59-app
; START 2020-07-14 02:52:46
; WARNING Using internal pfind, will ignore any arguments to the external script
[ior-easy-write]
t_start = 2020-07-14 02:52:46
exe = ./ior -C -Q 1 -g -G 271 -k -e -o ./out//2020.07.14-01.17.59-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=./out//2020.07.14-01.17.59-app/ior-easy/stonewall -t 1m -b 200000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX
throughput-stonewall = 1331690.67
score = 483.719797
t_delta = 789.8664
t_end = 2020-07-14 03:05:56
[mdtest-easy-write]
t_start = 2020-07-14 03:05:56
exe = ./mdtest -n 5000000 -u -L -F -P -N 1 -d ./out//2020.07.14-01.17.59-app/mdtest-easy -x ./out//2020.07.14-01.17.59-app/mdtest-easy-stonewall -C -Y -W 300 -a POSIX
rate-stonewall = 3623.543965
score = 2908.530413
t_delta = 772.1078
t_end = 2020-07-14 03:18:48
[timestamp]
t_start = 2020-07-14 03:18:48
t_delta = 0.0009
t_end = 2020-07-14 03:18:48
[ior-hard-write]
t_start = 2020-07-14 03:18:48
exe = ./ior -C -Q 1 -g -G 27 -k -e -o ./out//2020.07.14-01.17.59-app/ior-hard/file -O stoneWallingStatusFile=./out//2020.07.14-01.17.59-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 2200000 -w -D 300 -a POSIX
throughput-stonewall = 1098.30
score = 1.317638
t_delta = 1997.5676
t_end = 2020-07-14 03:52:06
[mdtest-hard-write]
t_start = 2020-07-14 03:52:06
exe = ./mdtest -n 160000 -t -w 3901 -e 3901 -P -N 1 -F -d ./out//2020.07.14-01.17.59-app/mdtest-hard -x ./out//2020.07.14-01.17.59-app/mdtest-hard-stonewall -C -Y -W 300 -a POSIX
rate-stonewall = 2.936237
score = 2.221464
t_delta = 515.9973
t_end = 2020-07-14 04:00:42
[find]
t_start = 2020-07-14 04:00:42
exe = ./pfind ./out//2020.07.14-01.17.59-app -newer ./out//2020.07.14-01.17.59-app/timestampfile -size 3901c -name *01* -C -N -H 1 -q 15000
found = 31420
total-files = 2190910686
score = 11217.629155
t_delta = 195.3589
t_end = 2020-07-14 04:03:57
[ior-easy-read]
t_start = 2020-07-14 04:03:57
exe = ./ior -C -Q 1 -g -G 271 -k -e -o ./out//2020.07.14-01.17.59-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=./out//2020.07.14-01.17.59-app/ior-easy/stonewall -t 1m -b 200000m -F -r -R -a POSIX
score = 746.734899
t_delta = 425.4484
t_end = 2020-07-14 04:11:02
[mdtest-easy-stat]
t_start = 2020-07-14 04:11:02
exe = ./mdtest -n 5000000 -u -L -F -P -N 1 -d ./out//2020.07.14-01.17.59-app/mdtest-easy -x ./out//2020.07.14-01.17.59-app/mdtest-easy-stonewall -T -a POSIX
score = 14312.108177
t_delta = 153.0121
t_end = 2020-07-14 04:13:35
[ior-hard-read]
t_start = 2020-07-14 04:13:35
exe = ./ior -C -Q 1 -g -G 27 -k -e -o ./out//2020.07.14-01.17.59-app/ior-hard/file -O stoneWallingStatusFile=./out//2020.07.14-01.17.59-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 2200000 -r -R -a POSIX
score = 10.641976
t_delta = 223.9856
t_end = 2020-07-14 04:17:19
[mdtest-hard-stat]
t_start = 2020-07-14 04:17:19
exe = ./mdtest -n 160000 -t -w 3901 -e 3901 -P -N 1 -F -d ./out//2020.07.14-01.17.59-app/mdtest-hard -x ./out//2020.07.14-01.17.59-app/mdtest-hard-stonewall -T -a POSIX
score = 19.554671
t_delta = 58.5132
t_end = 2020-07-14 04:18:18
[mdtest-easy-delete]
t_start = 2020-07-14 04:18:18
exe = ./mdtest -n 5000000 -u -L -F -P -N 1 -d ./out//2020.07.14-01.17.59-app/mdtest-easy -x ./out//2020.07.14-01.17.59-app/mdtest-easy-stonewall -r -a POSIX
score = 3147.974543
t_delta = 736.4367
t_end = 2020-07-14 04:30:34
[mdtest-hard-read]
t_start = 2020-07-14 04:30:34
exe = ./mdtest -n 160000 -t -w 3901 -e 3901 -P -N 1 -F -d ./out//2020.07.14-01.17.59-app/mdtest-hard -x ./out//2020.07.14-01.17.59-app/mdtest-hard-stonewall -E -X -a POSIX
score = 17.027393
t_delta = 67.1973
t_end = 2020-07-14 04:31:42
[mdtest-hard-delete]
t_start = 2020-07-14 04:31:42
exe = ./mdtest -n 160000 -t -w 3901 -e 3901 -P -N 1 -F -d ./out//2020.07.14-01.17.59-app/mdtest-hard -x ./out//2020.07.14-01.17.59-app/mdtest-hard-stonewall -r -a POSIX
score = 7.035865
t_delta = 163.8746
t_end = 2020-07-14 04:34:25
[SCORE]
MD = 229.328552
BW = 47.439998
SCORE = 104.304104
hash = 5F141BF5
; END 2020-07-14 04:34:25
- result_summary
-
IO500 version io500-isc20_v4
[RESULT] ior-easy-write 483.719797 GiB/s : time 789.866 seconds
[RESULT] mdtest-easy-write 2908.530413 kIOPS : time 772.108 seconds
[RESULT] ior-hard-write 1.317638 GiB/s : time 1997.568 seconds
[RESULT] mdtest-hard-write 2.221464 kIOPS : time 515.997 seconds
[RESULT] find 11217.629155 kIOPS : time 195.359 seconds
[RESULT] ior-easy-read 746.734899 GiB/s : time 425.448 seconds
[RESULT] mdtest-easy-stat 14312.108177 kIOPS : time 153.012 seconds
[RESULT] ior-hard-read 10.641976 GiB/s : time 223.986 seconds
[RESULT] mdtest-hard-stat 19.554671 kIOPS : time 58.513 seconds
[RESULT] mdtest-easy-delete 3147.974543 kIOPS : time 736.437 seconds
[RESULT] mdtest-hard-read 17.027393 kIOPS : time 67.197 seconds
[RESULT] mdtest-hard-delete 7.035865 kIOPS : time 163.875 seconds
[SCORE] Bandwidth 47.439998 GB/s : IOPS 229.328552 kiops : TOTAL 104.304104