- ior-easy-read
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Tue Jul 7 16:55:46 2020
Command line : ./ior -v -C -Q 1 -g -G 271 -k -e -o /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/stonewall -t 2m -b 9920000m -F -r -R -a POSIX --posix.odirect
Machine : Linux n02p001.gvr.local
Start time skew across all tasks: 260452.10 sec
TestID : 0
StartTime : Tue Jul 7 16:55:46 2020
Path : /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy
FS : 259.3 TiB Used FS: 55.1% Inodes: 1202.1 Mi Used Inodes: 17.8%
Participating tasks: 800
Using reorderTasks '-C' (useful to avoid read cache in client)
Options:
api : POSIX
apiVersion :
test filename : /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/ior_file_easy
access : file-per-process
type : independent
segments : 1
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 50
tasks : 800
clients per node : 16
repetitions : 1
xfersize : 2 MiB
blocksize : 9.46 TiB
aggregate filesize : 7568.36 TiB
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
Commencing read performance test: Tue Jul 7 16:55:48 2020
WARNING: Expected aggregate file size = 8321499136000000.
WARNING: Stat() of aggregate file size = 79440117760000.
WARNING: Using actual aggregate bytes moved = 79440117760000.
read 82525 41300 0.000185 10158080000 2048.00 0.853655 917.20 0.754555 918.03 0
Max Read: 82524.93 MiB/sec (86533.66 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
read 82524.93 82524.93 82524.93 0.00 41262.46 41262.46 41262.46 0.00 918.02565 NA NA 0 800 16 1 1 1 1 0 0 1 10401873920000 2097152 75760000.0 POSIX 0
Finished : Tue Jul 7 17:11:07 2020
- ior-easy-write
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Tue Jul 7 16:09:46 2020
Command line : ./ior -v -C -Q 1 -g -G 271 -k -e -o /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/stonewall -t 2m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX --posix.odirect
Machine : Linux n02p001.gvr.local
Start time skew across all tasks: 260452.01 sec
TestID : 0
StartTime : Tue Jul 7 16:09:46 2020
Path : /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy
FS : 259.3 TiB Used FS: 25.7% Inodes: 1047.6 Mi Used Inodes: 2.7%
Participating tasks: 800
Using reorderTasks '-C' (useful to avoid read cache in client)
Options:
api : POSIX
apiVersion :
test filename : /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/ior_file_easy
access : file-per-process
type : independent
segments : 1
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 50
tasks : 800
clients per node : 16
repetitions : 1
xfersize : 2 MiB
blocksize : 9.46 TiB
aggregate filesize : 7568.36 TiB
stonewallingTime : 300
stoneWallingWearOut : 1
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
Commencing write performance test: Tue Jul 7 16:09:48 2020
0: stonewalling pairs accessed: 41748
stonewalling pairs accessed min: 21781 max: 47350 -- min data: 42.5 GiB mean data: 69.1 GiB time: 300.5s
WARNING: Expected aggregate file size = 8321499136000000.
WARNING: Stat() of aggregate file size = 79440117760000.
WARNING: Using actual aggregate bytes moved = 79440117760000.
WARNING: maybe caused by deadlineForStonewalling
write 139879 70049 0.000068 10158080000 2048.00 0.491230 540.76 0.438729 541.61 0
Max Write: 139879.45 MiB/sec (146674.24 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
write 139879.45 139879.45 139879.45 0.00 69939.73 69939.73 69939.73 0.00 541.60921 300.53 188461.65 0 800 16 1 1 1 1 0 0 1 10401873920000 2097152 75760000.0 POSIX 0
Finished : Tue Jul 7 16:18:51 2020
- ior-hard-read
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Tue Jul 7 17:14:51 2020
Command line : ./ior -v -C -Q 1 -g -G 27 -k -e -o /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/file -O stoneWallingStatusFile=/lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 110000 -r -R -a MPIIO
Machine : Linux n02p001.gvr.local
Start time skew across all tasks: 260452.29 sec
TestID : 0
StartTime : Tue Jul 7 17:14:51 2020
Path : /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard
FS : 259.3 TiB Used FS: 55.1% Inodes: 1202.1 Mi Used Inodes: 17.8%
Participating tasks: 800
Using reorderTasks '-C' (useful to avoid read cache in client)
Options:
api : MPIIO
apiVersion : (3.1)
test filename : /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/file
access : single-shared-file
type : independent
segments : 110000
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 50
tasks : 800
clients per node : 16
repetitions : 1
xfersize : 47008 bytes
blocksize : 47008 bytes
aggregate filesize : 3.76 TiB
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
Commencing read performance test: Tue Jul 7 17:14:58 2020
0: stonewalling pairs accessed: 110000
stonewalling pairs accessed min: 110000 max: 110000 -- min data: 4.8 GiB mean data: 4.8 GiB time: 260.3s
read 14740 336201 260.84 45.91 45.91 5.78 261.75 1.21 267.65 0
Max Read: 14739.58 MiB/sec (15455.57 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
read 14739.58 14739.58 14739.58 0.00 328785.95 328785.95 328785.95 0.00 267.65134 260.28 15156.91 0 800 16 1 0 1 1 0 0 110000 47008 47008 3945068.2 MPIIO 0
Finished : Tue Jul 7 17:19:27 2020
- ior-hard-write
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Tue Jul 7 16:25:20 2020
Command line : ./ior -v -C -Q 1 -g -G 27 -k -e -o /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/file -O stoneWallingStatusFile=/lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 110000 -w -D 300 -a MPIIO
Machine : Linux n02p001.gvr.local
Start time skew across all tasks: 260451.91 sec
TestID : 0
StartTime : Tue Jul 7 16:25:20 2020
Path : /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard
FS : 259.3 TiB Used FS: 53.6% Inodes: 1202.1 Mi Used Inodes: 15.2%
Participating tasks: 800
Using reorderTasks '-C' (useful to avoid read cache in client)
Options:
api : MPIIO
apiVersion : (3.1)
test filename : /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/file
access : single-shared-file
type : independent
segments : 110000
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 50
tasks : 800
clients per node : 16
repetitions : 1
xfersize : 47008 bytes
blocksize : 47008 bytes
aggregate filesize : 3.76 TiB
stonewallingTime : 300
stoneWallingWearOut : 1
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
Commencing write performance test: Tue Jul 7 16:26:20 2020
0: stonewalling pairs accessed: 110000
stonewalling pairs accessed min: 110000 max: 110000 -- min data: 4.8 GiB mean data: 4.8 GiB time: 292.8s
write 11190 299428 293.43 45.91 45.91 58.30 293.89 1.13 352.56 0
Max Write: 11189.74 MiB/sec (11733.29 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
write 11189.74 11189.74 11189.74 0.00 249602.07 249602.07 249602.07 0.00 352.56118 292.76 13475.35 0 800 16 1 0 1 1 0 0 110000 47008 47008 3945068.2 MPIIO 0
Finished : Tue Jul 7 16:31:23 2020
- mdtest-easy-delete
-
-- started at 07/07/2020 17:22:27 --
mdtest-3.3.0+dev was launched with 800 total task(s) on 50 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy' '-x' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy-stonewall' '-r' '-a' 'POSIX'
Path: /lustre/stor1/io500/2020.07.07-16.09.21-app
FS: 259.3 TiB Used FS: 55.1% Inodes: 1202.1 Mi Used Inodes: 17.8%
Nodemap: 11111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 16 for each phase.
800 tasks, 800000000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 354373.813 354064.083 354208.387 55.782
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.781 0.781 0.781 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 457.260 456.860 457.073 0.072
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 1.281 1.281 1.281 0.000
-- finished at 07/07/2020 17:30:52 --
- mdtest-easy-stat
-
-- started at 07/07/2020 17:11:17 --
mdtest-3.3.0+dev was launched with 800 total task(s) on 50 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy' '-x' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy-stonewall' '-T' '-a' 'POSIX'
Path: /lustre/stor1/io500/2020.07.07-16.09.21-app
FS: 259.3 TiB Used FS: 55.1% Inodes: 1202.1 Mi Used Inodes: 17.8%
Nodemap: 11111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 16 for each phase.
800 tasks, 800000000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 816836.658 815367.031 816070.154 278.066
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 198.560 198.203 198.389 0.068
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/07/2020 17:14:41 --
- mdtest-easy-write
-
-- started at 07/07/2020 16:19:02 --
mdtest-3.3.0+dev was launched with 800 total task(s) on 50 node(s)
Command line used: ./mdtest '-n' '1000000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy' '-x' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy-stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /lustre/stor1/io500/2020.07.07-16.09.21-app
FS: 259.3 TiB Used FS: 53.6% Inodes: 1047.6 Mi Used Inodes: 2.7%
Nodemap: 11111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 16 for each phase.
800 tasks, 800000000 files
Continue stonewall hit min: 181525 max: 202374 avg: 192275.9
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 472273.244 469846.052 472016.626 225.831
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 515556.233 NA
Tree creation : 1.083 1.083 1.083 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 344.579 342.808 342.995 0.165
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 298.359 NA
Tree creation : 0.923 0.923 0.923 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/07/2020 16:25:08 --
- mdtest-hard-delete
-
-- started at 07/07/2020 17:34:25 --
mdtest-3.3.0+dev was launched with 800 total task(s) on 50 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard' '-x' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard-stonewall' '-r' '-a' 'POSIX'
Path: /lustre/stor1/io500/2020.07.07-16.09.21-app
FS: 259.3 TiB Used FS: 55.1% Inodes: 1047.6 Mi Used Inodes: 5.7%
Nodemap: 11111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 16 for each phase.
800 tasks, 800000000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 66885.062 66837.808 66859.725 7.889
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.108 0.108 0.108 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 480.949 480.610 480.792 0.057
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 9.289 9.289 9.289 0.000
-- finished at 07/07/2020 17:42:40 --
- mdtest-hard-read
-
-- started at 07/07/2020 17:31:02 --
mdtest-3.3.0+dev was launched with 800 total task(s) on 50 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard' '-x' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard-stonewall' '-E' '-X' '-a' 'POSIX'
Path: /lustre/stor1/io500/2020.07.07-16.09.21-app
FS: 259.3 TiB Used FS: 55.1% Inodes: 1047.6 Mi Used Inodes: 5.7%
Nodemap: 11111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 16 for each phase.
800 tasks, 800000000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 170876.264 170600.992 170756.194 54.466
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 188.426 188.122 188.254 0.060
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/07/2020 17:34:15 --
- mdtest-hard-stat
-
-- started at 07/07/2020 17:19:36 --
mdtest-3.3.0+dev was launched with 800 total task(s) on 50 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard' '-x' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard-stonewall' '-T' '-a' 'POSIX'
Path: /lustre/stor1/io500/2020.07.07-16.09.21-app
FS: 259.3 TiB Used FS: 55.1% Inodes: 1202.1 Mi Used Inodes: 17.8%
Nodemap: 11111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 16 for each phase.
800 tasks, 800000000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 205554.347 205015.935 205365.077 92.612
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 156.796 156.385 156.529 0.071
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/07/2020 17:22:18 --
- mdtest-hard-write
-
-- started at 07/07/2020 16:31:34 --
mdtest-3.3.0+dev was launched with 800 total task(s) on 50 node(s)
Command line used: ./mdtest '-n' '1000000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard' '-x' '/lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard-stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX'
Path: /lustre/stor1/io500/2020.07.07-16.09.21-app
FS: 259.3 TiB Used FS: 55.0% Inodes: 1202.1 Mi Used Inodes: 15.2%
Nodemap: 11111111111111110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 16 for each phase.
800 tasks, 800000000 files
Continue stonewall hit min: 37381 max: 40182 avg: 38548.1
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 72431.256 72383.478 72409.336 11.832
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 103319.515 NA
Tree creation : 1.332 1.332 1.332 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 444.101 443.808 443.943 0.073
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 298.476 NA
Tree creation : 0.751 0.751 0.751 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/07/2020 16:39:44 --
- result
-
version = io500-isc20_v4-1-g18c6d03b40b0
config-hash = 9141A89C
result-dir = ./results/2020.07.07-16.09.21-app
; START 2020-07-07 16:09:32
[ior-easy-write]
t_start = 2020-07-07 16:09:33
exe = ./ior -v -C -Q 1 -g -G 271 -k -e -o /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/stonewall -t 2m -b 9920000m -F -w -D 300 -O stoneWallingWearOut=1 -a POSIX --posix.odirect
throughput-stonewall = 81697.76
score = 136.601028
t_delta = 558.3757
t_end = 2020-07-07 16:18:51
[mdtest-easy-write]
t_start = 2020-07-07 16:18:51
exe = ./mdtest -n 1000000 -u -L -F -P -N 1 -d /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy -x /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy-stonewall -C -Y -W 300 -a POSIX
rate-stonewall = 515.556233
score = 471.965676
t_delta = 377.0627
t_end = 2020-07-07 16:25:08
[timestamp]
t_start = 2020-07-07 16:25:09
t_delta = 0.0180
t_end = 2020-07-07 16:25:09
[ior-hard-write]
t_start = 2020-07-07 16:25:09
exe = ./ior -v -C -Q 1 -g -G 27 -k -e -o /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/file -O stoneWallingStatusFile=/lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 110000 -w -D 300 -a MPIIO
throughput-stonewall = 8741.99
score = 10.927482
t_delta = 373.6386
t_end = 2020-07-07 16:31:23
[mdtest-hard-write]
t_start = 2020-07-07 16:31:23
exe = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard -x /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard-stonewall -C -Y -W 300 -a POSIX
rate-stonewall = 103.319515
score = 72.405949
t_delta = 500.7195
t_end = 2020-07-07 16:39:44
[find]
t_start = 2020-07-07 16:39:44
exe = ./pfind /lustre/stor1/io500/2020.07.07-16.09.21-app -newer /lustre/stor1/io500/2020.07.07-16.09.21-app/timestampfile -size 3901c -name *01* -C -q 10000
found = 1216221
total-files = 194045606
score = 204.177825
t_delta = 951.0903
t_end = 2020-07-07 16:55:35
[ior-easy-read]
t_start = 2020-07-07 16:55:35
exe = ./ior -v -C -Q 1 -g -G 271 -k -e -o /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/lustre/stor1/io500/2020.07.07-16.09.21-app/ior-easy/stonewall -t 2m -b 9920000m -F -r -R -a POSIX --posix.odirect
score = 80.590750
t_delta = 931.6662
t_end = 2020-07-07 17:11:07
[mdtest-easy-stat]
t_start = 2020-07-07 17:11:07
exe = ./mdtest -n 1000000 -u -L -F -P -N 1 -d /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy -x /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy-stonewall -T -a POSIX
score = 816.135984
t_delta = 213.3078
t_end = 2020-07-07 17:14:41
[ior-hard-read]
t_start = 2020-07-07 17:14:41
exe = ./ior -v -C -Q 1 -g -G 27 -k -e -o /lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/file -O stoneWallingStatusFile=/lustre/stor1/io500/2020.07.07-16.09.21-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 110000 -r -R -a MPIIO
score = 14.394121
t_delta = 285.9153
t_end = 2020-07-07 17:19:27
[mdtest-hard-stat]
t_start = 2020-07-07 17:19:27
exe = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard -x /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard-stonewall -T -a POSIX
score = 205.475790
t_delta = 171.1807
t_end = 2020-07-07 17:22:18
[mdtest-easy-delete]
t_start = 2020-07-07 17:22:18
exe = ./mdtest -n 1000000 -u -L -F -P -N 1 -d /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy -x /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-easy-stonewall -r -a POSIX
score = 354.219982
t_delta = 514.1532
t_end = 2020-07-07 17:30:52
[mdtest-hard-read]
t_start = 2020-07-07 17:30:52
exe = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard -x /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard-stonewall -E -X -a POSIX
score = 170.793932
t_delta = 202.5748
t_end = 2020-07-07 17:34:15
[mdtest-hard-delete]
t_start = 2020-07-07 17:34:15
exe = ./mdtest -n 1000000 -t -w 3901 -e 3901 -P -N 1 -F -d /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard -x /lustre/stor1/io500/2020.07.07-16.09.21-app/mdtest-hard-stonewall -r -a POSIX
score = 66.860460
t_delta = 504.8564
t_end = 2020-07-07 17:42:40
[SCORE]
MD = 215.968997
BW = 36.275307
SCORE = 88.511817
hash = 1623C448
; END 2020-07-07 17:42:44
- result_summary
-
IO500 version io500-isc20_v4-1-g18c6d03b40b0
[RESULT] ior-easy-write 136.601028 GiB/s : time 558.376 seconds
[RESULT] mdtest-easy-write 471.965676 kIOPS : time 377.063 seconds
[RESULT] ior-hard-write 10.927482 GiB/s : time 373.639 seconds
[RESULT] mdtest-hard-write 72.405949 kIOPS : time 500.719 seconds
[RESULT] find 204.177825 kIOPS : time 951.090 seconds
[RESULT] ior-easy-read 80.590750 GiB/s : time 931.666 seconds
[RESULT] mdtest-easy-stat 816.135984 kIOPS : time 213.308 seconds
[RESULT] ior-hard-read 14.394121 GiB/s : time 285.915 seconds
[RESULT] mdtest-hard-stat 205.475790 kIOPS : time 171.181 seconds
[RESULT] mdtest-easy-delete 354.219982 kIOPS : time 514.153 seconds
[RESULT] mdtest-hard-read 170.793932 kIOPS : time 202.575 seconds
[RESULT] mdtest-hard-delete 66.860460 kIOPS : time 504.856 seconds
[SCORE] Bandwidth 36.275307 GB/s : IOPS 215.968997 kiops : TOTAL 88.511817