- ior-easy-read
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Thu Jul 9 00:27:59 2020
Command line : ./ior -C -Q 1 -g -G 271 -k -e -o /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/stonewall -t 512k -b 9920000m -r -R -a POSIX --posix.odirect
Machine : Linux node0026
TestID : 0
StartTime : Thu Jul 9 00:27:59 2020
Path : /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy
FS : 18679.1 TiB Used FS: 41.1% Inodes: 3993.6 Mi Used Inodes: 12.3%
Options:
api : POSIX
apiVersion :
test filename : /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/ior_file_easy
access : single-shared-file
type : independent
segments : 1
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 2048
tasks : 2048
clients per node : 1
repetitions : 1
xfersize : 524288 bytes
blocksize : 9.46 TiB
aggregate filesize : 19375 TiB
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 21303037788160000.
WARNING: Stat() of aggregate file size = 21292808373010432.
WARNING: Using actual aggregate bytes moved = 353195561844736.
read 707361 1414794 0.000024 10158080000 512.00 0.024373 476.16 0.016056 476.18 0
Max Read: 707360.57 MiB/sec (741721.31 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
read 707360.57 707360.57 707360.57 0.00 1414721.13 1414721.13 1414721.13 0.00 476.18365 NA NA 0 2048 1 1 0 1 1 0 0 1 10401873920000 524288 336833536.0 POSIX 0
Finished : Thu Jul 9 00:35:56 2020
- ior-easy-write
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Thu Jul 9 00:01:06 2020
Command line : ./ior -C -Q 1 -g -G 271 -k -e -o /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/stonewall -t 512k -b 9920000m -w -D 300 -O stoneWallingWearOut=1 -a POSIX --posix.odirect
Machine : Linux node0026
TestID : 0
StartTime : Thu Jul 9 00:01:06 2020
Path : /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy
FS : 18679.1 TiB Used FS: 41.1% Inodes: 3993.6 Mi Used Inodes: 12.1%
Options:
api : POSIX
apiVersion :
test filename : /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/ior_file_easy
access : single-shared-file
type : independent
segments : 1
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 2048
tasks : 2048
clients per node : 1
repetitions : 1
xfersize : 524288 bytes
blocksize : 9.46 TiB
aggregate filesize : 19375 TiB
stonewallingTime : 300
stoneWallingWearOut : 1
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
stonewalling pairs accessed min: 187995 max: 328939 -- min data: 91.8 GiB mean data: 118.2 GiB time: 300.4s
WARNING: Expected aggregate file size = 21303037788160000.
WARNING: Stat() of aggregate file size = 21292808373010432.
WARNING: Using actual aggregate bytes moved = 353195561844736.
WARNING: maybe caused by deadlineForStonewalling
write 806622 1613345 0.000017 10158080000 512.00 0.024491 417.56 0.003990 417.59 0
Max Write: 806621.75 MiB/sec (845804.21 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
write 806621.75 806621.75 806621.75 0.00 1613243.50 1613243.50 1613243.50 0.00 417.58549 300.40 824842.43 0 2048 1 1 0 1 1 0 0 1 10401873920000 524288 336833536.0 POSIX 0
Finished : Thu Jul 9 00:08:04 2020
- ior-hard-read
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Thu Jul 9 00:36:00 2020
Command line : ./ior -C -Q 1 -g -G 27 -k -e -o /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/file -O stoneWallingStatusFile=/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 4800000 -r -R -a POSIX --posix.odirect
Machine : Linux node0026
TestID : 0
StartTime : Thu Jul 9 00:36:00 2020
Path : /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard
FS : 18679.1 TiB Used FS: 41.1% Inodes: 3993.6 Mi Used Inodes: 12.2%
Options:
api : POSIX
apiVersion :
test filename : /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/file
access : single-shared-file
type : independent
segments : 4800000
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 2048
tasks : 2048
clients per node : 1
repetitions : 1
xfersize : 47008 bytes
blocksize : 47008 bytes
aggregate filesize : 420.28 TiB
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
stonewalling pairs accessed min: 3005027 max: 3005027 -- min data: 131.6 GiB mean data: 131.6 GiB time: 1535.7s
WARNING: Expected aggregate file size = 462107443200000.
WARNING: Stat() of aggregate file size = 289301113274368.
WARNING: Using actual aggregate bytes moved = 289301113274368.
read 179605 4006473 1536.07 45.91 45.91 0.027609 1536.09 0.060491 1536.14 0
Max Read: 179605.40 MiB/sec (188329.92 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
read 179605.40 179605.40 179605.40 0.00 4006337.57 4006337.57 4006337.57 0.00 1536.13998 1535.66 179662.01 0 2048 1 1 0 1 1 0 0 4800000 47008 47008 275899040.0 POSIX 0
Finished : Thu Jul 9 01:01:37 2020
- ior-hard-write
-
IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O
Began : Thu Jul 9 00:14:43 2020
Command line : ./ior -C -Q 1 -g -G 27 -k -e -o /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/file -O stoneWallingStatusFile=/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 4800000 -w -D 300 -a POSIX --posix.odirect
Machine : Linux node0026
TestID : 0
StartTime : Thu Jul 9 00:14:43 2020
Path : /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard
FS : 18679.1 TiB Used FS: 41.1% Inodes: 3993.6 Mi Used Inodes: 12.2%
Options:
api : POSIX
apiVersion :
test filename : /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/file
access : single-shared-file
type : independent
segments : 4800000
ordering in a file : sequential
ordering inter file : constant task offset
task offset : 1
nodes : 2048
tasks : 2048
clients per node : 1
repetitions : 1
xfersize : 47008 bytes
blocksize : 47008 bytes
aggregate filesize : 420.28 TiB
stonewallingTime : 300
stoneWallingWearOut : 1
Results:
access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ----
stonewalling pairs accessed min: 2065597 max: 3005027 -- min data: 90.4 GiB mean data: 115.3 GiB time: 300.2s
WARNING: Expected aggregate file size = 462107443200000.
WARNING: Stat() of aggregate file size = 289301113274368.
WARNING: Using actual aggregate bytes moved = 289301113274368.
WARNING: maybe caused by deadlineForStonewalling
write 758186 16929052 319.06 45.91 45.91 0.359954 363.53 0.007313 363.89 0
Max Write: 758185.91 MiB/sec (795015.55 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum
write 758185.91 758185.91 758185.91 0.00 16912345.77 16912345.77 16912345.77 0.00 363.89365 300.21 805217.92 0 2048 1 1 0 1 1 0 0 4800000 47008 47008 275899040.0 POSIX 0
Finished : Thu Jul 9 00:20:47 2020
- mdtest-easy-delete
-
-- started at 07/09/2020 01:01:40 --
mdtest-3.3.0+dev was launched with 2048 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '250000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy' '-x' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy-stonewall' '-r' '-a' 'POSIX' '--posix.odirect'
Path: /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app
FS: 18679.1 TiB Used FS: 41.1% Inodes: 3993.6 Mi Used Inodes: 12.2%
Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 1 for each phase.
2048 tasks, 512000000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 28238.310 28238.047 28238.170 0.048
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 1.107 1.107 1.107 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 276.107 276.105 276.106 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.903 0.903 0.903 0.000
-- finished at 07/09/2020 01:06:17 --
- mdtest-easy-stat
-
-- started at 07/09/2020 00:35:56 --
mdtest-3.3.0+dev was launched with 2048 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '250000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy' '-x' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy-stonewall' '-T' '-a' 'POSIX' '--posix.odirect'
Path: /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app
FS: 18679.1 TiB Used FS: 41.1% Inodes: 3993.6 Mi Used Inodes: 12.2%
Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 1 for each phase.
2048 tasks, 512000000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 1667478.808 1667442.656 1667459.547 4.810
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 4.676 4.676 4.676 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/09/2020 00:36:00 --
- mdtest-easy-write
-
-- started at 07/09/2020 00:08:04 --
mdtest-3.3.0+dev was launched with 2048 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '250000' '-u' '-L' '-F' '-P' '-N' '1' '-d' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy' '-x' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy-stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX' '--posix.odirect'
Path: /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app
FS: 18679.1 TiB Used FS: 41.1% Inodes: 3993.6 Mi Used Inodes: 12.0%
Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 1 for each phase.
2048 tasks, 512000000 files
Continue stonewall hit min: 3210 max: 3807 avg: 3477.7
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 22622.252 22622.044 22622.140 0.038
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 23689.373 NA
Tree creation : 17.977 17.977 17.977 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 344.652 344.649 344.651 0.001
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 300.659 NA
Tree creation : 0.056 0.056 0.056 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/09/2020 00:14:43 --
- mdtest-hard-delete
-
-- started at 07/09/2020 01:06:21 --
mdtest-3.3.0+dev was launched with 2048 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '20000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard' '-x' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard-stonewall' '-r' '-a' 'POSIX' '--posix.odirect'
Path: /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app
FS: 18679.1 TiB Used FS: 41.1% Inodes: 3993.6 Mi Used Inodes: 12.0%
Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 1 for each phase.
2048 tasks, 40960000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 18377.692 18377.523 18377.601 0.031
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 1.712 1.712 1.712 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 297.435 297.432 297.433 0.001
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.584 0.584 0.584 0.000
-- finished at 07/09/2020 01:11:20 --
- mdtest-hard-read
-
-- started at 07/09/2020 01:06:17 --
mdtest-3.3.0+dev was launched with 2048 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '20000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard' '-x' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard-stonewall' '-E' '-X' '-a' 'POSIX' '--posix.odirect'
Path: /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app
FS: 18679.1 TiB Used FS: 41.1% Inodes: 3993.6 Mi Used Inodes: 12.0%
Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 1 for each phase.
2048 tasks, 40960000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 1320610.907 1320585.418 1320598.426 4.431
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 0.000 0.000 0.000 0.000
File read : 4.139 4.139 4.139 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/09/2020 01:06:21 --
- mdtest-hard-stat
-
-- started at 07/09/2020 01:01:37 --
mdtest-3.3.0+dev was launched with 2048 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '20000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard' '-x' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard-stonewall' '-T' '-a' 'POSIX' '--posix.odirect'
Path: /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app
FS: 18679.1 TiB Used FS: 41.1% Inodes: 3993.6 Mi Used Inodes: 12.2%
Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 1 for each phase.
2048 tasks, 40960000 files
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 1659266.805 1659221.029 1659243.584 8.225
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 0.000 0.000 0.000 0.000
File stat : 3.294 3.294 3.294 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
Tree creation : 0.000 0.000 0.000 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/09/2020 01:01:40 --
- mdtest-hard-write
-
-- started at 07/09/2020 00:20:47 --
mdtest-3.3.0+dev was launched with 2048 total task(s) on 2048 node(s)
Command line used: ./mdtest '-n' '20000' '-t' '-w' '3901' '-e' '3901' '-P' '-N' '1' '-F' '-d' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard' '-x' '/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard-stonewall' '-C' '-Y' '-W' '300' '-a' 'POSIX' '--posix.odirect'
Path: /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app
FS: 18679.1 TiB Used FS: 41.1% Inodes: 3993.6 Mi Used Inodes: 12.2%
Nodemap: 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
V-0: Rank 0 Line 2166 Shifting ranks by 1 for each phase.
2048 tasks, 40960000 files
Continue stonewall hit min: 2213 max: 2669 avg: 2433.1
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 15667.155 15667.011 15667.077 0.027
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 16589.971 NA
Tree creation : 452.017 452.017 452.017 0.000
Tree removal : 0.000 0.000 0.000 0.000
SUMMARY time: (of 1 iterations)
Operation Max Min Mean Std Dev
--------- --- --- ---- -------
File creation : 348.893 348.890 348.892 0.001
File stat : 0.000 0.000 0.000 0.000
File read : 0.000 0.000 0.000 0.000
File removal : 0.000 0.000 0.000 0.000
File create (stonewall) : NA NA 300.366 NA
Tree creation : 0.002 0.002 0.002 0.000
Tree removal : 0.000 0.000 0.000 0.000
-- finished at 07/09/2020 00:27:32 --
- result
-
version = io500-isc20_v4-2-g20efd247106c
config-hash = 14F6BBD8
result-dir = /scratch/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/results/2020.07.08-23.59.24-app
; START 2020-07-09 00:01:06
[ior-easy-write]
t_start = 2020-07-09 00:01:06
exe = ./ior -C -Q 1 -g -G 271 -k -e -o /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/stonewall -t 512k -b 9920000m -w -D 300 -O stoneWallingWearOut=1 -a POSIX --posix.odirect
throughput-stonewall = 1186728.23
score = 787.716552
t_delta = 417.7037
t_end = 2020-07-09 00:08:04
[mdtest-easy-write]
t_start = 2020-07-09 00:08:04
exe = ./mdtest -n 250000 -u -L -F -P -N 1 -d /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy -x /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy-stonewall -C -Y -W 300 -a POSIX --posix.odirect
rate-stonewall = 23.689373
score = 22.622094
t_delta = 398.6679
t_end = 2020-07-09 00:14:43
[timestamp]
t_start = 2020-07-09 00:14:43
t_delta = 0.0978
t_end = 2020-07-09 00:14:43
[ior-hard-write]
t_start = 2020-07-09 00:14:43
exe = ./ior -C -Q 1 -g -G 27 -k -e -o /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/file -O stoneWallingStatusFile=/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 4800000 -w -D 300 -a POSIX --posix.odirect
throughput-stonewall = 1328595.19
score = 740.415929
t_delta = 364.0428
t_end = 2020-07-09 00:20:47
[mdtest-hard-write]
t_start = 2020-07-09 00:20:47
exe = ./mdtest -n 20000 -t -w 3901 -e 3901 -P -N 1 -F -d /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard -x /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard-stonewall -C -Y -W 300 -a POSIX --posix.odirect
rate-stonewall = 16.589971
score = 15.667045
t_delta = 405.2319
t_end = 2020-07-09 00:27:32
[find]
t_start = 2020-07-09 00:27:32
exe = ./pfind /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app -newer /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/timestampfile -size 3901c -name *01* -C -q 10000
found = 199128
total-files = 13262855
score = 489.039063
t_delta = 27.3482
t_end = 2020-07-09 00:27:59
[ior-easy-read]
t_start = 2020-07-09 00:27:59
exe = ./ior -C -Q 1 -g -G 271 -k -e -o /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/ior_file_easy -O stoneWallingStatusFile=/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-easy/stonewall -t 512k -b 9920000m -r -R -a POSIX --posix.odirect
score = 690.781803
t_delta = 476.3229
t_end = 2020-07-09 00:35:56
[mdtest-easy-stat]
t_start = 2020-07-09 00:35:56
exe = ./mdtest -n 250000 -u -L -F -P -N 1 -d /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy -x /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy-stonewall -T -a POSIX --posix.odirect
score = 1667.462554
t_delta = 4.7633
t_end = 2020-07-09 00:36:00
[ior-hard-read]
t_start = 2020-07-09 00:36:00
exe = ./ior -C -Q 1 -g -G 27 -k -e -o /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/file -O stoneWallingStatusFile=/scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/ior-hard/stonewall -O stoneWallingWearOut=1 -t 47008 -b 47008 -s 4800000 -r -R -a POSIX --posix.odirect
score = 175.395903
t_delta = 1536.2529
t_end = 2020-07-09 01:01:37
[mdtest-hard-stat]
t_start = 2020-07-09 01:01:37
exe = ./mdtest -n 20000 -t -w 3901 -e 3901 -P -N 1 -F -d /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard -x /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard-stonewall -T -a POSIX --posix.odirect
score = 1659.243152
t_delta = 3.3771
t_end = 2020-07-09 01:01:40
[mdtest-easy-delete]
t_start = 2020-07-09 01:01:40
exe = ./mdtest -n 250000 -u -L -F -P -N 1 -d /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy -x /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-easy-stonewall -r -a POSIX --posix.odirect
score = 28.238112
t_delta = 277.1142
t_end = 2020-07-09 01:06:17
[mdtest-hard-read]
t_start = 2020-07-09 01:06:17
exe = ./mdtest -n 20000 -t -w 3901 -e 3901 -P -N 1 -F -d /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard -x /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard-stonewall -E -X -a POSIX --posix.odirect
score = 1320.590582
t_delta = 4.2257
t_end = 2020-07-09 01:06:21
[mdtest-hard-delete]
t_start = 2020-07-09 01:06:21
exe = ./mdtest -n 20000 -t -w 3901 -e 3901 -P -N 1 -F -d /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard -x /scratch_ime/jwyoon/IO500/ISC20/isc20v2/io500-app-7PM/datafiles/2020.07.08-23.59.24-app/mdtest-hard-stonewall -r -a POSIX --posix.odirect
score = 18.377565
t_delta = 298.2373
t_end = 2020-07-09 01:11:20
[SCORE]
MD = 154.735576
BW = 515.586473
SCORE = 282.452774
hash = 237BEA02
; END 2020-07-09 01:11:21
- result_summary
-
IO500 version io500-isc20_v4-2-g20efd247106c
[RESULT] ior-easy-write 787.716552 GiB/s : time 417.704 seconds
[RESULT] mdtest-easy-write 22.622094 kIOPS : time 398.668 seconds
[RESULT] ior-hard-write 740.415929 GiB/s : time 364.043 seconds
[RESULT] mdtest-hard-write 15.667045 kIOPS : time 405.232 seconds
[RESULT] find 489.039063 kIOPS : time 27.348 seconds
[RESULT] ior-easy-read 690.781803 GiB/s : time 476.323 seconds
[RESULT] mdtest-easy-stat 1667.462554 kIOPS : time 4.763 seconds
[RESULT] ior-hard-read 175.395903 GiB/s : time 1536.253 seconds
[RESULT] mdtest-hard-stat 1659.243152 kIOPS : time 3.377 seconds
[RESULT] mdtest-easy-delete 28.238112 kIOPS : time 277.114 seconds
[RESULT] mdtest-hard-read 1320.590582 kIOPS : time 4.226 seconds
[RESULT] mdtest-hard-delete 18.377565 kIOPS : time 298.237 seconds
[SCORE] Bandwidth 515.586473 GB/s : IOPS 154.735576 kiops : TOTAL 282.452774