-- started at 07/14/2020 15:02:26 -- mdtest-3.3.0+dev was launched with 1280 total task(s) on 40 node(s) Command line used: /work/BMLab/Lustre/io500_isc20/io500-app/bin/mdtest '-Y' '-C' '-F' '-P' '-d' '/ai400/datafiles/2020.07.14-14.55.35-scr/mdt_easy' '-n' '95000' '-u' '-L' '-a' 'POSIX' '-x' '/ai400/datafiles/2020.07.14-14.55.35-scr/mdt_easy-stonewall' '-N' '1' '-W' '300' Path: /ai400/datafiles/2020.07.14-14.55.35-scr FS: 94.9 TiB Used FS: 29.9% Inodes: 768.0 Mi Used Inodes: 0.0% Nodemap: 11111111111111111111111111111111000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 V-0: Rank 0 Line 2166 Shifting ranks by 32 for each phase. 1280 tasks, 121600000 files [1594706700.740772] [ec31:23151:0] tcp_cm.c:604 UCX WARN tcp_iface 0x100e2e0: connection establishment for socket fd 58 from to 10.128.11.31:44321 was unsuccessful [1594706731.932133] [ec31:23151:1] sock.c:344 UCX ERROR recv(fd=55) failed: Connection reset by peer [1594706731.932170] [ec31:23151:1] sock.c:344 UCX ERROR recv(fd=56) failed: Connection reset by peer [1594706731.932180] [ec31:23151:1] sock.c:344 UCX ERROR recv(fd=57) failed: Connection reset by peer [1594706731.932188] [ec31:23151:1] sock.c:344 UCX ERROR recv(fd=54) failed: Connection reset by peer [1594706731.932196] [ec31:23151:1] sock.c:344 UCX ERROR recv(fd=58) failed: Connection reset by peer [1594706820.116146] [ec31:23151:1] sock.c:344 UCX ERROR recv(fd=54) failed: Connection reset by peer [1594706830.770108] [ec31:23151:1] sock.c:344 UCX ERROR recv(fd=54) failed: Connection reset by peer Continue stonewall hit min: 85758 max: 95000 avg: 94184.3 SUMMARY rate: (of 1 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 378835.691 378835.480 378835.575 0.038 File stat : 0.000 0.000 0.000 0.000 File read : 0.000 0.000 0.000 0.000 File removal : 0.000 0.000 0.000 0.000 File create (stonewall) : NA NA 402457.892 NA Tree creation : 2.333 2.333 2.333 0.000 Tree removal : 0.000 0.000 0.000 0.000 SUMMARY time: (of 1 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 320.984 320.983 320.984 0.000 File stat : 0.000 0.000 0.000 0.000 File read : 0.000 0.000 0.000 0.000 File removal : 0.000 0.000 0.000 0.000 File create (stonewall) : NA NA 299.549 NA Tree creation : 0.429 0.429 0.429 0.000 Tree removal : 0.000 0.000 0.000 0.000 -- finished at 07/14/2020 15:07:48 --