IOR-3.3.0+dev: MPI Coordinated Test of Parallel I/O Began : Sat Jul 11 07:45:55 2020 Command line : /opt/basis/rsc/io500/io500-app/bin/ior -w -s 34000 -a MPIIO --posix.odirect -v -i 1 -C -Q 1 -g -G 27 -k -e -t 47008 -b 47008 -o /lustre/iospb/io500-2020spring/datafiles/2020.07.11-05.26.49-scr/ior_hard/IOR_file -O stoneWallingStatusFile=/lustre/iospb/io500-2020spring/datafiles/2020.07.11-05.26.49-scr/ior_hard/stonewall -O stoneWallingWearOut=1 -D 300 Machine : Linux n06p001 Start time skew across all tasks: 983255.36 sec TestID : 0 StartTime : Sat Jul 11 07:45:55 2020 Path : /lustre/iospb/io500-2020spring/datafiles/2020.07.11-05.26.49-scr/ior_hard FS : 196.3 TiB Used FS: 22.8% Inodes: 905.9 Mi Used Inodes: 12.4% Participating tasks: 944 Using reorderTasks '-C' (useful to avoid read cache in client) Options: api : MPIIO apiVersion : (3.1) test filename : /lustre/iospb/io500-2020spring/datafiles/2020.07.11-05.26.49-scr/ior_hard/IOR_file access : single-shared-file type : independent segments : 34000 ordering in a file : sequential ordering inter file : constant task offset task offset : 1 nodes : 59 tasks : 944 clients per node : 16 repetitions : 1 xfersize : 47008 bytes blocksize : 47008 bytes aggregate filesize : 1.37 TiB stonewallingTime : 300 stoneWallingWearOut : 1 Results: access bw(MiB/s) IOPS Latency(s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter ------ --------- ---- ---------- ---------- --------- -------- -------- -------- -------- ---- Commencing write performance test: Sat Jul 11 07:46:12 2020 696: stonewalling pairs accessed: 19869 497: stonewalling pairs accessed: 19938 498: stonewalling pairs accessed: 19451 697: stonewalling pairs accessed: 20260 23: stonewalling pairs accessed: 20071 446: stonewalling pairs accessed: 19633 7: stonewalling pairs accessed: 15008 694: stonewalling pairs accessed: 19632 445: stonewalling pairs accessed: 19387 496: stonewalling pairs accessed: 19539 432: stonewalling pairs accessed: 19347 19: stonewalling pairs accessed: 19372 31: stonewalling pairs accessed: 20043 22: stonewalling pairs accessed: 19745 1: stonewalling pairs accessed: 15273 689: stonewalling pairs accessed: 20261 434: stonewalling pairs accessed: 19317 505: stonewalling pairs accessed: 19556 501: stonewalling pairs accessed: 19810 502: stonewalling pairs accessed: 20156 438: stonewalling pairs accessed: 19089 692: stonewalling pairs accessed: 20255 439: stonewalling pairs accessed: 19582 693: stonewalling pairs accessed: 19426 5: stonewalling pairs accessed: 15123 700: stonewalling pairs accessed: 19869 21: stonewalling pairs accessed: 20071 444: stonewalling pairs accessed: 19938 511: stonewalling pairs accessed: 20164 691: stonewalling pairs accessed: 19994 620: stonewalling pairs accessed: 23199 508: stonewalling pairs accessed: 20163 690: stonewalling pairs accessed: 19655 591: stonewalling pairs accessed: 20852 291: stonewalling pairs accessed: 19951 442: stonewalling pairs accessed: 19927 796: stonewalling pairs accessed: 20787 353: stonewalling pairs accessed: 19569 623: stonewalling pairs accessed: 23505 437: stonewalling pairs accessed: 18870 440: stonewalling pairs accessed: 19163 431: stonewalling pairs accessed: 23550 17: stonewalling pairs accessed: 19409 16: stonewalling pairs accessed: 19182 4: stonewalling pairs accessed: 16041 584: stonewalling pairs accessed: 20812 298: stonewalling pairs accessed: 20450 216: stonewalling pairs accessed: 19805 535: stonewalling pairs accessed: 20085 418: stonewalling pairs accessed: 23153 504: stonewalling pairs accessed: 20161 829: stonewalling pairs accessed: 20127 784: stonewalling pairs accessed: 20784 357: stonewalling pairs accessed: 20126 703: stonewalling pairs accessed: 20354 611: stonewalling pairs accessed: 22933 528: stonewalling pairs accessed: 19713 436: stonewalling pairs accessed: 18675 509: stonewalling pairs accessed: 19780 365: stonewalling pairs accessed: 19736 688: stonewalling pairs accessed: 19814 621: stonewalling pairs accessed: 23499 680: stonewalling pairs accessed: 22583 587: stonewalling pairs accessed: 20877 27: stonewalling pairs accessed: 19410 695: stonewalling pairs accessed: 19757 58: stonewalling pairs accessed: 20329 0: stonewalling pairs accessed: 14999 28: stonewalling pairs accessed: 19856 752: stonewalling pairs accessed: 20482 12: stonewalling pairs accessed: 15069 292: stonewalling pairs accessed: 19998 112: stonewalling pairs accessed: 20229 209: stonewalling pairs accessed: 19376 537: stonewalling pairs accessed: 20008 429: stonewalling pairs accessed: 23265 510: stonewalling pairs accessed: 20011 820: stonewalling pairs accessed: 20270 795: stonewalling pairs accessed: 20309 358: stonewalling pairs accessed: 19695 678: stonewalling pairs accessed: 23105 615: stonewalling pairs accessed: 22942 289: stonewalling pairs accessed: 19987 223: stonewalling pairs accessed: 19301 563: stonewalling pairs accessed: 22520 427: stonewalling pairs accessed: 23081 506: stonewalling pairs accessed: 19744 817: stonewalling pairs accessed: 20044 793: stonewalling pairs accessed: 20486 362: stonewalling pairs accessed: 19308 685: stonewalling pairs accessed: 23102 613: stonewalling pairs accessed: 22695 296: stonewalling pairs accessed: 20042 217: stonewalling pairs accessed: 20290 541: stonewalling pairs accessed: 20256 426: stonewalling pairs accessed: 23435 507: stonewalling pairs accessed: 19936 824: stonewalling pairs accessed: 20264 766: stonewalling pairs accessed: 20589 364: stonewalling pairs accessed: 19590 673: stonewalling pairs accessed: 22939 580: stonewalling pairs accessed: 19841 301: stonewalling pairs accessed: 20718 536: stonewalling pairs accessed: 20892 428: stonewalling pairs accessed: 23553 822: stonewalling pairs accessed: 19593 791: stonewalling pairs accessed: 19995 366: stonewalling pairs accessed: 19828 687: stonewalling pairs accessed: 23113 582: stonewalling pairs accessed: 20853 307: stonewalling pairs accessed: 20406 567: stonewalling pairs accessed: 22473 423: stonewalling pairs accessed: 22911 830: stonewalling pairs accessed: 20263 797: stonewalling pairs accessed: 20372 352: stonewalling pairs accessed: 19890 654: stonewalling pairs accessed: 20210 617: stonewalling pairs accessed: 22975 317: stonewalling pairs accessed: 20576 572: stonewalling pairs accessed: 22992 789: stonewalling pairs accessed: 20490 29: stonewalling pairs accessed: 20075 297: stonewalling pairs accessed: 20290 123: stonewalling pairs accessed: 22606 534: stonewalling pairs accessed: 20891 417: stonewalling pairs accessed: 22813 831: stonewalling pairs accessed: 19500 790: stonewalling pairs accessed: 20772 363: stonewalling pairs accessed: 19484 679: stonewalling pairs accessed: 22282 48: stonewalling pairs accessed: 19245 156: stonewalling pairs accessed: 23559 312: stonewalling pairs accessed: 20083 208: stonewalling pairs accessed: 19956 574: stonewalling pairs accessed: 22833 441: stonewalling pairs accessed: 19596 476: stonewalling pairs accessed: 23540 753: stonewalling pairs accessed: 19993 367: stonewalling pairs accessed: 20107 576: stonewalling pairs accessed: 19865 63: stonewalling pairs accessed: 20350 303: stonewalling pairs accessed: 20687 212: stonewalling pairs accessed: 20275 512: stonewalling pairs accessed: 20124 430: stonewalling pairs accessed: 23395 792: stonewalling pairs accessed: 20229 356: stonewalling pairs accessed: 19566 55: stonewalling pairs accessed: 19716 272: stonewalling pairs accessed: 19715 531: stonewalling pairs accessed: 19641 676: stonewalling pairs accessed: 22142 53: stonewalling pairs accessed: 20042 306: stonewalling pairs accessed: 19959 513: stonewalling pairs accessed: 22167 49: stonewalling pairs accessed: 19354 300: stonewalling pairs accessed: 20483 579: stonewalling pairs accessed: 20873 3: stonewalling pairs accessed: 15177 287: stonewalling pairs accessed: 19484 218: stonewalling pairs accessed: 19628 20: stonewalling pairs accessed: 19564 295: stonewalling pairs accessed: 20585 118: stonewalling pairs accessed: 22500 526: stonewalling pairs accessed: 22860 765: stonewalling pairs accessed: 20028 59: stonewalling pairs accessed: 20101 316: stonewalling pairs accessed: 20491 788: stonewalling pairs accessed: 20261 9: stonewalling pairs accessed: 16070 309: stonewalling pairs accessed: 20182 794: stonewalling pairs accessed: 20779 261: stonewalling pairs accessed: 23124 762: stonewalling pairs accessed: 20860 275: stonewalling pairs accessed: 20241 755: stonewalling pairs accessed: 20366 279: stonewalling pairs accessed: 19818 759: stonewalling pairs accessed: 20819 756: stonewalling pairs accessed: 20857 889: stonewalling pairs accessed: 20128 521: stonewalling pairs accessed: 22543 798: stonewalling pairs accessed: 20783 360: stonewalling pairs accessed: 19924 583: stonewalling pairs accessed: 20507 293: stonewalling pairs accessed: 20194 422: stonewalling pairs accessed: 23154 761: stonewalling pairs accessed: 20467 610: stonewalling pairs accessed: 22606 18: stonewalling pairs accessed: 19778 318: stonewalling pairs accessed: 20233 115: stonewalling pairs accessed: 22752 214: stonewalling pairs accessed: 19943 573: stonewalling pairs accessed: 22647 757: stonewalling pairs accessed: 20063 853: stonewalling pairs accessed: 22901 682: stonewalling pairs accessed: 23119 313: stonewalling pairs accessed: 20236 98: stonewalling pairs accessed: 20312 517: stonewalling pairs accessed: 22856 424: stonewalling pairs accessed: 23379 787: stonewalling pairs accessed: 19973 24: stonewalling pairs accessed: 19589 277: stonewalling pairs accessed: 19832 568: stonewalling pairs accessed: 22975 290: stonewalling pairs accessed: 19848 299: stonewalling pairs accessed: 20724 686: stonewalling pairs accessed: 22696 315: stonewalling pairs accessed: 20438 108: stonewalling pairs accessed: 20178 577: stonewalling pairs accessed: 19968 158: stonewalling pairs accessed: 23241 107: stonewalling pairs accessed: 19935 529: stonewalling pairs accessed: 19906 368: stonewalling pairs accessed: 19379 581: stonewalling pairs accessed: 20323 542: stonewalling pairs accessed: 20401 210: stonewalling pairs accessed: 19689 622: stonewalling pairs accessed: 23234 152: stonewalling pairs accessed: 23517 561: stonewalling pairs accessed: 22091 270: stonewalling pairs accessed: 23201 624: stonewalling pairs accessed: 19403 213: stonewalling pairs accessed: 19624 569: stonewalling pairs accessed: 22388 826: stonewalling pairs accessed: 19823 631: stonewalling pairs accessed: 19634 522: stonewalling pairs accessed: 22618 636: stonewalling pairs accessed: 20352 532: stonewalling pairs accessed: 19917 425: stonewalling pairs accessed: 23055 674: stonewalling pairs accessed: 22678 104: stonewalling pairs accessed: 20296 543: stonewalling pairs accessed: 20881 382: stonewalling pairs accessed: 20447 100: stonewalling pairs accessed: 19758 285: stonewalling pairs accessed: 20087 646: stonewalling pairs accessed: 19805 159: stonewalling pairs accessed: 23657 310: stonewalling pairs accessed: 20570 93: stonewalling pairs accessed: 23607 886: stonewalling pairs accessed: 20736 147: stonewalling pairs accessed: 23508 374: stonewalling pairs accessed: 20465 190: stonewalling pairs accessed: 22709 308: stonewalling pairs accessed: 19956 40: stonewalling pairs accessed: 20535 575: stonewalling pairs accessed: 22974 651: stonewalling pairs accessed: 19956 539: stonewalling pairs accessed: 20792 785: stonewalling pairs accessed: 20457 648: stonewalling pairs accessed: 20126 99: stonewalling pairs accessed: 19392 248: stonewalling pairs accessed: 20065 903: stonewalling pairs accessed: 19754 856: stonewalling pairs accessed: 23207 499: stonewalling pairs accessed: 19836 885: stonewalling pairs accessed: 20468 381: stonewalling pairs accessed: 20435 383: stonewalling pairs accessed: 20451 6: stonewalling pairs accessed: 15601 267: stonewalling pairs accessed: 23586 880: stonewalling pairs accessed: 19914 278: stonewalling pairs accessed: 20264 215: stonewalling pairs accessed: 20286 684: stonewalling pairs accessed: 22821 627: stonewalling pairs accessed: 19676 221: stonewalling pairs accessed: 19776 873: stonewalling pairs accessed: 22535 264: stonewalling pairs accessed: 23293 80: stonewalling pairs accessed: 22215 250: stonewalling pairs accessed: 19981 412: stonewalling pairs accessed: 23357 288: stonewalling pairs accessed: 19732 819: stonewalling pairs accessed: 19600 652: stonewalling pairs accessed: 20209 626: stonewalling pairs accessed: 19886 47: stonewalling pairs accessed: 20496 274: stonewalling pairs accessed: 19746 500: stonewalling pairs accessed: 19938 282: stonewalling pairs accessed: 20073 862: stonewalling pairs accessed: 23383 656: stonewalling pairs accessed: 23607 625: stonewalling pairs accessed: 19677 176: stonewalling pairs accessed: 22587 612: stonewalling pairs accessed: 22469 372: stonewalling pairs accessed: 20457 87: stonewalling pairs accessed: 23601 113: stonewalling pairs accessed: 22971 514: stonewalling pairs accessed: 22216 262: stonewalling pairs accessed: 23456 754: stonewalling pairs accessed: 20444 180: stonewalling pairs accessed: 22488 249: stonewalling pairs accessed: 20375 562: stonewalling pairs accessed: 22635 717: stonewalling pairs accessed: 20674 189: stonewalling pairs accessed: 23125 523: stonewalling pairs accessed: 22683 864: stonewalling pairs accessed: 22401 355: stonewalling pairs accessed: 19404 518: stonewalling pairs accessed: 22693 478: stonewalling pairs accessed: 23308 530: stonewalling pairs accessed: 20077 52: stonewalling pairs accessed: 20333 520: stonewalling pairs accessed: 22356 900: stonewalling pairs accessed: 19641 888: stonewalling pairs accessed: 20772 370: stonewalling pairs accessed: 19581 642: stonewalling pairs accessed: 18930 260: stonewalling pairs accessed: 23263 110: stonewalling pairs accessed: 20000 850: stonewalling pairs accessed: 22405 377: stonewalling pairs accessed: 19719 380: stonewalling pairs accessed: 20258 578: stonewalling pairs accessed: 20293 406: stonewalling pairs accessed: 23258 660: stonewalling pairs accessed: 23510 105: stonewalling pairs accessed: 19427 126: stonewalling pairs accessed: 22762 909: stonewalling pairs accessed: 19895 56: stonewalling pairs accessed: 19887 319: stonewalling pairs accessed: 20585 89: stonewalling pairs accessed: 22945 409: stonewalling pairs accessed: 23501 33: stonewalling pairs accessed: 19943 179: stonewalling pairs accessed: 22435 265: stonewalling pairs accessed: 23588 85: stonewalling pairs accessed: 22723 238: stonewalling pairs accessed: 19905 479: stonewalling pairs accessed: 23546 898: stonewalling pairs accessed: 20172 718: stonewalling pairs accessed: 19963 881: stonewalling pairs accessed: 20205 675: stonewalling pairs accessed: 22935 638: stonewalling pairs accessed: 20384 191: stonewalling pairs accessed: 23135 266: stonewalling pairs accessed: 23153 92: stonewalling pairs accessed: 23142 243: stonewalling pairs accessed: 19710 470: stonewalling pairs accessed: 23104 904: stonewalling pairs accessed: 20160 893: stonewalling pairs accessed: 20451 635: stonewalling pairs accessed: 20306 155: stonewalling pairs accessed: 23448 224: stonewalling pairs accessed: 19515 910: stonewalling pairs accessed: 20177 857: stonewalling pairs accessed: 23391 375: stonewalling pairs accessed: 20108 36: stonewalling pairs accessed: 19937 144: stonewalling pairs accessed: 23373 228: stonewalling pairs accessed: 20127 566: stonewalling pairs accessed: 22999 464: stonewalling pairs accessed: 22323 902: stonewalling pairs accessed: 20160 711: stonewalling pairs accessed: 20331 30: stonewalling pairs accessed: 19844 150: stonewalling pairs accessed: 23509 276: stonewalling pairs accessed: 19522 83: stonewalling pairs accessed: 22752 233: stonewalling pairs accessed: 19627 519: stonewalling pairs accessed: 22857 407: stonewalling pairs accessed: 23601 474: stonewalling pairs accessed: 23535 911: stonewalling pairs accessed: 19556 832: stonewalling pairs accessed: 22645 705: stonewalling pairs accessed: 20027 770: stonewalling pairs accessed: 22910 887: stonewalling pairs accessed: 20250 373: stonewalling pairs accessed: 19936 650: stonewalling pairs accessed: 19610 634: stonewalling pairs accessed: 20237 43: stonewalling pairs accessed: 19978 157: stonewalling pairs accessed: 23654 302: stonewalling pairs accessed: 20274 109: stonewalling pairs accessed: 20319 241: stonewalling pairs accessed: 19720 533: stonewalling pairs accessed: 20077 410: stonewalling pairs accessed: 23601 488: stonewalling pairs accessed: 20136 906: stonewalling pairs accessed: 20180 821: stonewalling pairs accessed: 19378 708: stonewalling pairs accessed: 20066 772: stonewalling pairs accessed: 22308 869: stonewalling pairs accessed: 22789 359: stonewalling pairs accessed: 20076 671: stonewalling pairs accessed: 24194 614: stonewalling pairs accessed: 22820 50: stonewalling pairs accessed: 19498 146: stonewalling pairs accessed: 23044 284: stonewalling pairs accessed: 19877 95: stonewalling pairs accessed: 23595 237: stonewalling pairs accessed: 20207 525: stonewalling pairs accessed: 22586 435: stonewalling pairs accessed: 19867 467: stonewalling pairs accessed: 22768 896: stonewalling pairs accessed: 19513 844: stonewalling pairs accessed: 23324 716: stonewalling pairs accessed: 20301 786: stonewalling pairs accessed: 20783 861: stonewalling pairs accessed: 22904 361: stonewalling pairs accessed: 20103 647: stonewalling pairs accessed: 20206 590: stonewalling pairs accessed: 20392 32: stonewalling pairs accessed: 19536 183: stonewalling pairs accessed: 23121 283: stonewalling pairs accessed: 20231 125: stonewalling pairs accessed: 23352 253: stonewalling pairs accessed: 20196 524: stonewalling pairs accessed: 23091 413: stonewalling pairs accessed: 23623 487: stonewalling pairs accessed: 19801 899: stonewalling pairs accessed: 19353 823: stonewalling pairs accessed: 19807 710: stonewalling pairs accessed: 20266 767: stonewalling pairs accessed: 20812 871: stonewalling pairs accessed: 22364 378: stonewalling pairs accessed: 19852 666: stonewalling pairs accessed: 24205 148: stonewalling pairs accessed: 23040 257: stonewalling pairs accessed: 23070 114: stonewalling pairs accessed: 22485 538: stonewalling pairs accessed: 20588 416: stonewalling pairs accessed: 22511 704: stonewalling pairs accessed: 19820 783: stonewalling pairs accessed: 20716 891: stonewalling pairs accessed: 20771 649: stonewalling pairs accessed: 20208 185: stonewalling pairs accessed: 23119 94: stonewalling pairs accessed: 23154 714: stonewalling pairs accessed: 20677 764: stonewalling pairs accessed: 20668 863: stonewalling pairs accessed: 23572 184: stonewalling pairs accessed: 22703 96: stonewalling pairs accessed: 19544 712: stonewalling pairs accessed: 19852 890: stonewalling pairs accessed: 20478 26: stonewalling pairs accessed: 20060 178: stonewalling pairs accessed: 22270 281: stonewalling pairs accessed: 19655 88: stonewalling pairs accessed: 22571 201: stonewalling pairs accessed: 23285 404: stonewalling pairs accessed: 22878 469: stonewalling pairs accessed: 23217 713: stonewalling pairs accessed: 20175 892: stonewalling pairs accessed: 20316 665: stonewalling pairs accessed: 23598 37: stonewalling pairs accessed: 19822 145: stonewalling pairs accessed: 22773 256: stonewalling pairs accessed: 22674 102: stonewalling pairs accessed: 19934 251: stonewalling pairs accessed: 20668 411: stonewalling pairs accessed: 23099 482: stonewalling pairs accessed: 20203 825: stonewalling pairs accessed: 19520 894: stonewalling pairs accessed: 20628 698: stonewalling pairs accessed: 19973 46: stonewalling pairs accessed: 20144 160: stonewalling pairs accessed: 19171 258: stonewalling pairs accessed: 22746 106: stonewalling pairs accessed: 19756 252: stonewalling pairs accessed: 19865 414: stonewalling pairs accessed: 23240 483: stonewalling pairs accessed: 19486 837: stonewalling pairs accessed: 22777 715: stonewalling pairs accessed: 19843 866: stonewalling pairs accessed: 22398 702: stonewalling pairs accessed: 19943 44: stonewalling pairs accessed: 20537 162: stonewalling pairs accessed: 19869 90: stonewalling pairs accessed: 23173 236: stonewalling pairs accessed: 19928 481: stonewalling pairs accessed: 19851 852: stonewalling pairs accessed: 22624 664: stonewalling pairs accessed: 23420 42: stonewalling pairs accessed: 19769 169: stonewalling pairs accessed: 19708 103: stonewalling pairs accessed: 20233 239: stonewalling pairs accessed: 20219 851: stonewalling pairs accessed: 22499 668: stonewalling pairs accessed: 23546 188: stonewalling pairs accessed: 22811 91: stonewalling pairs accessed: 23595 246: stonewalling pairs accessed: 19631 883: stonewalling pairs accessed: 20183 657: stonewalling pairs accessed: 23039 166: stonewalling pairs accessed: 19537 255: stonewalling pairs accessed: 19962 865: stonewalling pairs accessed: 22223 699: stonewalling pairs accessed: 20400 192: stonewalling pairs accessed: 22281 870: stonewalling pairs accessed: 22158 655: stonewalling pairs accessed: 20208 193: stonewalling pairs accessed: 22447 877: stonewalling pairs accessed: 22252 669: stonewalling pairs accessed: 23793 204: stonewalling pairs accessed: 22894 879: stonewalling pairs accessed: 23075 683: stonewalling pairs accessed: 22522 245: stonewalling pairs accessed: 20144 855: stonewalling pairs accessed: 22925 661: stonewalling pairs accessed: 23608 222: stonewalling pairs accessed: 20269 895: stonewalling pairs accessed: 20764 57: stonewalling pairs accessed: 19974 196: stonewalling pairs accessed: 23091 874: stonewalling pairs accessed: 23035 230: stonewalling pairs accessed: 20044 816: stonewalling pairs accessed: 19637 116: stonewalling pairs accessed: 22966 200: stonewalling pairs accessed: 23360 485: stonewalling pairs accessed: 19844 827: stonewalling pairs accessed: 20253 867: stonewalling pairs accessed: 22789 670: stonewalling pairs accessed: 24017 294: stonewalling pairs accessed: 20348 839: stonewalling pairs accessed: 23054 724: stonewalling pairs accessed: 24395 371: stonewalling pairs accessed: 20057 782: stonewalling pairs accessed: 22985 25: stonewalling pairs accessed: 19908 928: stonewalling pairs accessed: 23007 491: stonewalling pairs accessed: 20333 737: stonewalling pairs accessed: 23523 940: stonewalling pairs accessed: 24258 458: stonewalling pairs accessed: 22692 943: stonewalling pairs accessed: 24211 738: stonewalling pairs accessed: 23764 740: stonewalling pairs accessed: 23324 779: stonewalling pairs accessed: 22977 907: stonewalling pairs accessed: 20043 847: stonewalling pairs accessed: 23528 875: stonewalling pairs accessed: 22455 662: stonewalling pairs accessed: 23543 205: stonewalling pairs accessed: 23196 473: stonewalling pairs accessed: 23220 778: stonewalling pairs accessed: 22694 203: stonewalling pairs accessed: 23570 846: stonewalling pairs accessed: 23051 720: stonewalling pairs accessed: 23360 641: stonewalling pairs accessed: 20193 743: stonewalling pairs accessed: 23825 165: stonewalling pairs accessed: 19169 749: stonewalling pairs accessed: 23699 174: stonewalling pairs accessed: 20045 472: stonewalling pairs accessed: 23093 771: stonewalling pairs accessed: 22176 643: stonewalling pairs accessed: 19124 175: stonewalling pairs accessed: 19475 919: stonewalling pairs accessed: 20136 734: stonewalling pairs accessed: 23962 170: stonewalling pairs accessed: 20004 64: stonewalling pairs accessed: 20249 199: stonewalling pairs accessed: 23085 776: stonewalling pairs accessed: 22363 845: stonewalling pairs accessed: 23475 941: stonewalling pairs accessed: 23841 273: stonewalling pairs accessed: 20068 465: stonewalling pairs accessed: 22476 930: stonewalling pairs accessed: 22963 119: stonewalling pairs accessed: 22796 746: stonewalling pairs accessed: 24327 420: stonewalling pairs accessed: 22689 448: stonewalling pairs accessed: 22998 931: stonewalling pairs accessed: 23255 77: stonewalling pairs accessed: 23000 742: stonewalling pairs accessed: 24261 79: stonewalling pairs accessed: 23698 206: stonewalling pairs accessed: 23425 923: stonewalling pairs accessed: 19525 195: stonewalling pairs accessed: 22955 419: stonewalling pairs accessed: 22468 463: stonewalling pairs accessed: 23154 934: stonewalling pairs accessed: 23392 722: stonewalling pairs accessed: 23499 667: stonewalling pairs accessed: 23408 72: stonewalling pairs accessed: 22959 741: stonewalling pairs accessed: 23773 62: stonewalling pairs accessed: 20195 149: stonewalling pairs accessed: 23284 925: stonewalling pairs accessed: 19887 835: stonewalling pairs accessed: 23002 733: stonewalling pairs accessed: 24395 653: stonewalling pairs accessed: 20145 736: stonewalling pairs accessed: 23405 471: stonewalling pairs accessed: 23519 726: stonewalling pairs accessed: 24393 186: stonewalling pairs accessed: 22471 415: stonewalling pairs accessed: 23595 475: stonewalling pairs accessed: 23353 922: stonewalling pairs accessed: 20143 707: stonewalling pairs accessed: 19867 868: stonewalling pairs accessed: 22591 177: stonewalling pairs accessed: 22894 197: stonewalling pairs accessed: 22769 455: stonewalling pairs accessed: 23054 836: stonewalling pairs accessed: 23125 540: stonewalling pairs accessed: 19968 477: stonewalling pairs accessed: 23076 828: stonewalling pairs accessed: 19828 492: stonewalling pairs accessed: 19819 451: stonewalling pairs accessed: 22390 773: stonewalling pairs accessed: 22539 781: stonewalling pairs accessed: 22729 838: stonewalling pairs accessed: 22895 681: stonewalling pairs accessed: 22830 453: stonewalling pairs accessed: 23054 484: stonewalling pairs accessed: 19663 745: stonewalling pairs accessed: 23824 515: stonewalling pairs accessed: 22364 731: stonewalling pairs accessed: 24384 633: stonewalling pairs accessed: 20400 153: stonewalling pairs accessed: 23188 493: stonewalling pairs accessed: 19963 927: stonewalling pairs accessed: 19968 421: stonewalling pairs accessed: 23001 608: stonewalling pairs accessed: 22243 486: stonewalling pairs accessed: 20191 728: stonewalling pairs accessed: 23309 263: stonewalling pairs accessed: 22983 721: stonewalling pairs accessed: 24409 163: stonewalling pairs accessed: 19386 202: stonewalling pairs accessed: 23352 937: stonewalling pairs accessed: 23985 774: stonewalling pairs accessed: 22604 449: stonewalling pairs accessed: 22401 701: stonewalling pairs accessed: 20393 732: stonewalling pairs accessed: 23906 609: stonewalling pairs accessed: 22357 938: stonewalling pairs accessed: 24261 751: stonewalling pairs accessed: 24322 271: stonewalling pairs accessed: 23607 593: stonewalling pairs accessed: 19708 777: stonewalling pairs accessed: 22544 78: stonewalling pairs accessed: 23228 735: stonewalling pairs accessed: 23219 39: stonewalling pairs accessed: 19928 503: stonewalling pairs accessed: 19845 605: stonewalling pairs accessed: 19874 134: stonewalling pairs accessed: 23547 921: stonewalling pairs accessed: 19989 833: stonewalling pairs accessed: 23113 124: stonewalling pairs accessed: 22874 405: stonewalling pairs accessed: 23054 916: stonewalling pairs accessed: 20143 747: stonewalling pairs accessed: 23823 897: stonewalling pairs accessed: 19720 462: stonewalling pairs accessed: 22687 70: stonewalling pairs accessed: 23319 65: stonewalling pairs accessed: 22806 232: stonewalling pairs accessed: 19459 130: stonewalling pairs accessed: 23175 841: stonewalling pairs accessed: 22840 606: stonewalling pairs accessed: 20038 76: stonewalling pairs accessed: 23696 137: stonewalling pairs accessed: 23570 854: stonewalling pairs accessed: 23392 8: stonewalling pairs accessed: 15261 443: stonewalling pairs accessed: 19243 128: stonewalling pairs accessed: 22622 926: stonewalling pairs accessed: 20143 920: stonewalling pairs accessed: 19625 780: stonewalling pairs accessed: 22561 644: stonewalling pairs accessed: 19246 565: stonewalling pairs accessed: 22217 461: stonewalling pairs accessed: 23134 181: stonewalling pairs accessed: 23031 571: stonewalling pairs accessed: 22800 235: stonewalling pairs accessed: 20221 849: stonewalling pairs accessed: 22282 901: stonewalling pairs accessed: 19990 878: stonewalling pairs accessed: 22841 354: stonewalling pairs accessed: 19891 456: stonewalling pairs accessed: 22418 136: stonewalling pairs accessed: 24128 384: stonewalling pairs accessed: 23344 596: stonewalling pairs accessed: 20285 706: stonewalling pairs accessed: 20190 304: stonewalling pairs accessed: 19620 122: stonewalling pairs accessed: 23352 939: stonewalling pairs accessed: 23519 834: stonewalling pairs accessed: 22659 842: stonewalling pairs accessed: 23388 69: stonewalling pairs accessed: 23193 131: stonewalling pairs accessed: 23535 600: stonewalling pairs accessed: 20510 602: stonewalling pairs accessed: 20510 744: stonewalling pairs accessed: 24325 663: stonewalling pairs accessed: 24041 527: stonewalling pairs accessed: 23102 392: stonewalling pairs accessed: 23241 607: stonewalling pairs accessed: 20203 142: stonewalling pairs accessed: 24333 82: stonewalling pairs accessed: 22585 254: stonewalling pairs accessed: 20666 395: stonewalling pairs accessed: 23599 398: stonewalling pairs accessed: 23595 604: stonewalling pairs accessed: 20594 11: stonewalling pairs accessed: 16105 393: stonewalling pairs accessed: 23571 632: stonewalling pairs accessed: 19892 391: stonewalling pairs accessed: 22865 139: stonewalling pairs accessed: 24300 595: stonewalling pairs accessed: 20060 760: stonewalling pairs accessed: 20138 859: stonewalling pairs accessed: 23244 45: stonewalling pairs accessed: 19665 915: stonewalling pairs accessed: 20015 599: stonewalling pairs accessed: 20445 399: stonewalling pairs accessed: 22562 259: stonewalling pairs accessed: 22995 394: stonewalling pairs accessed: 23310 41: stonewalling pairs accessed: 19647 141: stonewalling pairs accessed: 23997 386: stonewalling pairs accessed: 22948 597: stonewalling pairs accessed: 20160 390: stonewalling pairs accessed: 23557 598: stonewalling pairs accessed: 20506 194: stonewalling pairs accessed: 22637 389: stonewalling pairs accessed: 23234 628: stonewalling pairs accessed: 20305 564: stonewalling pairs accessed: 22820 242: stonewalling pairs accessed: 20148 490: stonewalling pairs accessed: 19860 286: stonewalling pairs accessed: 20264 231: stonewalling pairs accessed: 20134 396: stonewalling pairs accessed: 23005 187: stonewalling pairs accessed: 22607 343: stonewalling pairs accessed: 20540 397: stonewalling pairs accessed: 23417 117: stonewalling pairs accessed: 22249 14: stonewalling pairs accessed: 16113 466: stonewalling pairs accessed: 22529 348: stonewalling pairs accessed: 19993 101: stonewalling pairs accessed: 19814 129: stonewalling pairs accessed: 23083 408: stonewalling pairs accessed: 23205 876: stonewalling pairs accessed: 23077 320: stonewalling pairs accessed: 19715 314: stonewalling pairs accessed: 20290 74: stonewalling pairs accessed: 23697 324: stonewalling pairs accessed: 20244 328: stonewalling pairs accessed: 20070 111: stonewalling pairs accessed: 20317 872: stonewalling pairs accessed: 23026 516: stonewalling pairs accessed: 22565 495: stonewalling pairs accessed: 19331 336: stonewalling pairs accessed: 19837 338: stonewalling pairs accessed: 20506 322: stonewalling pairs accessed: 20309 121: stonewalling pairs accessed: 23119 229: stonewalling pairs accessed: 19861 337: stonewalling pairs accessed: 19980 133: stonewalling pairs accessed: 23437 311: stonewalling pairs accessed: 19871 942: stonewalling pairs accessed: 24080 2: stonewalling pairs accessed: 15903 933: stonewalling pairs accessed: 23802 447: stonewalling pairs accessed: 19944 347: stonewalling pairs accessed: 20546 127: stonewalling pairs accessed: 23414 339: stonewalling pairs accessed: 19989 207: stonewalling pairs accessed: 23589 334: stonewalling pairs accessed: 20538 86: stonewalling pairs accessed: 23128 480: stonewalling pairs accessed: 20172 342: stonewalling pairs accessed: 20219 321: stonewalling pairs accessed: 19947 335: stonewalling pairs accessed: 19680 858: stonewalling pairs accessed: 22968 51: stonewalling pairs accessed: 19750 143: stonewalling pairs accessed: 23153 120: stonewalling pairs accessed: 23353 350: stonewalling pairs accessed: 20536 138: stonewalling pairs accessed: 23954 560: stonewalling pairs accessed: 21967 331: stonewalling pairs accessed: 20539 637: stonewalling pairs accessed: 19986 344: stonewalling pairs accessed: 20259 10: stonewalling pairs accessed: 15340 860: stonewalling pairs accessed: 23582 327: stonewalling pairs accessed: 20495 211: stonewalling pairs accessed: 19808 333: stonewalling pairs accessed: 20338 73: stonewalling pairs accessed: 23211 332: stonewalling pairs accessed: 20418 345: stonewalling pairs accessed: 20547 268: stonewalling pairs accessed: 23059 468: stonewalling pairs accessed: 22917 280: stonewalling pairs accessed: 20151 329: stonewalling pairs accessed: 20257 800: stonewalling pairs accessed: 22077 346: stonewalling pairs accessed: 20220 433: stonewalling pairs accessed: 19927 805: stonewalling pairs accessed: 22558 140: stonewalling pairs accessed: 23591 810: stonewalling pairs accessed: 22698 351: stonewalling pairs accessed: 19740 182: stonewalling pairs accessed: 22547 639: stonewalling pairs accessed: 19408 340: stonewalling pairs accessed: 20247 135: stonewalling pairs accessed: 23591 758: stonewalling pairs accessed: 20481 619: stonewalling pairs accessed: 22875 84: stonewalling pairs accessed: 23383 132: stonewalling pairs accessed: 24128 802: stonewalling pairs accessed: 22557 349: stonewalling pairs accessed: 20371 234: stonewalling pairs accessed: 19862 570: stonewalling pairs accessed: 22622 168: stonewalling pairs accessed: 19449 813: stonewalling pairs accessed: 22709 812: stonewalling pairs accessed: 22508 173: stonewalling pairs accessed: 19636 809: stonewalling pairs accessed: 22895 815: stonewalling pairs accessed: 23155 54: stonewalling pairs accessed: 20336 171: stonewalling pairs accessed: 19218 61: stonewalling pairs accessed: 19963 450: stonewalling pairs accessed: 23053 97: stonewalling pairs accessed: 20002 67: stonewalling pairs accessed: 22869 672: stonewalling pairs accessed: 22197 803: stonewalling pairs accessed: 22356 630: stonewalling pairs accessed: 20374 629: stonewalling pairs accessed: 19797 804: stonewalling pairs accessed: 22495 848: stonewalling pairs accessed: 22179 814: stonewalling pairs accessed: 22820 549: stonewalling pairs accessed: 22655 603: stonewalling pairs accessed: 20285 326: stonewalling pairs accessed: 20479 13: stonewalling pairs accessed: 15598 808: stonewalling pairs accessed: 22563 843: stonewalling pairs accessed: 23500 81: stonewalling pairs accessed: 22508 219: stonewalling pairs accessed: 20090 748: stonewalling pairs accessed: 24324 556: stonewalling pairs accessed: 22491 544: stonewalling pairs accessed: 20796 559: stonewalling pairs accessed: 23047 775: stonewalling pairs accessed: 22971 198: stonewalling pairs accessed: 23013 801: stonewalling pairs accessed: 22191 68: stonewalling pairs accessed: 23042 71: stonewalling pairs accessed: 22693 557: stonewalling pairs accessed: 22719 401: stonewalling pairs accessed: 22901 558: stonewalling pairs accessed: 22735 555: stonewalling pairs accessed: 22352 151: stonewalling pairs accessed: 23082 167: stonewalling pairs accessed: 19991 840: stonewalling pairs accessed: 23472 547: stonewalling pairs accessed: 22431 240: stonewalling pairs accessed: 19537 154: stonewalling pairs accessed: 23584 545: stonewalling pairs accessed: 22164 548: stonewalling pairs accessed: 22557 60: stonewalling pairs accessed: 20343 402: stonewalling pairs accessed: 23103 15: stonewalling pairs accessed: 14818 330: stonewalling pairs accessed: 20364 454: stonewalling pairs accessed: 22694 594: stonewalling pairs accessed: 19866 585: stonewalling pairs accessed: 20175 918: stonewalling pairs accessed: 19939 884: stonewalling pairs accessed: 20771 385: stonewalling pairs accessed: 22652 677: stonewalling pairs accessed: 22413 269: stonewalling pairs accessed: 23609 739: stonewalling pairs accessed: 24109 457: stonewalling pairs accessed: 22585 66: stonewalling pairs accessed: 23337 388: stonewalling pairs accessed: 23594 400: stonewalling pairs accessed: 22743 659: stonewalling pairs accessed: 23348 452: stonewalling pairs accessed: 22800 799: stonewalling pairs accessed: 20101 369: stonewalling pairs accessed: 20471 750: stonewalling pairs accessed: 23856 932: stonewalling pairs accessed: 23694 935: stonewalling pairs accessed: 23856 811: stonewalling pairs accessed: 23149 34: stonewalling pairs accessed: 19641 807: stonewalling pairs accessed: 22231 305: stonewalling pairs accessed: 19878 323: stonewalling pairs accessed: 20107 768: stonewalling pairs accessed: 20773 727: stonewalling pairs accessed: 23100 658: stonewalling pairs accessed: 23201 75: stonewalling pairs accessed: 23246 936: stonewalling pairs accessed: 24255 914: stonewalling pairs accessed: 19930 379: stonewalling pairs accessed: 20053 38: stonewalling pairs accessed: 20395 719: stonewalling pairs accessed: 20332 586: stonewalling pairs accessed: 20512 924: stonewalling pairs accessed: 19717 588: stonewalling pairs accessed: 20468 489: stonewalling pairs accessed: 19535 882: stonewalling pairs accessed: 20726 908: stonewalling pairs accessed: 20175 723: stonewalling pairs accessed: 24232 818: stonewalling pairs accessed: 19378 494: stonewalling pairs accessed: 20345 725: stonewalling pairs accessed: 23500 220: stonewalling pairs accessed: 20161 247: stonewalling pairs accessed: 19852 709: stonewalling pairs accessed: 20333 164: stonewalling pairs accessed: 19997 161: stonewalling pairs accessed: 19406 589: stonewalling pairs accessed: 20922 376: stonewalling pairs accessed: 20468 459: stonewalling pairs accessed: 23110 546: stonewalling pairs accessed: 22232 917: stonewalling pairs accessed: 19519 554: stonewalling pairs accessed: 23053 905: stonewalling pairs accessed: 19807 387: stonewalling pairs accessed: 23121 640: stonewalling pairs accessed: 19649 769: stonewalling pairs accessed: 22611 341: stonewalling pairs accessed: 20544 806: stonewalling pairs accessed: 22886 35: stonewalling pairs accessed: 19763 913: stonewalling pairs accessed: 19707 763: stonewalling pairs accessed: 20237 403: stonewalling pairs accessed: 23455 929: stonewalling pairs accessed: 23768 460: stonewalling pairs accessed: 22879 618: stonewalling pairs accessed: 23438 645: stonewalling pairs accessed: 19513 325: stonewalling pairs accessed: 20489 730: stonewalling pairs accessed: 23951 225: stonewalling pairs accessed: 19688 226: stonewalling pairs accessed: 20044 244: stonewalling pairs accessed: 19942 616: stonewalling pairs accessed: 23263 552: stonewalling pairs accessed: 23045 550: stonewalling pairs accessed: 23035 227: stonewalling pairs accessed: 19825 553: stonewalling pairs accessed: 22762 551: stonewalling pairs accessed: 22855 912: stonewalling pairs accessed: 20140 729: stonewalling pairs accessed: 23445 592: stonewalling pairs accessed: 19611 172: stonewalling pairs accessed: 20082 601: stonewalling pairs accessed: 20168 stonewalling pairs accessed min: 14818 max: 24409 -- min data: 0.6 GiB mean data: 0.9 GiB time: 300.1s WARNING: Expected aggregate file size = 1508768768000. WARNING: Stat() of aggregate file size = 1083162848768. WARNING: Using actual aggregate bytes moved = 1083162848768. WARNING: maybe caused by deadlineForStonewalling write 2874.86 67314 340.79 45.91 45.91 16.94 342.31 0.126268 359.32 0 Max Write: 2874.86 MiB/sec (3014.51 MB/sec) Summary of all tests: Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Max(OPs) Min(OPs) Mean(OPs) StdDev Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggs(MiB) API RefNum write 2874.86 2874.86 2874.86 0.00 64127.64 64127.64 64127.64 0.00 359.31613 300.11 3009.19 0 944 16 1 0 1 1 0 0 34000 47008 47008 1032984.6 MPIIO 0 Finished : Sat Jul 11 07:51:55 2020