-----============= acceptance-small: sanityn ============----- Fri May 16 08:43:29 EDT 2025 oleg329-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/sanityn.*ex: No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 Started clients oleg329-client.virtnet: 192.168.203.129@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt) oleg329-client.virtnet: executing check_config_client /mnt/lustre oleg329-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg329-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800aa899000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800aab93800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aa899000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aab93800.idle_timeout=debug disable quota as required oleg329-server: oleg329-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131593 s, 79.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6857] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 08:43:50 (1747399430) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (4s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 08:43:54 (1747399434) -rw-r--r-- 1 root root 0 May 16 08:43 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (4s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 08:43:58 (1747399438) -rw-r--r-- 1 root root 0 May 16 08:43 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (4s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 08:44:03 (1747399443) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (4s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 08:44:07 (1747399447) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (4s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 08:44:11 (1747399451) total 1 -rwxrwxrwx 1 root root 0 May 16 08:43 f2a -rwxrwxrwx 1 root root 0 May 16 08:43 f2b total 1 -rwxrwxrwx 1 root root 0 May 16 08:43 f2a -rwxrwxrwx 1 root root 0 May 16 08:43 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00885106 s, 57.8 kB/s PASS 2e (3s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 08:44:14 (1747399454) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (2s) == sanityn test 2g: check blocks update on sync write ==== 08:44:16 (1747399456) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.306699 s, 6.8 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (4s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 08:44:21 (1747399461) PASS 3 (3s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 08:44:24 (1747399464) PASS 4 (5s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 08:44:29 (1747399469) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (3s) == sanityn test 6: remove of open file on other node ============================================================================ 08:44:32 (1747399472) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (3s) == sanityn test 7: remove of open directory on other node ======================================================================= 08:44:35 (1747399475) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (3s) == sanityn test 8: remove of open special file on other node ==================================================================== 08:44:38 (1747399478) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (2s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 08:44:40 (1747399480) PASS 9a (3s) == sanityn test 9b: append to striped sparse file ======== 08:44:43 (1747399483) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188293 s, 55.7 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0229204 s, 0.1 kB/s Data read (expecting 'foo'): foo PASS 9b (2s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 08:44:45 (1747399485) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0121217 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00388202 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0020741 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00264155 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00511166 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00466817 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00434341 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00375172 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0037878 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00594816 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00245325 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0033494 s, 0.3 kB/s PASS 10a (4s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 08:44:49 (1747399489) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.0160039 s, 192 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0345708 s, 118 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000750191 s, 4.1 MB/s PASS 10b (3s) == sanityn test 11: execution of file opened for write should return error ============================================================== 08:44:52 (1747399492) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6857 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (3s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 08:44:55 (1747399495) warning: '-runtime' deprecated, use '-t runtime' instead - link 848 (time 1747399506.86 total 10.01 last 84.69) - link 1802 (time 1747399516.86 total 20.02 last 95.34) - link 2447 (time 1747399526.87 total 30.03 last 64.43) - link 3087 (time 1747399536.88 total 40.04 last 63.96) - link 3787 (time 1747399546.89 total 50.05 last 69.92) - link 4461 (time 1747399556.90 total 60.05 last 67.36) - link 5073 (time 1747399566.91 total 70.06 last 61.13) - link 5668 (time 1747399576.91 total 80.07 last 59.47) - link 6265 (time 1747399586.92 total 90.07 last 59.69) total: 6895 link in 100.02 seconds: 68.94 ops/second using seed 2314086233 running for 100 seconds - stat 10000 (time 1747399502 ; total 5 ; last 5) - stat 20000 (time 1747399506 ; total 9 ; last 4) - stat 30000 (time 1747399511 ; total 14 ; last 5) - stat 40000 (time 1747399515 ; total 18 ; last 4) - stat 50000 (time 1747399520 ; total 23 ; last 5) - stat 60000 (time 1747399525 ; total 28 ; last 5) - stat 70000 (time 1747399530 ; total 33 ; last 5) - stat 80000 (time 1747399535 ; total 38 ; last 5) - stat 90000 (time 1747399540 ; total 43 ; last 5) - stat 100000 (time 1747399545 ; total 48 ; last 5) - stat 107816 (time 1747399556 ; total 59 ; last 11) - stat 110000 (time 1747399562 ; total 65 ; last 6) - stat 112593 (time 1747399573 ; total 76 ; last 11) - stat 116332 (time 1747399584 ; total 87 ; last 11) - stat 119810 (time 1747399595 ; total 98 ; last 11) - stat 120000 (time 1747399596 ; total 99 ; last 1) total: 120342 stats in 100 seconds: 1203.420044 stats/second - unlinked 0 (time 1747399547 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile6895) error: No such file or directory total: 6894 unlinks in 84 seconds: 82.071426 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (20585) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (20588) - No such process PASS 12 (139s) == sanityn test 13: test directory page revocation ======= 08:47:14 (1747399634) f13 f13 PASS 13 (5s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 08:47:19 (1747399639) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6857 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4279: 21966 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (3s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 08:47:22 (1747399642) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4279: 22595 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (4s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 08:47:26 (1747399646) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 23223 Terminated $DIR1/$tdir/sleep 60 PASS 14b (4s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 08:47:30 (1747399650) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 23853 Terminated $DIR1/$tdir/sleep 60 PASS 14c (3s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 08:47:34 (1747399654) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 24484 Terminated $DIR1/$tdir/sleep 60 PASS 14d (5s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 08:47:38 (1747399658) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7517184 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7517184kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (15s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 08:47:53 (1747399673) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.226455 s, 9.3 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.199679 s, 10.5 MB/s Chance of close/open is 1 in 50 Seed set to 9675 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x96d13b 000001[1] 1747399676.263749 trunc from 00000000 to 0x96d13a (0x96d13b bytes) 000002[1] 1747399676.279978 read 0x3ca6b9 thru 0x3d0ce3 (0x662b bytes) 000004[0] 1747399676.308246 read 0x005993 thru 0x00d410 (0x7a7e bytes) 000007[0] 1747399676.349051 trunc from 0x96d13b to 0x078f2e (0x8f420c bytes) 000008[0] 1747399676.389686 write 0x608ae7 thru 0x608ee3 (0x3fd bytes) 000010[0] 1747399676.419450 mapwrite 0x8f928f thru 0x9090c5 (0xfe37 bytes) 000011[1] 1747399676.612164 mapwrite 0x48b39f thru 0x496eb2 (0xbb14 bytes) 000013[0] 1747399676.731911 mapread 0x24261f thru 0x24718d (0x4b6f bytes) 000014[1] 1747399676.766530 mapwrite 0x697c77 thru 0x6a70c7 (0xf451 bytes) 000016[0] 1747399676.865932 mapwrite 0x35288b thru 0x35f478 (0xcbee bytes) 000017[1] 1747399676.973764 trunc from 0x9090c6 to 0x0ed406 (0x81bcbf bytes) 000019[1] 1747399676.999556 read 0x00c969 thru 0x00fbc7 (0x325f bytes) 000021[0] 1747399677.048227 write 0x41f078 thru 0x41f6c5 (0x64e bytes) 000022[1] 1747399677.084205 write 0x467b7f thru 0x475318 (0xd79a bytes) 000023[0] 1747399677.101393 trunc from 0x475319 to 0x62f2e3 (0x1b9fcb bytes) 000024[0] 1747399677.150987 trunc from 0x62f2e4 to 0x2ef638 (0x33fcab bytes) 000025[1] 1747399677.169283 read 0x1d29d9 thru 0x1dfb20 (0xd148 bytes) 000027[1] 1747399677.192381 trunc from 0x2ef639 to 0x39b60e (0xabfd6 bytes) 000028[1] 1747399677.243286 read 0x1e6d3d thru 0x1f2300 (0xb5c4 bytes) 000029[1] 1747399677.265952 write 0x06fa8f thru 0x07277e (0x2cf0 bytes) 000031[0] 1747399677.297718 write 0x5129ec thru 0x51899a (0x5faf bytes) 000032[0] 1747399677.319277 read 0x13fd49 thru 0x14c11c (0xc3d4 bytes) 000033[0] 1747399677.345246 write 0x0a50e5 thru 0x0ae0e3 (0x8fff bytes) 000036[0] 1747399677.440907 trunc from 0x51899b to 0x846d2e (0x32e394 bytes) 000037[1] 1747399677.466265 mapwrite 0x7f4927 thru 0x7f7d19 (0x33f3 bytes) 000039[1] 1747399677.564335 mapread 0x3320ec thru 0x33fe67 (0xdd7c bytes) 000041[1] 1747399677.626293 read 0x79b8df thru 0x79e275 (0x2997 bytes) 000042[0] 1747399677.642339 write 0x0b4b52 thru 0x0b63c9 (0x1878 bytes) 000044[0] 1747399677.675769 read 0x25ada6 thru 0x2671c7 (0xc422 bytes) 000046[1] 1747399677.697393 write 0x313a73 thru 0x3160cb (0x2659 bytes) 000049[0] 1747399677.731953 write 0x3bcd5b thru 0x3c90b4 (0xc35a bytes) 000050[0] 1747399677.761978 mapread 0x0c0a73 thru 0x0c9210 (0x879e bytes) 000053[0] 1747399677.809604 read 0x7a4655 thru 0x7aee42 (0xa7ee bytes) 000056[0] 1747399677.845054 write 0x9ad103 thru 0x9bad92 (0xdc90 bytes) 000058[0] 1747399677.869990 trunc from 0x9bad93 to 0x31c284 (0x69eb0e bytes) 000059[1] 1747399677.885940 mapread 0x151af7 thru 0x15d79a (0xbca4 bytes) 000061[1] 1747399677.981650 write 0x575b4c thru 0x57dacf (0x7f84 bytes) 000063[1] 1747399678.011232 mapread 0x241dac thru 0x241e8a (0xdf bytes) 000064[0] 1747399678.035166 mapread 0x04f029 thru 0x05a6f3 (0xb6cb bytes) 000065[1] 1747399678.132726 trunc from 0x57dad0 to 0x7f9373 (0x27b8a4 bytes) 000066[0] 1747399678.168399 read 0x7a5f47 thru 0x7b0c8c (0xad46 bytes) 000067[0] 1747399678.203936 read 0x7886fa thru 0x78a54f (0x1e56 bytes) 000069[1] 1747399678.232031 write 0x4d3a91 thru 0x4d7584 (0x3af4 bytes) 000070[1] 1747399678.276293 mapread 0x01baff thru 0x024113 (0x8615 bytes) 000073[1] 1747399678.350543 read 0x3976ae thru 0x39d674 (0x5fc7 bytes) 000074[0] 1747399678.435857 trunc from 0x7f9374 to 0x288458 (0x570f1b bytes) 000075[0] 1747399678.494212 mapread 0x04ed8e thru 0x0543fb (0x566e bytes) 000076[1] 1747399678.540479 mapwrite 0x43af1f thru 0x442327 (0x7409 bytes) 000077[1] 1747399678.670684 write 0x93f7db thru 0x94c33b (0xcb61 bytes) 000079[1] 1747399678.689587 mapwrite 0x0f2011 thru 0x0f2d65 (0xd55 bytes) 000081[1] 1747399678.739636 write 0x8648df thru 0x871621 (0xcd43 bytes) 000082[0] 1747399678.752668 mapwrite 0x03c857 thru 0x04c5cf (0xfd79 bytes) 000083[1] 1747399678.876534 mapread 0x4ee71c thru 0x4f6a82 (0x8367 bytes) 000087[0] 1747399678.949603 trunc from 0x94c33c to 0x46f9e6 (0x4dc955 bytes) 000088[1] 1747399679.001944 write 0x7afea6 thru 0x7bf8ac (0xfa07 bytes) 000090[0] 1747399679.033179 read 0x07417d thru 0x08002a (0xbeae bytes) 000092[0] 1747399679.061311 read 0x6088cc thru 0x60c34e (0x3a83 bytes) 000093[1] 1747399679.107885 mapread 0x51eb6d thru 0x524e50 (0x62e4 bytes) 000094[1] 1747399679.149067 read 0x5a770a thru 0x5b66fa (0xeff1 bytes) 000095[1] 1747399679.175768 trunc from 0x7bf8ad to 0x7d3f2d (0x14681 bytes) 000100[0] 1747399679.309170 write 0x810676 thru 0x81bfc9 (0xb954 bytes) 000105[1] 1747399679.402404 mapwrite 0x0546d0 thru 0x06129d (0xcbce bytes) 000107[0] 1747399679.561819 mapread 0x031766 thru 0x03e24b (0xcae6 bytes) 000108[0] 1747399679.632216 mapwrite 0x324f23 thru 0x331862 (0xc940 bytes) 000113[0] 1747399679.805406 mapread 0x022fe3 thru 0x02fec7 (0xcee5 bytes) 000115[1] 1747399679.846459 mapread 0x7e38b3 thru 0x7f316a (0xf8b8 bytes) 000117[0] 1747399679.898169 write 0x75ab8e thru 0x768c27 (0xe09a bytes) 000118[0] 1747399679.922178 write 0x33a335 thru 0x340043 (0x5d0f bytes) 000120[1] 1747399679.949163 mapwrite 0x117074 thru 0x123cb8 (0xcc45 bytes) 000121[1] 1747399680.083429 read 0x4bfdca thru 0x4cb3c1 (0xb5f8 bytes) 000122[1] 1747399680.145108 trunc from 0x81bfca to 0x1bbb73 (0x660456 bytes) 000125[0] 1747399680.225891 read 0x17a161 thru 0x17d0cd (0x2f6d bytes) 000126[0] 1747399680.258521 mapwrite 0x43f76c thru 0x44e295 (0xeb2a bytes) 000129[0] 1747399680.448393 mapread 0x228135 thru 0x22f476 (0x7342 bytes) 000130[0] 1747399680.449148 write 0x64cdf8 thru 0x651a12 (0x4c1b bytes) 000131[0] 1747399680.472977 read 0x0b1d04 thru 0x0b9dab (0x80a8 bytes) 000134[1] 1747399680.550319 write 0x0e6c44 thru 0x0ea70b (0x3ac8 bytes) 000136[0] 1747399680.596552 write 0x13f15a thru 0x146819 (0x76c0 bytes) 000138[1] 1747399680.617640 read 0x08c100 thru 0x0938c4 (0x77c5 bytes) 000139[1] 1747399680.635005 mapwrite 0x91c568 thru 0x91d168 (0xc01 bytes) 000145[1] 1747399680.886015 mapwrite 0x70d5b1 thru 0x713824 (0x6274 bytes) 000146[1] 1747399680.986402 trunc from 0x91d169 to 0x089f81 (0x8931e7 bytes) 000147[1] 1747399681.029008 read 0x0864b9 thru 0x089f81 (0x3ac9 bytes) 000151[0] 1747399681.070424 mapread 0x0140f7 thru 0x017678 (0x3582 bytes) 000152[1] 1747399681.100071 write 0x6730fe thru 0x67c4b9 (0x93bc bytes) 000158[0] 1747399681.169363 trunc from 0x67c4ba to 0x1e71b1 (0x495308 bytes) 000159[0] 1747399681.221969 write 0x3f2a9d thru 0x401b3d (0xf0a1 bytes) 000160[1] 1747399681.223466 write 0x49855f thru 0x4a678e (0xe230 bytes) 000162[0] 1747399681.251711 mapread 0x1fdd13 thru 0x203b31 (0x5e1f bytes) 000163[1] 1747399681.315324 mapwrite 0x08bdc8 thru 0x08cf0c (0x1145 bytes) 000165[1] 1747399681.433194 trunc from 0x4a678f to 0x8b0b0c (0x40a37e bytes) 000167[0] 1747399681.460727 trunc from 0x8b0b0d to 0x037b2f (0x878fdd bytes) 000168[1] 1747399681.502785 write 0x51c93f thru 0x5242fb (0x79bd bytes) 000169[0] 1747399681.537520 trunc from 0x5242fc to 0x29ca2b (0x2878d0 bytes) 000170[1] 1747399681.589263 mapread 0x0863af thru 0x0887ea (0x243c bytes) 000172[1] 1747399681.632112 trunc from 0x29ca2c to 0x9293ba (0x68c98f bytes) 000173[0] 1747399681.681651 trunc from 0x9293bb to 0x6165fb (0x312dbf bytes) 000174[0] 1747399681.718976 trunc from 0x6165fc to 0x454726 (0x1c1ed5 bytes) 000176[0] 1747399681.763360 write 0x0eecac thru 0x0ef784 (0xad9 bytes) 000177[1] 1747399681.789416 mapread 0x3fae7a thru 0x3fd065 (0x21ec bytes) 000180[0] 1747399681.835148 trunc from 0x454727 to 0x69d3b4 (0x248c8e bytes) 000181[1] 1747399681.876101 write 0x582d95 thru 0x58d160 (0xa3cc bytes) 000182[0] 1747399681.919829 read 0x4941cc thru 0x49f2b8 (0xb0ed bytes) 000184[1] 1747399681.969836 write 0x43747c thru 0x445dd6 (0xe95b bytes) 000186[1] 1747399682.000891 mapread 0x440c10 thru 0x44d845 (0xcc36 bytes) 000187[0] 1747399682.049518 read 0x42903a thru 0x4332f2 (0xa2b9 bytes) 000188[1] 1747399682.100401 trunc from 0x69d3b5 to 0x65896d (0x44a47 bytes) 000189[1] 1747399682.155756 mapwrite 0x872e65 thru 0x87f91c (0xcab8 bytes) 000193[0] 1747399682.427799 trunc from 0x87f91d to 0x040e78 (0x83eaa4 bytes) 000194[1] 1747399682.482669 trunc from 0x040e79 to 0x70e7e9 (0x6cd971 bytes) 000195[0] 1747399682.532910 mapwrite 0x59f78a thru 0x5ac286 (0xcafd bytes) 000197[0] 1747399682.650588 trunc from 0x70e7ea to 0x0ac1dd (0x66260c bytes) 000198[1] 1747399682.678214 trunc from 0x0ac1de to 0x4a056f (0x3f4392 bytes) 000199[0] 1747399682.734250 mapwrite 0x71029a thru 0x71f771 (0xf4d8 bytes) 000200[0] 1747399682.863605 mapread 0x23281b thru 0x240342 (0xdb28 bytes) 000201[1] 1747399682.937982 mapwrite 0x1175e1 thru 0x11b5d8 (0x3ff8 bytes) truncating to largest ever: 0x9dab6c 000202[0] 1747399683.019009 trunc from 0x71f772 to 0x9dab6b (0x2bb3fa bytes) 000204[0] 1747399683.061378 write 0x46939f thru 0x477317 (0xdf79 bytes) 000209[0] 1747399683.074413 mapwrite 0x74d155 thru 0x74d4e8 (0x394 bytes) 000210[1] 1747399683.120628 mapwrite 0x995aed thru 0x99bafa (0x600e bytes) 000212[1] 1747399683.218383 write 0x25a946 thru 0x266e37 (0xc4f2 bytes) 000213[1] 1747399683.240136 mapwrite 0x31470b thru 0x3203a5 (0xbc9b bytes) 000215[0] 1747399683.351568 mapwrite 0x7cae8f thru 0x7d29da (0x7b4c bytes) 000216[0] 1747399683.451539 read 0x631f41 thru 0x632378 (0x438 bytes) 000218[0] 1747399683.525226 mapread 0x9780e2 thru 0x97e17a (0x6099 bytes) 000219[1] 1747399683.536500 write 0x2f8795 thru 0x2fee25 (0x6691 bytes) 000220[0] 1747399683.568333 mapread 0x46d215 thru 0x474b62 (0x794e bytes) 000222[1] 1747399683.608793 write 0x361c4a thru 0x36b664 (0x9a1b bytes) 000223[1] 1747399683.637772 trunc from 0x9dab6c to 0x7f5bc5 (0x1e4fa6 bytes) 000224[0] 1747399683.672185 write 0x1df4c2 thru 0x1e9ea2 (0xa9e1 bytes) 000225[0] 1747399683.698020 mapread 0x4917e9 thru 0x494075 (0x288d bytes) 000227[0] 1747399683.735317 read 0x692f79 thru 0x6946e5 (0x176d bytes) 000228[1] 1747399683.757123 trunc from 0x7f5bc6 to 0x72077a (0xd544b bytes) 000234[0] 1747399683.920250 write 0x2789b2 thru 0x27afd1 (0x2620 bytes) 000235[1] 1747399683.968703 read 0x084f75 thru 0x08f484 (0xa510 bytes) 000239[0] 1747399684.090635 mapwrite 0x19a1b3 thru 0x1a8709 (0xe557 bytes) 000241[1] 1747399684.270975 trunc from 0x72077b to 0x9a4cd0 (0x284556 bytes) 000243[1] 1747399684.334142 read 0x52187e thru 0x52e6fa (0xce7d bytes) 000246[1] 1747399684.389144 mapread 0x99cc56 thru 0x99f4ad (0x2858 bytes) 000248[0] 1747399684.423622 trunc from 0x9a4cd1 to 0x24cc80 (0x758050 bytes) 000249[0] 1747399684.462815 write 0x6daad2 thru 0x6e6b1a (0xc049 bytes) 000250[0] 1747399684.468161 write 0x2b0dee thru 0x2b2b0c (0x1d1f bytes) 000251[1] 1747399684.487184 mapwrite 0x0dd142 thru 0x0ec27e (0xf13d bytes) 000253[1] 1747399684.713343 read 0x4db279 thru 0x4e4424 (0x91ac bytes) 000257[0] 1747399684.771605 mapread 0x404efe thru 0x408b46 (0x3c49 bytes) 000258[1] 1747399684.808149 write 0x4a1292 thru 0x4ae285 (0xcff4 bytes) 000259[0] 1747399684.836624 mapread 0x24dbf3 thru 0x250654 (0x2a62 bytes) 000260[1] 1747399684.879729 trunc from 0x6e6b1b to 0x0130a2 (0x6d3a78 bytes) 000263[1] 1747399684.936312 trunc from 0x0130a3 to 0x56b31c (0x55827a bytes) 000264[1] 1747399684.955229 read 0x0e0db6 thru 0x0e25a3 (0x17ee bytes) 000269[1] 1747399685.053582 read 0x3f4734 thru 0x3f5c94 (0x1561 bytes) 000270[1] 1747399685.054671 mapread 0x4be84b thru 0x4ce469 (0xfc1f bytes) 000271[0] 1747399685.069283 mapread 0x095c53 thru 0x09fe60 (0xa20e bytes) 000272[0] 1747399685.129067 mapwrite 0x695e15 thru 0x6a229a (0xc486 bytes) 000273[0] 1747399685.309324 trunc from 0x6a229b to 0x367c49 (0x33a651 bytes) 000274[1] 1747399685.335333 mapread 0x316839 thru 0x31fbb0 (0x9378 bytes) 000275[1] 1747399685.413984 trunc from 0x367c4a to 0x7477e6 (0x3dfb9d bytes) 000277[0] 1747399685.446161 mapread 0x08b2f8 thru 0x090f41 (0x5c4a bytes) 000278[1] 1747399685.508551 mapwrite 0x476627 thru 0x47eee5 (0x88bf bytes) 000279[1] 1747399685.634878 read 0x1d3f19 thru 0x1e264e (0xe736 bytes) 000280[0] 1747399685.691157 read 0x35555d thru 0x35a1be (0x4c62 bytes) 000281[0] 1747399685.723046 mapread 0x236010 thru 0x2411a1 (0xb192 bytes) 000283[0] 1747399685.822900 mapwrite 0x6ee3c1 thru 0x6f6a18 (0x8658 bytes) 000284[0] 1747399685.944794 write 0x025c78 thru 0x026ef6 (0x127f bytes) 000285[0] 1747399685.960877 trunc from 0x7477e7 to 0x65c8d5 (0xeaf11 bytes) 000286[1] 1747399685.998651 mapwrite 0x3cbaef thru 0x3cbbd4 (0xe6 bytes) 000287[1] 1747399686.073817 mapwrite 0x1bc5b5 thru 0x1c8f43 (0xc98f bytes) 000291[1] 1747399686.202211 mapread 0x1d454b thru 0x1ddcc4 (0x977a bytes) 000292[1] 1747399686.246008 mapread 0x243f9e thru 0x2508aa (0xc90d bytes) 000297[0] 1747399686.331355 read 0x222421 thru 0x226c4a (0x482a bytes) 000298[1] 1747399686.345189 write 0x4d82e0 thru 0x4e52f6 (0xd017 bytes) 000301[1] 1747399686.380001 mapread 0x07805b thru 0x07c977 (0x491d bytes) 000303[1] 1747399686.392110 trunc from 0x65c8d6 to 0x9b1bd5 (0x355300 bytes) 000304[0] 1747399686.411308 mapread 0x86c0f3 thru 0x86dd77 (0x1c85 bytes) 000305[1] 1747399686.440193 mapwrite 0x18ccf8 thru 0x19ab0f (0xde18 bytes) 000307[0] 1747399686.563016 mapread 0x728302 thru 0x72a667 (0x2366 bytes) 000308[1] 1747399686.629019 write 0x16ca0f thru 0x16e6f6 (0x1ce8 bytes) 000311[0] 1747399686.688664 read 0x33f3a8 thru 0x341b9c (0x27f5 bytes) 000312[0] 1747399686.745681 read 0x2027d3 thru 0x20461f (0x1e4d bytes) 000313[0] 1747399686.775684 write 0x5dc260 thru 0x5e3003 (0x6da4 bytes) 000315[0] 1747399686.800518 read 0x38d233 thru 0x39ac72 (0xda40 bytes) 000316[1] 1747399686.833513 mapwrite 0x6ff60a thru 0x7083b7 (0x8dae bytes) 000317[1] 1747399686.991418 write 0x943f6c thru 0x948312 (0x43a7 bytes) 000318[0] 1747399686.997954 mapwrite 0x74c60f thru 0x757c4f (0xb641 bytes) 000319[0] 1747399687.107735 mapread 0x7f627b thru 0x7f9882 (0x3608 bytes) 000320[1] 1747399687.136565 read 0x65672a thru 0x65e509 (0x7de0 bytes) 000321[1] 1747399687.157021 trunc from 0x9b1bd6 to 0x1eeb0b (0x7c30ca bytes) 000323[1] 1747399687.206633 mapwrite 0x3e9520 thru 0x3f64a8 (0xcf89 bytes) 000325[1] 1747399687.352964 write 0x24e8ea thru 0x24fbbc (0x12d3 bytes) 000328[1] 1747399687.391472 trunc from 0x3f64a9 to 0x9993dd (0x5a2f35 bytes) 000329[1] 1747399687.411783 mapread 0x5e13ad thru 0x5e43bf (0x3013 bytes) 000331[1] 1747399687.471826 read 0x506f0e thru 0x51414d (0xd240 bytes) 000332[1] 1747399687.495581 mapwrite 0x8603d8 thru 0x862a4f (0x2678 bytes) 000333[0] 1747399687.568895 read 0x48af09 thru 0x493206 (0x82fe bytes) 000334[0] 1747399687.601044 read 0x447699 thru 0x44b6ae (0x4016 bytes) 000335[1] 1747399687.607513 trunc from 0x9993de to 0x002473 (0x996f6a bytes) 000336[0] 1747399687.648789 mapread 0x0021ee thru 0x002473 (0x286 bytes) 000337[0] 1747399687.678190 trunc from 0x002474 to 0x019992 (0x1751f bytes) 000339[0] 1747399687.745829 write 0x8e3ba9 thru 0x8ec1ec (0x8644 bytes) 000345[1] 1747399687.796256 trunc from 0x8ec1ed to 0x5f84b2 (0x2f3d3a bytes) 000348[1] 1747399687.851182 read 0x382649 thru 0x38b602 (0x8fba bytes) 000349[1] 1747399687.889542 read 0x599cf5 thru 0x5a34ca (0x97d6 bytes) 000354[1] 1747399687.938324 mapread 0x56572c thru 0x567086 (0x195b bytes) 000359[1] 1747399688.018635 write 0x1ce5ed thru 0x1d06ca (0x20de bytes) 000360[1] 1747399688.044067 mapread 0x5e8c7f thru 0x5ef396 (0x6718 bytes) 000361[0] 1747399688.045541 write 0x35b800 thru 0x36948e (0xdc8f bytes) 000362[0] 1747399688.097880 mapread 0x02b7b3 thru 0x031f65 (0x67b3 bytes) 000363[1] 1747399688.167989 read 0x29970c thru 0x29f07a (0x596f bytes) 000365[1] 1747399688.191215 mapread 0x5a8ca0 thru 0x5afd1c (0x707d bytes) 000367[1] 1747399688.273393 mapwrite 0x80c787 thru 0x80e5f2 (0x1e6c bytes) 000369[0] 1747399688.450873 write 0x9cee41 thru 0x9d0ae7 (0x1ca7 bytes) 000371[1] 1747399688.490018 trunc from 0x9d0ae8 to 0x72be54 (0x2a4c93 bytes) 000372[1] 1747399688.525477 read 0x467c4f thru 0x471a1f (0x9dd1 bytes) 000374[1] 1747399688.575348 read 0x603823 thru 0x60ace8 (0x74c6 bytes) 000375[1] 1747399688.588279 write 0x00d5df thru 0x01b6aa (0xe0cc bytes) 000376[1] 1747399688.595733 read 0x1dab3c thru 0x1e9c2e (0xf0f3 bytes) 000377[0] 1747399688.631986 read 0x5c68f9 thru 0x5cb172 (0x487a bytes) 000378[0] 1747399688.669013 mapread 0x4958f3 thru 0x4a0671 (0xad7f bytes) 000379[1] 1747399688.791906 write 0x4a03bb thru 0x4a9eb1 (0x9af7 bytes) 000380[1] 1747399688.816167 mapwrite 0x2cde70 thru 0x2d449e (0x662f bytes) 000381[1] 1747399688.943957 read 0x1b067e thru 0x1ba402 (0x9d85 bytes) 000382[0] 1747399689.031546 trunc from 0x72be55 to 0x577128 (0x1b4d2c bytes) 000386[0] 1747399689.100981 mapwrite 0x6c4ae1 thru 0x6c705a (0x257a bytes) 000387[1] 1747399689.198250 mapread 0x0ac2b2 thru 0x0b74ab (0xb1fa bytes) 000388[0] 1747399689.255297 mapwrite 0x8d95c6 thru 0x8dcac9 (0x3504 bytes) 000390[0] 1747399689.373192 mapread 0x2c645d thru 0x2ca80a (0x43ae bytes) 000391[1] 1747399689.406793 trunc from 0x8dcaca to 0x0f8053 (0x7e4a76 bytes) 000394[0] 1747399689.445231 read 0x03cd71 thru 0x048626 (0xb8b6 bytes) 000395[1] 1747399689.470444 write 0x2b3aff thru 0x2c2a1c (0xef1e bytes) 000396[0] 1747399689.498329 trunc from 0x2c2a1d to 0x787234 (0x4c4818 bytes) 000397[1] 1747399689.535790 write 0x147265 thru 0x14e8ee (0x768a bytes) 000398[0] 1747399689.573006 write 0x2ed9dd thru 0x2fb695 (0xdcb9 bytes) 000401[0] 1747399689.637556 trunc from 0x787235 to 0x3070e5 (0x48014f bytes) 000403[0] 1747399689.685535 trunc from 0x3070e6 to 0x85c192 (0x5550ad bytes) 000404[0] 1747399689.718442 mapwrite 0x1c4dbe thru 0x1cc7e5 (0x7a28 bytes) 000406[0] 1747399689.867989 trunc from 0x85c193 to 0x95fd83 (0x103bf1 bytes) 000408[1] 1747399689.897759 mapread 0x5aebe2 thru 0x5b9552 (0xa971 bytes) 000410[0] 1747399689.972358 mapwrite 0x1030a9 thru 0x10cdd2 (0x9d2a bytes) 000412[0] 1747399690.073622 mapwrite 0x54648a thru 0x555775 (0xf2ec bytes) 000413[1] 1747399690.169452 trunc from 0x95fd84 to 0x6abf8d (0x2b3df6 bytes) 000416[0] 1747399690.250860 trunc from 0x6abf8e to 0x8c033a (0x2143ad bytes) truncating to largest ever: 0x9ff7a2 000417[0] 1747399690.280331 trunc from 0x8c033b to 0x9ff7a1 (0x13f467 bytes) 000419[1] 1747399690.306018 mapread 0x6a8c7d thru 0x6b3f80 (0xb304 bytes) 000420[1] 1747399690.394866 write 0x61cf14 thru 0x6284b6 (0xb5a3 bytes) 000422[0] 1747399690.449984 read 0x9fb447 thru 0x9fc3f4 (0xfae bytes) 000424[0] 1747399690.479659 write 0x78c80a thru 0x78d31c (0xb13 bytes) 000425[0] 1747399690.502365 mapwrite 0x879b7c thru 0x88527c (0xb701 bytes) 000427[0] 1747399690.692554 read 0x301874 thru 0x30bd45 (0xa4d2 bytes) 000428[1] 1747399690.743604 trunc from 0x9ff7a2 to 0x799ea9 (0x2658f8 bytes) 000432[1] 1747399690.864286 mapread 0x073d5d thru 0x079304 (0x55a8 bytes) 000435[0] 1747399690.920229 read 0x0e30a3 thru 0x0f0e2d (0xdd8b bytes) 000439[1] 1747399690.987599 mapread 0x5a4230 thru 0x5b0830 (0xc601 bytes) 000440[0] 1747399691.041584 mapread 0x31c3d8 thru 0x32a4ba (0xe0e3 bytes) 000441[1] 1747399691.116404 mapread 0x7887d6 thru 0x78d36b (0x4b96 bytes) 000442[0] 1747399691.139075 write 0x672d34 thru 0x677375 (0x4642 bytes) 000443[0] 1747399691.167997 read 0x0df8a6 thru 0x0ed933 (0xe08e bytes) 000446[0] 1747399691.215549 mapwrite 0x16f8a9 thru 0x1769c3 (0x711b bytes) 000447[1] 1747399691.321223 mapwrite 0x383631 thru 0x38d4db (0x9eab bytes) 000449[0] 1747399691.455931 read 0x116845 thru 0x126515 (0xfcd1 bytes) 000451[0] 1747399691.501968 write 0x7a0745 thru 0x7a6268 (0x5b24 bytes) 000453[0] 1747399691.555738 mapwrite 0x8473f1 thru 0x854598 (0xd1a8 bytes) 000458[0] 1747399691.696426 write 0x9a2e45 thru 0x9a6dd9 (0x3f95 bytes) 000465[0] 1747399691.786786 write 0x753e3a thru 0x75c35a (0x8521 bytes) 000469[0] 1747399691.829573 mapread 0x1f5f83 thru 0x1f79ab (0x1a29 bytes) 000470[1] 1747399691.840923 read 0x11b3a4 thru 0x126e73 (0xbad0 bytes) 000471[1] 1747399691.874244 trunc from 0x9a6dda to 0x1d3dfb (0x7d2fde bytes) 000472[0] 1747399691.938664 mapwrite 0x534245 thru 0x5368c1 (0x267d bytes) 000473[1] 1747399692.050371 read 0x0e3cef thru 0x0e6248 (0x255a bytes) 000474[0] 1747399692.085091 read 0x21d89c thru 0x229205 (0xb96a bytes) 000476[0] 1747399692.130157 read 0x0af484 thru 0x0b6115 (0x6c92 bytes) 000477[0] 1747399692.154646 mapwrite 0x64dc86 thru 0x6518af (0x3c2a bytes) 000479[1] 1747399692.270973 mapread 0x2be08c thru 0x2c6292 (0x8207 bytes) 000480[0] 1747399692.316616 write 0x09fa78 thru 0x0af8fb (0xfe84 bytes) 000481[0] 1747399692.345401 trunc from 0x6518b0 to 0x7922b7 (0x140a08 bytes) 000482[1] 1747399692.369017 write 0x46ee00 thru 0x47d650 (0xe851 bytes) 000484[1] 1747399692.408534 mapwrite 0x2d56f9 thru 0x2dd464 (0x7d6c bytes) 000485[1] 1747399692.506894 mapwrite 0x38d938 thru 0x39af75 (0xd63e bytes) 000486[0] 1747399692.673958 mapwrite 0x3381b4 thru 0x33edce (0x6c1b bytes) 000489[0] 1747399692.808506 write 0x45130a thru 0x4604c9 (0xf1c0 bytes) 000490[0] 1747399692.829055 write 0x0bf47a thru 0x0c4949 (0x54d0 bytes) 000491[1] 1747399692.840369 write 0x7bd662 thru 0x7cc7fb (0xf19a bytes) 000494[1] 1747399692.890221 mapwrite 0x8a7e7f thru 0x8b5fc0 (0xe142 bytes) 000495[0] 1747399693.065105 read 0x75453a thru 0x762efa (0xe9c1 bytes) 000496[1] 1747399693.112686 mapwrite 0x5f32ac thru 0x5fcf2a (0x9c7f bytes) 000498[0] 1747399693.224825 trunc from 0x8b5fc1 to 0x30451c (0x5b1aa4 bytes) 000500[1] 1747399693.287327 trunc from 0x30451d to 0x44f791 (0x14b275 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 9693 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x85d401 000002[1] 1747399693.836966 trunc from 00000000 to 0x85d400 (0x85d401 bytes) 000005[0] 1747399693.918537 mapread 0x279000 thru 0x287ed0 (0xeed1 bytes) 000006[0] 1747399693.959279 mapread 0x15a000 thru 0x1688fb (0xe8fc bytes) 000008[0] 1747399693.986200 write_OD 0x5e6000 thru 0x5e9fff (0x4000 bytes) 000010[1] 1747399694.052931 read_OD 0x08a000 thru 0x08ffff (0x6000 bytes) 000012[0] 1747399694.064168 mapwrite 0x5c2000 thru 0x5cc5fc (0xa5fd bytes) 000013[1] 1747399694.192753 mapwrite 0x2b0000 thru 0x2bfc64 (0xfc65 bytes) 000016[0] 1747399694.295210 read_OD 0x273000 thru 0x275fff (0x3000 bytes) 000018[1] 1747399694.313290 mapread 0x446000 thru 0x452f00 (0xcf01 bytes) 000020[0] 1747399694.335110 mapread 0x376000 thru 0x378bdd (0x2bde bytes) 000021[0] 1747399694.357680 write_OD 0x795000 thru 0x7a0fff (0xc000 bytes) 000024[1] 1747399694.418569 mapwrite 0x27c000 thru 0x285b1e (0x9b1f bytes) 000032[0] 1747399694.516955 mapread 0x527000 thru 0x52f860 (0x8861 bytes) 000034[1] 1747399694.540463 read_OD 0x686000 thru 0x68ffff (0xa000 bytes) 000035[0] 1747399694.546633 trunc from 0x85d401 to 0x665967 (0x1f7a99 bytes) 000036[1] 1747399694.569282 write_OD 0x768000 thru 0x776fff (0xf000 bytes) 000039[1] 1747399694.646936 mapwrite 0x80f000 thru 0x810ca2 (0x1ca3 bytes) 000040[0] 1747399694.763253 mapwrite 0x797000 thru 0x79b408 (0x4409 bytes) 000041[1] 1747399694.870521 read_OD 0x3eb000 thru 0x3eefff (0x4000 bytes) 000042[0] 1747399694.898511 mapread 0x29b000 thru 0x2a231c (0x731d bytes) 000043[0] 1747399694.913432 mapwrite 0x637000 thru 0x643690 (0xc691 bytes) 000045[1] 1747399694.993304 mapread 0x7a5000 thru 0x7b1454 (0xc455 bytes) 000047[0] 1747399695.049237 mapread 0x126000 thru 0x12935b (0x335c bytes) 000049[0] 1747399695.066953 mapread 0x39d000 thru 0x3abd5e (0xed5f bytes) 000050[1] 1747399695.101126 mapwrite 0x690000 thru 0x6918b3 (0x18b4 bytes) 000053[0] 1747399695.193217 mapread 0x454000 thru 0x454645 (0x646 bytes) 000054[1] 1747399695.225763 mapread 0x34d000 thru 0x355721 (0x8722 bytes) 000056[0] 1747399695.258624 read_OD 0x122000 thru 0x124fff (0x3000 bytes) 000058[1] 1747399695.262780 mapwrite 0x722000 thru 0x72790b (0x590c bytes) 000061[0] 1747399695.353117 mapwrite 0x62f000 thru 0x63176b (0x276c bytes) truncating to largest ever: 0x9001a3 000062[1] 1747399695.503469 trunc from 0x810ca3 to 0x9001a2 (0xef500 bytes) 000064[1] 1747399695.536782 mapwrite 0x3fe000 thru 0x40b5ef (0xd5f0 bytes) 000065[0] 1747399695.654725 mapread 0x295000 thru 0x29b0ed (0x60ee bytes) 000067[1] 1747399695.697509 mapread 0x526000 thru 0x532edd (0xcede bytes) 000072[0] 1747399695.735674 mapread 0x6e5000 thru 0x6e9cdd (0x4cde bytes) 000073[1] 1747399695.749440 write_OD 0x80c000 thru 0x81afff (0xf000 bytes) 000074[1] 1747399695.807688 mapwrite 0x777000 thru 0x783b36 (0xcb37 bytes) 000075[0] 1747399695.928362 mapwrite 0x4ca000 thru 0x4cc15f (0x2160 bytes) 000078[0] 1747399696.000676 write_OD 0x302000 thru 0x30efff (0xd000 bytes) 000079[1] 1747399696.045161 write_OD 0x079000 thru 0x087fff (0xf000 bytes) 000080[1] 1747399696.096935 mapread 0x314000 thru 0x31745d (0x345e bytes) 000081[1] 1747399696.112136 mapwrite 0x166000 thru 0x168fa4 (0x2fa5 bytes) 000082[1] 1747399696.218143 read_OD 0x4a2000 thru 0x4affff (0xe000 bytes) 000085[1] 1747399696.243025 read_OD 0x821000 thru 0x82dfff (0xd000 bytes) 000087[1] 1747399696.249482 read_OD 0x273000 thru 0x27dfff (0xb000 bytes) 000088[0] 1747399696.254947 read_OD 0x3ef000 thru 0x3f8fff (0xa000 bytes) 000090[1] 1747399696.261641 mapread 0x203000 thru 0x20f93e (0xc93f bytes) 000091[0] 1747399696.371756 trunc from 0x9001a3 to 0x605774 (0x2faa2e bytes) 000094[0] 1747399696.427891 trunc from 0x605775 to 0x26a739 (0x39b03b bytes) 000095[1] 1747399696.458037 mapread 0x10e000 thru 0x10ed4b (0xd4c bytes) 000100[1] 1747399696.486098 mapread 0x21b000 thru 0x224761 (0x9762 bytes) 000101[0] 1747399696.510385 read_OD 0x0f2000 thru 0x0fefff (0xd000 bytes) 000102[1] 1747399696.517007 write_OD 0x3e3000 thru 0x3e8fff (0x6000 bytes) 000104[0] 1747399696.577023 write_OD 0x304000 thru 0x30efff (0xb000 bytes) 000105[1] 1747399696.628006 mapread 0x037000 thru 0x03b6d4 (0x46d5 bytes) 000106[1] 1747399696.646270 read_OD 0x029000 thru 0x02efff (0x6000 bytes) 000108[1] 1747399696.654310 mapwrite 0x877000 thru 0x87a163 (0x3164 bytes) 000110[1] 1747399696.866540 mapwrite 0x34f000 thru 0x352052 (0x3053 bytes) 000113[0] 1747399696.919180 mapwrite 0x158000 thru 0x1631db (0xb1dc bytes) 000114[0] 1747399697.087119 trunc from 0x87a164 to 0x0aaca0 (0x7cf4c3 bytes) 000115[0] 1747399697.128462 read_OD 0x04a000 thru 0x04afff (0x1000 bytes) 000117[0] 1747399697.142834 trunc from 0x0aaca1 to 0x80fcd1 (0x765031 bytes) 000118[0] 1747399697.163260 mapread 0x6bb000 thru 0x6c7b27 (0xcb28 bytes) 000119[1] 1747399697.236520 mapwrite 0x805000 thru 0x80ec55 (0x9c56 bytes) 000120[1] 1747399697.336033 write_OD 0x203000 thru 0x209fff (0x7000 bytes) 000127[0] 1747399697.391389 write_OD 0x3e8000 thru 0x3f1fff (0xa000 bytes) 000129[0] 1747399697.427384 write_OD 0x96c000 thru 0x978fff (0xd000 bytes) 000130[1] 1747399697.480138 mapread 0x163000 thru 0x1704f6 (0xd4f7 bytes) 000131[1] 1747399697.547129 mapread 0x75e000 thru 0x7680d0 (0xa0d1 bytes) 000132[0] 1747399697.591259 read_OD 0x4b8000 thru 0x4bcfff (0x5000 bytes) 000133[1] 1747399697.598541 write_OD 0x873000 thru 0x881fff (0xf000 bytes) 000134[0] 1747399697.670025 trunc from 0x979000 to 0x6599c7 (0x31f638 bytes) 000135[1] 1747399697.712613 trunc from 0x6599c8 to 0x63763e (0x22389 bytes) 000136[1] 1747399697.741319 write_OD 0x52f000 thru 0x538fff (0xa000 bytes) 000139[1] 1747399697.789901 trunc from 0x63763f to 0x8c1d04 (0x28a6c6 bytes) 000140[0] 1747399697.822713 mapwrite 0x3b7000 thru 0x3ba81c (0x381d bytes) 000143[1] 1747399697.928890 mapwrite 0x935000 thru 0x93ef9b (0x9f9c bytes) 000144[0] 1747399698.046139 trunc from 0x93ef9c to 0x4434d9 (0x4fbac2 bytes) 000145[0] 1747399698.076248 write_OD 0x74e000 thru 0x74efff (0x1000 bytes) 000146[1] 1747399698.125778 write_OD 0x856000 thru 0x856fff (0x1000 bytes) 000150[0] 1747399698.193130 trunc from 0x857000 to 0x7300c0 (0x126f3f bytes) 000151[1] 1747399698.218933 mapwrite 0x7dd000 thru 0x7e92dd (0xc2de bytes) 000154[0] 1747399698.356587 read_OD 0x795000 thru 0x79afff (0x6000 bytes) 000157[0] 1747399698.380095 write_OD 0x670000 thru 0x670fff (0x1000 bytes) 000160[1] 1747399698.438834 write_OD 0x045000 thru 0x049fff (0x5000 bytes) 000161[1] 1747399698.491131 mapread 0x0c5000 thru 0x0cfafc (0xaafd bytes) 000163[1] 1747399698.532114 mapread 0x1cc000 thru 0x1d2ded (0x6dee bytes) 000165[0] 1747399698.556496 mapwrite 0x530000 thru 0x53996e (0x996f bytes) 000166[1] 1747399698.674614 trunc from 0x7e92de to 0x789d7a (0x5f563 bytes) 000167[1] 1747399698.700397 write_OD 0x40c000 thru 0x410fff (0x5000 bytes) 000169[1] 1747399698.756338 mapwrite 0x05a000 thru 0x05bb00 (0x1b01 bytes) 000170[1] 1747399698.821043 read_OD 0x5e8000 thru 0x5f5fff (0xe000 bytes) 000173[1] 1747399698.837290 mapwrite 0x049000 thru 0x057706 (0xe707 bytes) 000174[0] 1747399698.976506 read_OD 0x1c4000 thru 0x1c8fff (0x5000 bytes) 000175[1] 1747399698.993122 trunc from 0x789d7b to 0x4640e7 (0x325c93 bytes) 000177[0] 1747399699.025423 mapwrite 0x7d8000 thru 0x7dac41 (0x2c42 bytes) 000181[1] 1747399699.186249 mapwrite 0x990000 thru 0x999666 (0x9667 bytes) 000182[0] 1747399699.299126 mapwrite 0x5d4000 thru 0x5e389a (0xf89b bytes) 000183[1] 1747399699.424693 mapread 0x121000 thru 0x1271ba (0x61bb bytes) 000184[0] 1747399699.470941 read_OD 0x6a2000 thru 0x6a8fff (0x7000 bytes) 000185[0] 1747399699.476358 mapwrite 0x1a4000 thru 0x1a88c9 (0x48ca bytes) 000188[1] 1747399699.601102 write_OD 0x9ce000 thru 0x9d5fff (0x8000 bytes) 000190[1] 1747399699.651103 trunc from 0x9d6000 to 0x3e5bf2 (0x5f040d bytes) 000194[0] 1747399699.689776 write_OD 0x0ec000 thru 0x0ecfff (0x1000 bytes) 000195[0] 1747399699.727079 mapwrite 0x216000 thru 0x2247db (0xe7dc bytes) 000198[0] 1747399699.877635 mapwrite 0x70d000 thru 0x7109a1 (0x39a2 bytes) 000203[1] 1747399700.002508 mapread 0x69f000 thru 0x6a5c33 (0x6c34 bytes) 000205[0] 1747399700.032788 trunc from 0x7109a2 to 0x44f2c9 (0x2c16d8 bytes) 000207[0] 1747399700.080765 read_OD 0x447000 thru 0x44dfff (0x7000 bytes) 000209[1] 1747399700.102168 mapwrite 0x358000 thru 0x35bc44 (0x3c45 bytes) 000212[1] 1747399700.205618 mapwrite 0x8fc000 thru 0x906069 (0xa06a bytes) 000213[0] 1747399700.276683 read_OD 0x188000 thru 0x192fff (0xb000 bytes) 000215[0] 1747399700.298804 mapread 0x119000 thru 0x12032c (0x732d bytes) 000216[1] 1747399700.418764 trunc from 0x90606a to 0x6db62b (0x22aa3e bytes) 000217[1] 1747399700.450620 read_OD 0x65e000 thru 0x65efff (0x1000 bytes) 000218[0] 1747399700.467489 read_OD 0x39c000 thru 0x3aafff (0xf000 bytes) 000220[1] 1747399700.483878 mapread 0x667000 thru 0x66dd59 (0x6d5a bytes) 000222[1] 1747399700.501249 write_OD 0x25c000 thru 0x25efff (0x3000 bytes) 000224[1] 1747399700.540358 mapwrite 0x696000 thru 0x6a31fe (0xd1ff bytes) 000225[0] 1747399700.630062 trunc from 0x6db62c to 0x051668 (0x689fc3 bytes) 000226[0] 1747399700.673645 trunc from 0x051669 to 0x332636 (0x2e0fce bytes) 000227[0] 1747399700.704780 mapwrite 0x21f000 thru 0x21f95d (0x95e bytes) 000229[1] 1747399700.777401 mapread 0x036000 thru 0x03d1b7 (0x71b8 bytes) 000231[0] 1747399700.803517 mapread 0x299000 thru 0x29c83f (0x3840 bytes) 000234[0] 1747399700.813889 mapwrite 0x9fb000 thru 0x9ff032 (0x4033 bytes) 000238[1] 1747399700.971379 trunc from 0x9ff033 to 0x10228d (0x8fcda5 bytes) 000243[1] 1747399701.040562 read_OD 0x011000 thru 0x012fff (0x2000 bytes) 000244[1] 1747399701.052643 write_OD 0x00d000 thru 0x018fff (0xc000 bytes) 000247[1] 1747399701.104151 mapread 0x090000 thru 0x09fcd9 (0xfcda bytes) 000248[0] 1747399701.119318 mapread 0x0a9000 thru 0x0ae993 (0x5994 bytes) 000249[0] 1747399701.136610 write_OD 0x4eb000 thru 0x4f2fff (0x8000 bytes) 000250[0] 1747399701.191344 read_OD 0x093000 thru 0x09efff (0xc000 bytes) 000251[0] 1747399701.204942 read_OD 0x305000 thru 0x30efff (0xa000 bytes) 000252[0] 1747399701.213404 write_OD 0x282000 thru 0x28bfff (0xa000 bytes) 000253[1] 1747399701.269012 write_OD 0x700000 thru 0x708fff (0x9000 bytes) 000256[0] 1747399701.324459 read_OD 0x24b000 thru 0x253fff (0x9000 bytes) 000257[1] 1747399701.333900 trunc from 0x709000 to 0x5715b8 (0x197a47 bytes) truncating to largest ever: 0x9f4880 000258[0] 1747399701.362514 trunc from 0x5715b9 to 0x9f487f (0x4832c7 bytes) 000259[0] 1747399701.384034 write_OD 0x9b3000 thru 0x9b5fff (0x3000 bytes) 000260[1] 1747399701.441752 write_OD 0x316000 thru 0x31afff (0x5000 bytes) 000262[0] 1747399701.496969 read_OD 0x4de000 thru 0x4e9fff (0xc000 bytes) 000263[0] 1747399701.511062 mapwrite 0x5f8000 thru 0x601441 (0x9442 bytes) 000264[1] 1747399701.653940 mapwrite 0x8f3000 thru 0x8ffbfd (0xcbfe bytes) 000265[1] 1747399701.792038 write_OD 0x7b6000 thru 0x7b7fff (0x2000 bytes) 000267[0] 1747399701.843874 trunc from 0x9f4880 to 0x05cf70 (0x99790f bytes) 000270[1] 1747399701.878458 trunc from 0x05cf71 to 0x31fe83 (0x2c2f13 bytes) 000271[0] 1747399701.923941 mapwrite 0x94c000 thru 0x9567bc (0xa7bd bytes) 000273[1] 1747399702.055198 trunc from 0x9567bd to 0x2234a2 (0x73331a bytes) 000274[1] 1747399702.089068 mapread 0x1ba000 thru 0x1bb554 (0x1555 bytes) 000275[0] 1747399702.098187 mapread 0x02a000 thru 0x02ef5c (0x4f5d bytes) 000276[0] 1747399702.125118 trunc from 0x2234a3 to 0x02e3eb (0x1f50b7 bytes) 000277[1] 1747399702.156543 write_OD 0x712000 thru 0x715fff (0x4000 bytes) 000279[0] 1747399702.216245 mapread 0x619000 thru 0x627131 (0xe132 bytes) 000280[0] 1747399702.274246 trunc from 0x716000 to 0x918358 (0x202359 bytes) 000282[0] 1747399702.326545 mapwrite 0x3ed000 thru 0x3f252a (0x552b bytes) 000284[1] 1747399702.419882 mapwrite 0x595000 thru 0x5a137a (0xc37b bytes) 000285[1] 1747399702.538775 trunc from 0x918359 to 0x0b1c0d (0x86674b bytes) 000286[0] 1747399702.561188 write_OD 0x1cc000 thru 0x1d3fff (0x8000 bytes) 000287[0] 1747399702.608160 read_OD 0x1a7000 thru 0x1b5fff (0xf000 bytes) 000289[0] 1747399702.616687 mapwrite 0x7f1000 thru 0x7fca3d (0xba3e bytes) 000290[1] 1747399702.754106 read_OD 0x72a000 thru 0x738fff (0xf000 bytes) 000291[0] 1747399702.781903 mapread 0x733000 thru 0x73d0ca (0xa0cb bytes) 000293[0] 1747399702.828885 read_OD 0x0b4000 thru 0x0bffff (0xc000 bytes) 000296[0] 1747399702.835059 mapread 0x1bd000 thru 0x1c3c22 (0x6c23 bytes) 000299[0] 1747399702.885621 trunc from 0x7fca3e to 0x9beca3 (0x1c2266 bytes) 000301[0] 1747399702.933391 write_OD 0x42b000 thru 0x432fff (0x8000 bytes) 000302[1] 1747399702.987376 read_OD 0x67b000 thru 0x67ffff (0x5000 bytes) 000307[1] 1747399702.995040 mapread 0x1d2000 thru 0x1d7a6b (0x5a6c bytes) 000309[0] 1747399703.049368 trunc from 0x9beca4 to 0x388bb1 (0x6360f2 bytes) 000312[0] 1747399703.114543 mapread 0x2e9000 thru 0x2f7160 (0xe161 bytes) 000313[1] 1747399703.162273 mapwrite 0x231000 thru 0x23b50c (0xa50d bytes) 000314[1] 1747399703.252111 write_OD 0x0b1000 thru 0x0befff (0xe000 bytes) 000315[1] 1747399703.299299 read_OD 0x04f000 thru 0x059fff (0xb000 bytes) 000316[1] 1747399703.312967 read_OD 0x169000 thru 0x172fff (0xa000 bytes) 000323[1] 1747399703.320222 mapwrite 0x622000 thru 0x62ab9e (0x8b9f bytes) 000324[0] 1747399703.452974 write_OD 0x3ac000 thru 0x3adfff (0x2000 bytes) 000326[0] 1747399703.520429 write_OD 0x127000 thru 0x128fff (0x2000 bytes) 000327[1] 1747399703.559646 read_OD 0x06c000 thru 0x06dfff (0x2000 bytes) 000328[1] 1747399703.565511 read_OD 0x32e000 thru 0x33bfff (0xe000 bytes) 000330[1] 1747399703.569729 read_OD 0x3bd000 thru 0x3cbfff (0xf000 bytes) 000331[1] 1747399703.577854 mapread 0x378000 thru 0x37bf08 (0x3f09 bytes) 000332[1] 1747399703.625947 read_OD 0x5f1000 thru 0x5f7fff (0x7000 bytes) 000336[0] 1747399703.632645 read_OD 0x212000 thru 0x220fff (0xf000 bytes) 000337[1] 1747399703.637385 mapread 0x530000 thru 0x5362ec (0x62ed bytes) 000338[1] 1747399703.637796 mapwrite 0x4b6000 thru 0x4c5c70 (0xfc71 bytes) 000339[0] 1747399703.748360 mapread 0x0b0000 thru 0x0ba0d9 (0xa0da bytes) 000341[1] 1747399703.837163 trunc from 0x62ab9f to 0x83cb67 (0x211fc9 bytes) 000343[1] 1747399703.892247 mapwrite 0x9ed000 thru 0x9f48a6 (0x78a7 bytes) 000344[0] 1747399703.995518 mapread 0x7dc000 thru 0x7e541b (0x941c bytes) 000346[1] 1747399704.056401 trunc from 0x9f48a7 to 0x8273b6 (0x1cd4f0 bytes) 000348[1] 1747399704.088000 mapwrite 0x261000 thru 0x26eea6 (0xdea7 bytes) 000352[0] 1747399704.231028 mapread 0x38d000 thru 0x397324 (0xa325 bytes) 000354[0] 1747399704.282571 mapwrite 0x98e000 thru 0x99d6c2 (0xf6c3 bytes) 000357[1] 1747399704.440768 trunc from 0x99d6c3 to 0x5a6b51 (0x3f6b71 bytes) 000358[1] 1747399704.475246 mapread 0x56e000 thru 0x5770da (0x90db bytes) 000361[1] 1747399704.523655 read_OD 0x31e000 thru 0x323fff (0x6000 bytes) 000362[0] 1747399704.536205 mapread 0x011000 thru 0x01bc47 (0xac48 bytes) 000363[1] 1747399704.577489 mapread 0x127000 thru 0x1314ea (0xa4eb bytes) 000366[1] 1747399704.643887 read_OD 0x2c3000 thru 0x2c3fff (0x1000 bytes) 000367[1] 1747399704.648874 trunc from 0x5a6b52 to 0x9d1fda (0x42b489 bytes) 000369[1] 1747399704.687643 mapwrite 0x09e000 thru 0x0a7e68 (0x9e69 bytes) 000372[0] 1747399704.864470 mapwrite 0x4ea000 thru 0x4f9955 (0xf956 bytes) 000376[1] 1747399705.020100 mapwrite 0x89c000 thru 0x8a715d (0xb15e bytes) 000377[0] 1747399705.121780 mapread 0x539000 thru 0x53b7e3 (0x27e4 bytes) 000378[1] 1747399705.145792 mapwrite 0x346000 thru 0x352a35 (0xca36 bytes) 000379[1] 1747399705.239308 trunc from 0x9d1fdb to 0x92633d (0xabc9d bytes) 000380[0] 1747399705.253483 mapread 0x0f2000 thru 0x0f9c26 (0x7c27 bytes) 000382[0] 1747399705.324173 trunc from 0x92633e to 0x359f1c (0x5cc421 bytes) 000385[0] 1747399705.359433 mapread 0x265000 thru 0x273bd6 (0xebd7 bytes) 000386[1] 1747399705.378435 read_OD 0x2ae000 thru 0x2b7fff (0xa000 bytes) 000387[0] 1747399705.397161 write 0x5f2000 thru 0x5fe40c (0xc40d bytes) 000389[0] 1747399705.425723 mapwrite 0x910000 thru 0x9170af (0x70b0 bytes) 000390[0] 1747399705.506076 trunc from 0x9170b0 to 0x167a45 (0x7af66a bytes) 000391[1] 1747399705.532638 trunc from 0x167a46 to 0x05b8d1 (0x10c174 bytes) 000393[0] 1747399705.556304 read 0x04a000 thru 0x04fd16 (0x5d17 bytes) 000394[1] 1747399705.576625 read_OD 0x036000 thru 0x03ffff (0xa000 bytes) 000396[0] 1747399705.590472 write 0x2e3000 thru 0x2e9435 (0x6436 bytes) 000397[0] 1747399705.600364 write 0x513000 thru 0x51b23f (0x8240 bytes) 000402[1] 1747399705.634967 mapwrite 0x47a000 thru 0x47cc9a (0x2c9b bytes) 000404[1] 1747399705.762538 read_OD 0x25a000 thru 0x261fff (0x8000 bytes) 000406[1] 1747399705.784320 trunc from 0x51b240 to 0x8a6291 (0x38b052 bytes) 000407[0] 1747399705.812737 mapread 0x30a000 thru 0x310297 (0x6298 bytes) 000408[0] 1747399705.842435 mapwrite 0x127000 thru 0x13685c (0xf85d bytes) 000409[1] 1747399705.920024 write_OD 0x56c000 thru 0x578fff (0xd000 bytes) 000410[0] 1747399705.962128 mapwrite 0x3f8000 thru 0x4036ac (0xb6ad bytes) 000411[1] 1747399706.055438 mapwrite 0x653000 thru 0x65e263 (0xb264 bytes) 000414[1] 1747399706.186707 mapread 0x66e000 thru 0x676a90 (0x8a91 bytes) 000415[1] 1747399706.198724 mapwrite 0x39b000 thru 0x39e512 (0x3513 bytes) 000419[0] 1747399706.282024 trunc from 0x8a6292 to 0x4f8bda (0x3ad6b7 bytes) 000423[0] 1747399706.337027 mapwrite 0x251000 thru 0x25e4ea (0xd4eb bytes) 000424[0] 1747399706.398734 trunc from 0x4f8bdb to 0x04229e (0x4b693c bytes) 000426[0] 1747399706.413015 trunc from 0x04229f to 0x0d3cbd (0x91a1f bytes) 000427[1] 1747399706.436234 write_OD 0x6f3000 thru 0x6f5fff (0x3000 bytes) 000428[1] 1747399706.481513 trunc from 0x6f6000 to 0x75d510 (0x67511 bytes) 000429[0] 1747399706.520994 read 0x573000 thru 0x57b490 (0x8491 bytes) 000431[1] 1747399706.564958 mapwrite 0x5bc000 thru 0x5c792b (0xb92c bytes) 000432[0] 1747399706.654589 read 0x51d000 thru 0x5293bb (0xc3bc bytes) 000438[0] 1747399706.688481 trunc from 0x75d511 to 0x8db3bb (0x17deab bytes) 000439[0] 1747399706.719744 read 0x1f4000 thru 0x1f987f (0x5880 bytes) 000441[1] 1747399706.800683 write_OD 0x978000 thru 0x97ffff (0x8000 bytes) 000442[1] 1747399706.878221 mapwrite 0x2fd000 thru 0x30128d (0x428e bytes) 000443[0] 1747399706.990728 mapread 0x82b000 thru 0x83aae7 (0xfae8 bytes) 000446[1] 1747399707.073897 read_OD 0x8f4000 thru 0x8f6fff (0x3000 bytes) 000447[1] 1747399707.076927 mapwrite 0x357000 thru 0x365ab1 (0xeab2 bytes) 000448[0] 1747399707.175771 write 0x679000 thru 0x67be70 (0x2e71 bytes) 000449[1] 1747399707.186969 mapwrite 0x9e9000 thru 0x9ec4d8 (0x34d9 bytes) 000450[1] 1747399707.281549 mapwrite 0x7e1000 thru 0x7e9bfd (0x8bfe bytes) 000453[0] 1747399707.347224 read 0x570000 thru 0x57e336 (0xe337 bytes) 000454[0] 1747399707.396480 write 0x068000 thru 0x06f840 (0x7841 bytes) 000455[0] 1747399707.431532 read 0x2fe000 thru 0x30616c (0x816d bytes) 000457[0] 1747399707.471228 write 0x968000 thru 0x96fa58 (0x7a59 bytes) 000458[0] 1747399707.494408 trunc from 0x9ec4d9 to 0x6cd966 (0x31eb72 bytes) 000459[0] 1747399707.525876 mapwrite 0x60e000 thru 0x60e30f (0x310 bytes) 000462[1] 1747399707.609937 write_OD 0x0bc000 thru 0x0bcfff (0x1000 bytes) 000463[0] 1747399707.677456 mapread 0x652000 thru 0x659b14 (0x7b15 bytes) 000464[1] 1747399707.704129 read_OD 0x258000 thru 0x25afff (0x3000 bytes) 000467[0] 1747399707.708153 read 0x5c4000 thru 0x5d221f (0xe220 bytes) 000468[1] 1747399707.727117 mapread 0x095000 thru 0x0a4b89 (0xfb8a bytes) 000471[1] 1747399707.829916 read_OD 0x411000 thru 0x414fff (0x4000 bytes) 000476[1] 1747399707.836690 write_OD 0x34a000 thru 0x34bfff (0x2000 bytes) 000478[1] 1747399707.898206 read_OD 0x5d6000 thru 0x5e2fff (0xd000 bytes) 000479[1] 1747399707.907764 mapread 0x099000 thru 0x09a677 (0x1678 bytes) 000481[1] 1747399707.932337 trunc from 0x6cd967 to 0x93fcb8 (0x272352 bytes) 000482[1] 1747399707.957947 mapwrite 0x854000 thru 0x861eec (0xdeed bytes) 000483[0] 1747399708.082483 read 0x720000 thru 0x726fc1 (0x6fc2 bytes) 000484[1] 1747399708.122500 trunc from 0x93fcb9 to 0x5f1b52 (0x34e166 bytes) 000485[1] 1747399708.156475 mapread 0x370000 thru 0x371b8f (0x1b90 bytes) 000488[0] 1747399708.210996 mapread 0x067000 thru 0x067c44 (0xc45 bytes) 000490[1] 1747399708.235020 mapwrite 0x865000 thru 0x872b2a (0xdb2b bytes) 000492[0] 1747399708.365259 trunc from 0x872b2b to 0x2cb2ca (0x5a7860 bytes) 000493[1] 1747399708.397216 mapwrite 0x827000 thru 0x835873 (0xe874 bytes) 000495[0] 1747399708.545109 mapread 0x49d000 thru 0x4a6da0 (0x9da1 bytes) All operations completed A-OK! PASS 16a (37s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 08:48:31 (1747399711) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.272283 s, 7.7 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.16214 s, 12.9 MB/s Chance of close/open is 1 in 50 Seed set to 9712 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read truncating to largest ever: 0x3dd8b5 000004[1] 1747399712.984817 trunc from 00000000 to 0x3dd8b4 (0x3dd8b5 bytes) truncating to largest ever: 0x93db81 000005[1] 1747399713.003481 trunc from 0x3dd8b5 to 0x93db80 (0x5602cc bytes) 000009[1] 1747399713.064728 read 0x616296 thru 0x616b61 (0x8cc bytes) 000010[1] 1747399713.071094 mapread 0x0397d6 thru 0x03ae14 (0x163f bytes) 000011[0] 1747399713.078144 read 0x3171b1 thru 0x31908f (0x1edf bytes) 000012[0] 1747399713.093648 mapread 0x4ec8d4 thru 0x4edafd (0x122a bytes) 000014[1] 1747399713.142193 trunc from 0x93db81 to 0x552d5e (0x3eae22 bytes) 000015[0] 1747399713.204719 mapwrite 0x4c472e thru 0x4c4f12 (0x7e5 bytes) 000018[0] 1747399713.351789 trunc from 0x552d5f to 0x553903 (0xba5 bytes) 000019[1] 1747399713.386335 mapread 0x3c8edf thru 0x3ca841 (0x1963 bytes) 000021[0] 1747399713.435120 trunc from 0x553904 to 0x0353e1 (0x51e522 bytes) 000022[0] 1747399713.450522 mapwrite 0x4e1c30 thru 0x4e3a3c (0x1e0d bytes) 000023[1] 1747399713.512876 mapread 0x15d246 thru 0x15df51 (0xd0c bytes) 000024[1] 1747399713.542493 write 0x745c0c thru 0x7462be (0x6b3 bytes) 000025[0] 1747399713.561757 trunc from 0x7462bf to 0x7419f6 (0x48c8 bytes) 000027[1] 1747399713.615004 read 0x4564ac thru 0x457dbc (0x1911 bytes) 000031[1] 1747399713.662378 write 0x2b9677 thru 0x2b9f5e (0x8e8 bytes) 000034[1] 1747399713.698211 mapread 0x34812f thru 0x3489ab (0x87d bytes) 000036[0] 1747399713.760039 write 0x2f9ca1 thru 0x2fb593 (0x18f3 bytes) 000037[0] 1747399713.786363 write 0x1d3878 thru 0x1d4430 (0xbb9 bytes) 000038[1] 1747399713.799600 read 0x685493 thru 0x686872 (0x13e0 bytes) 000039[0] 1747399713.824661 mapwrite 0x82e627 thru 0x82f99a (0x1374 bytes) 000041[0] 1747399713.914965 write 0x973a80 thru 0x975738 (0x1cb9 bytes) 000042[0] 1747399713.918154 mapread 0x64a398 thru 0x64b6bd (0x1326 bytes) 000043[0] 1747399713.945924 write 0x52c724 thru 0x52ca47 (0x324 bytes) 000044[1] 1747399713.949632 trunc from 0x975739 to 0x843429 (0x13230f bytes) 000045[0] 1747399713.995598 write 0x6414a2 thru 0x641f6e (0xacd bytes) 000046[0] 1747399714.015724 mapwrite 0x302d11 thru 0x3041fd (0x14ed bytes) 000048[1] 1747399714.103566 mapread 0x3f0483 thru 0x3f1c82 (0x1800 bytes) 000049[0] 1747399714.144782 trunc from 0x84342a to 0x6e48b0 (0x15eb79 bytes) 000051[1] 1747399714.184526 mapread 0x5ebae5 thru 0x5ebd0e (0x22a bytes) 000052[0] 1747399714.208586 mapread 0x6d4755 thru 0x6d4c2c (0x4d8 bytes) 000053[1] 1747399714.213098 trunc from 0x6e48b1 to 0x7627d2 (0x7df22 bytes) 000054[1] 1747399714.255782 trunc from 0x7627d3 to 0x7133aa (0x4f428 bytes) 000055[0] 1747399714.283657 mapwrite 0x6691fa thru 0x66a760 (0x1567 bytes) 000056[0] 1747399714.370281 trunc from 0x7133ab to 0x1b1f35 (0x561475 bytes) 000057[1] 1747399714.392438 trunc from 0x1b1f36 to 0x6b3f68 (0x502033 bytes) 000058[1] 1747399714.435924 write 0x095d53 thru 0x09742c (0x16da bytes) 000059[1] 1747399714.462655 mapread 0x27b2d6 thru 0x27b5db (0x306 bytes) 000061[0] 1747399714.469723 read 0x2f670e thru 0x2f79db (0x12ce bytes) 000062[0] 1747399714.495395 mapwrite 0x49efaa thru 0x49f468 (0x4bf bytes) 000066[1] 1747399714.603230 read 0x120d16 thru 0x122290 (0x157b bytes) 000068[1] 1747399714.633715 mapread 0x2699cf thru 0x26a297 (0x8c9 bytes) 000069[1] 1747399714.712669 write 0x087d6f thru 0x0895ca (0x185c bytes) 000071[1] 1747399714.742255 mapread 0x29a661 thru 0x29be00 (0x17a0 bytes) 000073[1] 1747399714.751789 write 0x6292b0 thru 0x629a67 (0x7b8 bytes) 000076[0] 1747399714.781460 read 0x09e128 thru 0x09ecdd (0xbb6 bytes) 000077[0] 1747399714.820980 write 0x85b4d3 thru 0x85c3f4 (0xf22 bytes) 000078[1] 1747399714.849731 mapwrite 0x23b3a5 thru 0x23d263 (0x1ebf bytes) 000081[1] 1747399714.978199 write 0x04f44c thru 0x04ffaf (0xb64 bytes) 000083[1] 1747399715.011126 read 0x30ef5c thru 0x30fc9c (0xd41 bytes) 000085[0] 1747399715.038503 trunc from 0x85c3f5 to 0x011f6a (0x84a48a bytes) 000086[0] 1747399715.094381 mapread 0x00baee thru 0x00cf82 (0x1495 bytes) 000088[0] 1747399715.107904 write 0x821968 thru 0x822037 (0x6d0 bytes) 000090[1] 1747399715.131327 read 0x031e9d thru 0x032090 (0x1f4 bytes) 000091[1] 1747399715.165206 trunc from 0x822038 to 0x128be6 (0x6f9451 bytes) 000094[1] 1747399715.214715 write 0x32bb65 thru 0x32c55b (0x9f7 bytes) 000096[1] 1747399715.231360 write 0x93c640 thru 0x93d28e (0xc4f bytes) 000097[1] 1747399715.251368 write 0x6e76a6 thru 0x6e7c79 (0x5d4 bytes) 000098[1] 1747399715.297295 mapwrite 0x49c6bc thru 0x49d6cc (0x1011 bytes) 000106[0] 1747399715.419165 mapread 0x154329 thru 0x155327 (0xfff bytes) 000107[0] 1747399715.471150 write 0x8abb72 thru 0x8ad2d7 (0x1766 bytes) 000112[1] 1747399715.489270 mapread 0x558369 thru 0x559eb2 (0x1b4a bytes) 000115[0] 1747399715.543124 mapread 0x0c43d7 thru 0x0c56df (0x1309 bytes) 000116[1] 1747399715.551691 read 0x27cbb8 thru 0x27d2c6 (0x70f bytes) 000118[1] 1747399715.660611 write 0x6d2514 thru 0x6d404d (0x1b3a bytes) 000122[0] 1747399715.701895 trunc from 0x93d28f to 0x1da3e7 (0x762ea7 bytes) 000125[1] 1747399715.756349 write 0x7bc6ab thru 0x7bda6a (0x13c0 bytes) 000126[0] 1747399715.769050 write 0x75a085 thru 0x75a3a8 (0x324 bytes) 000127[0] 1747399715.806701 write 0x9ca965 thru 0x9cbad4 (0x1170 bytes) 000132[1] 1747399715.842524 read 0x1e52fd thru 0x1e6320 (0x1024 bytes) 000134[0] 1747399715.889718 mapread 0x8a61a6 thru 0x8a6422 (0x27d bytes) 000136[0] 1747399715.905259 mapread 0x460d5a thru 0x46290a (0x1bb1 bytes) 000137[1] 1747399715.917655 read 0x54401c thru 0x5459da (0x19bf bytes) 000139[1] 1747399715.947383 trunc from 0x9cbad5 to 0x6765fc (0x3554d8 bytes) 000140[1] 1747399715.989796 mapwrite 0x778d28 thru 0x779aa5 (0xd7e bytes) 000143[1] 1747399716.078233 read 0x673aa1 thru 0x6750b7 (0x1617 bytes) 000144[0] 1747399716.078591 mapwrite 0x87d09f thru 0x87ef5e (0x1ec0 bytes) 000148[0] 1747399716.228598 trunc from 0x87ef5f to 0x6c6c7f (0x1b82df bytes) 000150[0] 1747399716.254545 mapwrite 0x49a14f thru 0x49af8d (0xe3f bytes) 000152[1] 1747399716.376196 write 0x83b4a3 thru 0x83c5a3 (0x1101 bytes) 000153[0] 1747399716.394468 trunc from 0x83c5a4 to 0x509970 (0x332c33 bytes) 000155[1] 1747399716.420490 write 0x3db861 thru 0x3dd1b3 (0x1953 bytes) 000156[0] 1747399716.444430 mapwrite 0x5ca817 thru 0x5cb72c (0xf16 bytes) 000157[1] 1747399716.540054 write 0x19b198 thru 0x19bd6e (0xbd7 bytes) 000158[0] 1747399716.565294 mapwrite 0x3b66f6 thru 0x3b74e7 (0xdf2 bytes) 000159[1] 1747399716.646707 mapwrite 0x70aaa3 thru 0x70c65a (0x1bb8 bytes) 000161[1] 1747399716.760298 read 0x56a9ea thru 0x56b3b3 (0x9ca bytes) 000163[1] 1747399716.806803 write 0x4c13d0 thru 0x4c2a5d (0x168e bytes) 000165[0] 1747399716.847432 read 0x5a7b3d thru 0x5a945a (0x191e bytes) 000166[0] 1747399716.873392 read 0x2f1808 thru 0x2f18d4 (0xcd bytes) 000168[0] 1747399716.953907 read 0x6caa66 thru 0x6cb9e2 (0xf7d bytes) 000169[0] 1747399716.954266 trunc from 0x70c65b to 0x5f4722 (0x117f38 bytes) 000170[0] 1747399717.015487 mapread 0x075cd6 thru 0x0761d7 (0x502 bytes) 000172[0] 1747399717.055833 mapwrite 0x0406e1 thru 0x040f6c (0x88c bytes) 000173[0] 1747399717.133178 write 0x1cf692 thru 0x1d12b3 (0x1c22 bytes) 000174[1] 1747399717.150222 trunc from 0x5f4723 to 0x0c3da7 (0x53097b bytes) 000175[1] 1747399717.186424 trunc from 0x0c3da8 to 0x33c174 (0x2783cd bytes) 000176[1] 1747399717.195208 trunc from 0x33c175 to 0x063b5c (0x2d8618 bytes) 000177[0] 1747399717.212165 trunc from 0x063b5d to 0x0c5a88 (0x61f2c bytes) 000178[0] 1747399717.238976 trunc from 0x0c5a89 to 0x18f2e5 (0xc985d bytes) 000179[0] 1747399717.249558 read 0x1678ad thru 0x169775 (0x1ec9 bytes) 000182[1] 1747399717.281386 trunc from 0x18f2e6 to 0x48554f (0x2f626a bytes) 000183[1] 1747399717.326271 write 0x31011b thru 0x310c0b (0xaf1 bytes) 000184[1] 1747399717.345918 trunc from 0x485550 to 0x474ac0 (0x10a8f bytes) 000187[1] 1747399717.399633 read 0x36f09a thru 0x36ffa5 (0xf0c bytes) 000188[1] 1747399717.427329 trunc from 0x474ac1 to 0x32bf5b (0x148b65 bytes) 000191[0] 1747399717.477985 mapread 0x1a7d89 thru 0x1a860b (0x883 bytes) 000194[1] 1747399717.532529 mapread 0x087def thru 0x0894e7 (0x16f9 bytes) 000196[1] 1747399717.551794 trunc from 0x32bf5c to 0x69ada5 (0x36ee4a bytes) 000197[0] 1747399717.593596 mapread 0x2683f2 thru 0x269289 (0xe98 bytes) 000198[0] 1747399717.617147 mapwrite 0x8b6906 thru 0x8b8721 (0x1e1c bytes) 000203[0] 1747399717.719266 mapwrite 0x0c40d3 thru 0x0c4a2f (0x95d bytes) 000205[1] 1747399717.808310 trunc from 0x8b8722 to 0x1d8e7a (0x6df8a7 bytes) 000206[0] 1747399717.842833 trunc from 0x1d8e7b to 0x7ddd54 (0x604eda bytes) 000209[0] 1747399717.893343 mapwrite 0x20c116 thru 0x20d5b1 (0x149c bytes) 000210[0] 1747399717.959064 read 0x71ae37 thru 0x71cb2b (0x1cf5 bytes) 000211[0] 1747399717.973802 mapread 0x6acf14 thru 0x6aeb3c (0x1c29 bytes) 000212[1] 1747399717.990549 read 0x6b9c43 thru 0x6bba5e (0x1e1c bytes) 000213[1] 1747399718.023485 read 0x20b47e thru 0x20b884 (0x407 bytes) 000214[0] 1747399718.051141 read 0x4f61b5 thru 0x4f6e0b (0xc57 bytes) 000215[1] 1747399718.109499 mapwrite 0x519020 thru 0x5195d2 (0x5b3 bytes) 000216[0] 1747399718.217701 read 0x6630ab thru 0x665084 (0x1fda bytes) 000218[1] 1747399718.225232 trunc from 0x7ddd55 to 0x08110a (0x75cc4a bytes) 000220[0] 1747399718.281688 mapwrite 0x11b6bb thru 0x11c00b (0x951 bytes) 000221[1] 1747399718.391691 write 0x0b1b80 thru 0x0b345d (0x18de bytes) 000222[1] 1747399718.432146 trunc from 0x11c00c to 0x095e5b (0x861b0 bytes) 000223[0] 1747399718.456321 mapread 0x03c5a0 thru 0x03d1df (0xc40 bytes) 000225[0] 1747399718.503166 read 0x07c0ae thru 0x07dadd (0x1a30 bytes) 000226[1] 1747399718.532550 trunc from 0x095e5c to 0x8b3740 (0x81d8e5 bytes) 000227[1] 1747399718.581670 mapwrite 0x4ac7b4 thru 0x4ad559 (0xda6 bytes) 000229[1] 1747399718.701945 trunc from 0x8b3741 to 0x552cb4 (0x360a8c bytes) 000230[1] 1747399718.717230 read 0x1dfa31 thru 0x1e0fb5 (0x1585 bytes) 000231[1] 1747399718.759532 mapwrite 0x1590ce thru 0x15a3f1 (0x1324 bytes) 000232[0] 1747399718.813126 mapread 0x283196 thru 0x284f87 (0x1df2 bytes) 000235[1] 1747399718.903731 mapwrite 0x159d41 thru 0x15a800 (0xac0 bytes) 000238[1] 1747399718.940407 write 0x3d30cf thru 0x3d431d (0x124f bytes) 000239[1] 1747399718.941537 mapwrite 0x3efbb2 thru 0x3f1983 (0x1dd2 bytes) 000240[1] 1747399718.999706 mapwrite 0x49d2ef thru 0x49e750 (0x1462 bytes) 000241[1] 1747399719.131142 mapread 0x3a8021 thru 0x3a89d6 (0x9b6 bytes) 000244[1] 1747399719.191527 read 0x117452 thru 0x118c83 (0x1832 bytes) 000246[0] 1747399719.263214 write 0x4ef656 thru 0x4f1120 (0x1acb bytes) 000247[0] 1747399719.308224 read 0x27d0b7 thru 0x27d603 (0x54d bytes) 000250[1] 1747399719.378761 write 0x01daef thru 0x01daf7 (0x9 bytes) 000254[1] 1747399719.435694 read 0x18cc90 thru 0x18eb08 (0x1e79 bytes) 000255[1] 1747399719.460525 write 0x563454 thru 0x563b9d (0x74a bytes) 000256[0] 1747399719.475643 write 0x066ccd thru 0x067d68 (0x109c bytes) 000257[0] 1747399719.517352 mapwrite 0x31bb9a thru 0x31da79 (0x1ee0 bytes) 000259[1] 1747399719.637279 read 0x0c8a98 thru 0x0ca05a (0x15c3 bytes) 000260[1] 1747399719.663475 read 0x18545f thru 0x185f84 (0xb26 bytes) 000261[0] 1747399719.719189 trunc from 0x563b9e to 0x660e3d (0xfd2a0 bytes) 000263[0] 1747399719.765648 write 0x3a09e7 thru 0x3a1f13 (0x152d bytes) 000267[0] 1747399719.800391 mapwrite 0x6e7451 thru 0x6e7ba2 (0x752 bytes) 000268[1] 1747399719.873956 write 0x22f552 thru 0x22fe1a (0x8c9 bytes) 000269[1] 1747399719.893981 write 0x0e069c thru 0x0e0b5e (0x4c3 bytes) 000272[1] 1747399719.924679 trunc from 0x6e7ba3 to 0x8715ec (0x189a4a bytes) 000273[1] 1747399719.968625 read 0x7c5bf9 thru 0x7c6740 (0xb48 bytes) 000275[0] 1747399720.011420 trunc from 0x8715ed to 0x612d47 (0x25e8a5 bytes) truncating to largest ever: 0x98e6ad 000278[0] 1747399720.095181 trunc from 0x612d48 to 0x98e6ac (0x37b965 bytes) 000280[1] 1747399720.134020 read 0x534a6e thru 0x534ac3 (0x56 bytes) 000281[1] 1747399720.172803 read 0x4b6dfc thru 0x4b7333 (0x538 bytes) 000284[0] 1747399720.246390 trunc from 0x98e6ad to 0x1fb9fe (0x792cae bytes) 000285[1] 1747399720.298061 trunc from 0x1fb9ff to 0x29f699 (0xa3c9b bytes) 000287[1] 1747399720.332562 mapwrite 0x9f887c thru 0x9f91d7 (0x95c bytes) 000288[1] 1747399720.427444 read 0x6a52ce thru 0x6a6b09 (0x183c bytes) 000289[0] 1747399720.456027 mapread 0x80ee01 thru 0x80f883 (0xa83 bytes) 000290[0] 1747399720.477584 read 0x74321c thru 0x743b23 (0x908 bytes) 000291[1] 1747399720.549156 trunc from 0x9f91d8 to 0x34dd12 (0x6ab4c5 bytes) 000292[1] 1747399720.597261 mapread 0x1ce0e2 thru 0x1cf7a4 (0x16c3 bytes) 000294[0] 1747399720.632486 mapread 0x11813d thru 0x11a0b3 (0x1f77 bytes) 000295[1] 1747399720.667972 mapwrite 0x32ede0 thru 0x32ff53 (0x1174 bytes) 000296[0] 1747399720.733078 write 0x601869 thru 0x602005 (0x79d bytes) 000297[1] 1747399720.751589 mapread 0x5541ed thru 0x5558ad (0x16c1 bytes) 000300[1] 1747399720.811709 read 0x211657 thru 0x2130e3 (0x1a8d bytes) 000301[0] 1747399720.869190 write 0x0982d3 thru 0x0986a1 (0x3cf bytes) 000302[0] 1747399720.891706 trunc from 0x602006 to 0x5fed7a (0x328b bytes) 000304[1] 1747399720.910950 read 0x30cecc thru 0x30db16 (0xc4b bytes) 000305[0] 1747399720.958443 write 0x1bb168 thru 0x1bbb41 (0x9da bytes) 000306[1] 1747399721.012319 read 0x4c0b78 thru 0x4c1af4 (0xf7d bytes) 000307[0] 1747399721.049107 mapread 0x3c7fa3 thru 0x3c8997 (0x9f5 bytes) 000308[1] 1747399721.058721 mapread 0x58b3fe thru 0x58d0a5 (0x1ca8 bytes) 000310[0] 1747399721.093461 mapwrite 0x0a823e thru 0x0a97d7 (0x159a bytes) 000312[1] 1747399721.160368 mapwrite 0x0af66e thru 0x0af6fe (0x91 bytes) 000314[0] 1747399721.229607 read 0x1b3098 thru 0x1b3b52 (0xabb bytes) 000315[0] 1747399721.239091 read 0x2094c5 thru 0x209634 (0x170 bytes) 000317[1] 1747399721.294245 mapread 0x2fd8f1 thru 0x2fed4b (0x145b bytes) 000318[1] 1747399721.302212 write 0x96a696 thru 0x96bb14 (0x147f bytes) 000319[1] 1747399721.315006 read 0x6bfb3e thru 0x6c15db (0x1a9e bytes) 000323[0] 1747399721.372650 read 0x1b6c94 thru 0x1b7ced (0x105a bytes) 000324[1] 1747399721.443478 read 0x6eb285 thru 0x6ec4be (0x123a bytes) 000329[1] 1747399721.448902 mapwrite 0x347e1c thru 0x349cbf (0x1ea4 bytes) 000331[0] 1747399721.571856 read 0x1eb984 thru 0x1ec154 (0x7d1 bytes) 000332[1] 1747399721.618201 trunc from 0x96bb15 to 0x8caec5 (0xa0c4f bytes) 000333[0] 1747399721.662674 write 0x9cd9d2 thru 0x9ceb6a (0x1199 bytes) 000334[0] 1747399721.681070 trunc from 0x9ceb6b to 0x362bc5 (0x66bfa5 bytes) 000335[1] 1747399721.714721 mapread 0x08a54c thru 0x08a6b3 (0x168 bytes) 000336[1] 1747399721.736483 mapwrite 0x6d17d6 thru 0x6d362b (0x1e56 bytes) 000337[0] 1747399721.815415 trunc from 0x6d362c to 0x87ade9 (0x1a77be bytes) 000339[0] 1747399721.874261 trunc from 0x87adea to 0x25f99f (0x61b44a bytes) 000340[0] 1747399721.888820 mapwrite 0x8f4561 thru 0x8f6320 (0x1dc0 bytes) 000346[0] 1747399722.012778 mapwrite 0x6940ae thru 0x695109 (0x105c bytes) 000352[1] 1747399722.100160 read 0x364511 thru 0x365334 (0xe24 bytes) 000353[1] 1747399722.140148 mapread 0x791ef2 thru 0x792273 (0x382 bytes) 000355[1] 1747399722.170860 mapwrite 0x496380 thru 0x49827d (0x1efe bytes) 000356[1] 1747399722.321078 mapread 0x472f48 thru 0x474778 (0x1831 bytes) 000357[0] 1747399722.333422 trunc from 0x8f6321 to 0x65279d (0x2a3b83 bytes) 000360[0] 1747399722.404510 trunc from 0x65279e to 0x8fd09b (0x2aa8fe bytes) 000363[0] 1747399722.435357 read 0x3933b8 thru 0x39365a (0x2a3 bytes) 000365[1] 1747399722.487539 mapread 0x26eaa5 thru 0x26f68b (0xbe7 bytes) 000368[1] 1747399722.547972 trunc from 0x8fd09c to 0x309447 (0x5f3c54 bytes) 000369[0] 1747399722.586210 trunc from 0x309448 to 0x34b744 (0x422fd bytes) 000370[1] 1747399722.620530 read 0x0d64bd thru 0x0d7ec6 (0x1a0a bytes) 000372[1] 1747399722.651662 trunc from 0x34b745 to 0x1df987 (0x16bdbd bytes) 000373[0] 1747399722.678121 mapread 0x12bd54 thru 0x12d26f (0x151c bytes) 000376[0] 1747399722.703576 read 0x01920d thru 0x0199e9 (0x7dd bytes) 000377[0] 1747399722.738706 write 0x16081c thru 0x161050 (0x835 bytes) 000378[1] 1747399722.755967 mapwrite 0x6ef12e thru 0x6f0db9 (0x1c8c bytes) 000380[0] 1747399722.968921 trunc from 0x6f0dba to 0x837405 (0x14664c bytes) 000381[0] 1747399723.058468 write 0x6e2abf thru 0x6e3b06 (0x1048 bytes) 000385[1] 1747399723.190135 write 0x7c1af9 thru 0x7c1d79 (0x281 bytes) 000386[0] 1747399723.219753 trunc from 0x837406 to 0x2acca6 (0x58a75f bytes) 000387[1] 1747399723.251169 read 0x0727c3 thru 0x072aa4 (0x2e2 bytes) 000388[0] 1747399723.282977 write 0x5ae2a9 thru 0x5affdd (0x1d35 bytes) 000390[1] 1747399723.296256 mapwrite 0x8cf82b thru 0x8d0866 (0x103c bytes) 000391[1] 1747399723.486072 mapwrite 0x5ed918 thru 0x5ef3fb (0x1ae4 bytes) 000392[1] 1747399723.572957 trunc from 0x8d0867 to 0x39b574 (0x5352f2 bytes) 000394[0] 1747399723.630466 read 0x1f8cad thru 0x1fa1fe (0x1552 bytes) truncating to largest ever: 0x9b1e50 000397[1] 1747399723.682031 trunc from 0x39b575 to 0x9b1e4f (0x6168db bytes) 000398[0] 1747399723.723946 trunc from 0x9b1e50 to 0x80c76d (0x1a56e2 bytes) 000400[1] 1747399723.761502 trunc from 0x80c76e to 0x2200b7 (0x5ec6b6 bytes) 000404[1] 1747399723.817411 trunc from 0x2200b8 to 0x3a1746 (0x18168f bytes) 000405[0] 1747399723.835889 mapread 0x267a30 thru 0x2695be (0x1b8f bytes) 000408[1] 1747399723.890358 trunc from 0x3a1747 to 0x8aa9ae (0x509268 bytes) 000409[1] 1747399723.940899 trunc from 0x8aa9af to 0x108d90 (0x7a1c1e bytes) 000410[1] 1747399723.969246 trunc from 0x108d91 to 0x086a96 (0x822fa bytes) 000411[0] 1747399723.984009 mapwrite 0x7155da thru 0x71568b (0xb2 bytes) 000414[0] 1747399724.096728 mapread 0x2d977b thru 0x2d98bb (0x141 bytes) 000415[0] 1747399724.102602 write 0x077100 thru 0x07759e (0x49f bytes) 000418[1] 1747399724.152663 mapwrite 0x65b4d0 thru 0x65c2f8 (0xe29 bytes) 000419[0] 1747399724.237397 mapread 0x237d9d thru 0x239738 (0x199c bytes) 000420[0] 1747399724.262767 write 0x2247ca thru 0x22679a (0x1fd1 bytes) 000423[1] 1747399724.275686 mapread 0x0958eb thru 0x09622b (0x941 bytes) 000426[0] 1747399724.322926 mapwrite 0x8ea2df thru 0x8eae09 (0xb2b bytes) 000428[1] 1747399724.424606 trunc from 0x8eae0a to 0x1e9514 (0x7018f5 bytes) 000429[0] 1747399724.485134 trunc from 0x1e9515 to 0x62fe6f (0x44695b bytes) 000431[1] 1747399724.536156 write 0x1e8ca5 thru 0x1eaaa8 (0x1e04 bytes) 000432[0] 1747399724.559491 mapwrite 0x601abb thru 0x601c52 (0x198 bytes) 000434[0] 1747399724.663231 mapwrite 0x6ec118 thru 0x6ed867 (0x1750 bytes) 000437[1] 1747399724.791955 trunc from 0x6ed868 to 0x3e87a5 (0x3050c2 bytes) 000439[0] 1747399724.825383 write 0x26db7d thru 0x26eb45 (0xfc9 bytes) 000440[1] 1747399724.852951 mapwrite 0x07dbce thru 0x07f406 (0x1839 bytes) 000441[1] 1747399724.922363 trunc from 0x3e87a6 to 0x2ab012 (0x13d793 bytes) 000443[0] 1747399724.935086 mapwrite 0x41e860 thru 0x42018a (0x192b bytes) 000446[1] 1747399725.060055 trunc from 0x42018b to 0x6d9efe (0x2b9d74 bytes) 000449[0] 1747399725.106488 write 0x5420f2 thru 0x5437b3 (0x16c2 bytes) 000450[1] 1747399725.129178 read 0x37b239 thru 0x37c1e0 (0xfa8 bytes) 000454[1] 1747399725.214540 read 0x38ecd3 thru 0x38fd88 (0x10b6 bytes) 000457[1] 1747399725.217326 write 0x589c61 thru 0x58b55a (0x18fa bytes) 000458[0] 1747399725.241933 mapwrite 0x420236 thru 0x421b82 (0x194d bytes) 000461[0] 1747399725.381486 trunc from 0x6d9eff to 0x44b28a (0x28ec74 bytes) 000462[1] 1747399725.407589 read 0x40ebf4 thru 0x410b29 (0x1f36 bytes) 000463[0] 1747399725.431594 mapread 0x2ca8f1 thru 0x2cad5a (0x46a bytes) 000469[1] 1747399725.467182 trunc from 0x44b28b to 0x1762b3 (0x2d4fd7 bytes) 000470[1] 1747399725.498103 trunc from 0x1762b4 to 0x918c3b (0x7a2988 bytes) 000473[0] 1747399725.525549 mapwrite 0x8f8976 thru 0x8f9592 (0xc1d bytes) 000476[0] 1747399725.651235 trunc from 0x918c3c to 0x61ac64 (0x2fdfd7 bytes) 000477[0] 1747399725.673434 trunc from 0x61ac65 to 0x6b2d35 (0x980d1 bytes) 000479[1] 1747399725.698377 mapread 0x449f63 thru 0x44b388 (0x1426 bytes) truncating to largest ever: 0x9ddee2 000480[1] 1747399725.738186 trunc from 0x6b2d36 to 0x9ddee1 (0x32b1ac bytes) 000481[1] 1747399725.785648 trunc from 0x9ddee2 to 0x7c0e4b (0x21d096 bytes) 000482[1] 1747399725.802005 trunc from 0x7c0e4c to 0x89e645 (0xdd7fa bytes) 000485[1] 1747399725.854312 trunc from 0x89e646 to 0x6800fd (0x21e548 bytes) 000486[1] 1747399725.888521 read 0x148ba1 thru 0x1496a6 (0xb06 bytes) 000492[0] 1747399726.025368 write 0x77a846 thru 0x77c0a8 (0x1863 bytes) 000493[1] 1747399726.056792 write 0x2faca3 thru 0x2fbd84 (0x10e2 bytes) 000494[1] 1747399726.070754 mapwrite 0x4d2bc8 thru 0x4d3933 (0xd6c bytes) 000497[1] 1747399726.171455 mapread 0x2668d7 thru 0x267109 (0x833 bytes) All operations completed A-OK! PASS 16b (18s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 08:48:48 (1747399728) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (2s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 08:48:51 (1747399731) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.212164 s, 9.9 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.165052 s, 12.7 MB/s ldlm.namespaces.MGC192.168.203.129@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aa899000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aab93800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800aa899000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800aab93800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800aa899000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800aab93800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 7.87211 s, 13.3 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.85149 s, 56.6 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.22777 s, 85.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 3.35664 s, 31.2 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 6.08695 s, 17.2 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.81087 s, 57.9 MB/s PASS 16d (29s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 08:49:20 (1747399760) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.131143 s, 80.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0839419 s, 12.5 MB/s PASS 16e (4s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 08:49:24 (1747399764) TIMEOUT PASS 16f (23s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 08:49:47 (1747399787) TIMEOUT PASS 16g (23s) == sanityn test 16i: read after truncate file ============ 08:50:11 (1747399811) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (2s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 08:50:13 (1747399813) fail_loc=0x8000030a PASS 17 (6s) == sanityn test 18: mmap sanity check =========================================================================================== 08:50:19 (1747399819) mmap test1: basic mmap operation (PASS, 0.04912s) mmap test2: MAP_PRIVATE not write back (PASS, 0.052619s) mmap test3: concurrent mmap ops on two nodes (PASS, 13.543s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.2647s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.057726s) mmap test6: check mmap write/read content on two nodes (PASS, 0.16683s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0778s) PASS 18 (22s) == sanityn test 19: test concurrent uncached read races ========================================================================= 08:50:41 (1747399841) oleg329-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg329-client: oleg329-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (2s) == sanityn test 20: test extra readahead page left in cache ============================================================== 08:50:44 (1747399844) PASS 20 (4s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 08:50:47 (1747399847) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (4s) == sanityn test 23: others should see updated atime while another read============================================================== 08:50:51 (1747399851) now is 1747399852 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6857 new atime is 1747399913 PASS 23 (64s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 08:51:55 (1747399915) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 376 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 376 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 7.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 13.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118008 376 117632 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235665 369 235296 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 376 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118008 376 117632 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre[OST:1] filesystem_summary: 235665 369 235296 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118008 376 117632 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235665 369 235296 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre2 PASS 24a (2s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 08:51:57 (1747399917) PASS 24b (2s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 08:51:59 (1747399919) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (3s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 08:52:02 (1747399922) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 08:52:03 (1747399923) utime: good mknod times 1747399923 <= 1747399924 <= 1747399924 for /mnt/lustre/f26a utime: good mknod times 1747399923 <= 1747399924 <= 1747399924 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (4s) == sanityn test 26b: sync mtime between ost and mds ====== 08:52:07 (1747399927) PASS 26b (4s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 08:52:12 (1747399932) 0 PASS 30 (4s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 08:52:16 (1747399936) fail_loc=0x314 PASS 31a (4s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 08:52:20 (1747399940) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg329-server mds-ost sync done. sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.203.129@tcp:/lustre 7542784 14336 7520256 1% /mnt/lustre2 PASS 31b (13s) == sanityn test 31r: open-rename(replace) race =========== 08:52:33 (1747399953) -rw-r--r-- 1 root root 0 May 16 08:52 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (5s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 08:52:39 (1747399959) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 08:52:40 (1747399960) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 08:52:41 (1747399961) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 08:52:42 (1747399962) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 08:52:44 (1747399964) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg329-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa899000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800aab93800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800aa899000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8800aab93800.ost_server_uuid in IDLE state after 0 sec oleg329-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa899000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800aab93800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800aa899000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800aab93800.ost_server_uuid in FULL state after 0 sec oleg329-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa899000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800aab93800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800aa899000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8800aab93800.ost_server_uuid in IDLE state after 0 sec oleg329-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa899000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800aab93800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800aa899000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8800aab93800.ost_server_uuid in FULL state after 0 sec oleg329-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa899000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800aab93800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800aa899000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8800aab93800.ost_server_uuid in IDLE state after 0 sec oleg329-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa899000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800aab93800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800aa899000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8800aab93800.ost_server_uuid in FULL state after 0 sec PASS 34 (82s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 08:54:06 (1747400046) fail_loc=0x80000317 Race attempt 0 Wait for 15678 15817 for 60 sec... - open/close 1738 (time 1747400059.50 total 10.00 last 173.76) - open/close 3629 (time 1747400069.51 total 20.01 last 189.00) total: 4000 open/close in 22.06 seconds: 181.36 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.203.129@tcp:/lustre 7.2G 14M 7.2G 1% /mnt/lustre 192.168.203.129@tcp:/lustre 7.2G 14M 7.2G 1% /mnt/lustre2 PASS 35 (66s) == sanityn test 36: handle ESTALE/open-unlink correctly == 08:55:12 (1747400112) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg329-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.468493 s, 112 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6857 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.420126 s, 125 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6857 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7524352) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.396699 s, 132 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6857 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7520256) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.327218 s, 160 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6857 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7520256) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.326749 s, 160 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6857 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.317883 s, 165 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6857 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.315645 s, 166 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6857 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.348507 s, 150 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6857 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7524352) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.341398 s, 154 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6857 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7520256) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.373157 s, 141 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6857 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7520256) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.314156 s, 167 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6857 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7522304) after_dd(7471104) after(7522304) PASS 36 (229s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 08:59:01 (1747400341) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6857 - create 5656 (time 1747400352.29 total 10.00 last 565.58) total: 10000 create in 17.17 seconds: 582.47 ops/second PASS 37 (20s) == sanityn test 39a: test from 11063 ============================================================================================ 08:59:21 (1747400361) repeat after cancel_lru_locks PASS 39a (2s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 08:59:23 (1747400363) repeat after cancel_lru_locks PASS 39b (2s) == sanityn test 39c: check truncate mtime update ================================================================================ 08:59:25 (1747400365) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 08:59:28 (1747400368) fail_loc=0x411 fail_loc=0 PASS 39d (2s) == sanityn test 40a: pdirops: create vs others ======================================================================== 08:59:30 (1747400370) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (8s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 08:59:38 (1747400378) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (7s) == sanityn test 40c: pdirops: link and others ======================================================================== 08:59:45 (1747400385) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (8s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 08:59:53 (1747400393) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (7s) == sanityn test 40e: pdirops: rename and others ======================================================================== 09:00:00 (1747400400) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (7s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 09:00:07 (1747400407) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (4s) == sanityn test 41b: pdirops: create vs create ======================================================================== 09:00:11 (1747400411) open: File exists Conflict PASS 41b (5s) == sanityn test 41c: pdirops: create vs link ======================================================================== 09:00:16 (1747400416) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (4s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 09:00:20 (1747400420) Conflict PASS 41d (4s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 09:00:24 (1747400424) Conflict PASS 41e (5s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 09:00:29 (1747400429) Conflict PASS 41f (5s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 09:00:34 (1747400434) Conflict PASS 41g (4s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 09:00:38 (1747400438) Conflict PASS 41h (5s) == sanityn test 41i: reint_open: create vs create ======== 09:00:43 (1747400443) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (662s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 09:11:45 (1747401105) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (4s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 09:11:49 (1747401109) open: File exists Conflict PASS 42b (5s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 09:11:54 (1747401114) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (4s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 09:11:58 (1747401118) Conflict PASS 42d (4s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 09:12:03 (1747401123) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (5s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 09:12:07 (1747401127) Conflict PASS 42f (4s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 09:12:11 (1747401131) Conflict PASS 42g (4s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 09:12:15 (1747401135) Conflict PASS 42h (5s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 09:12:20 (1747401140) PASS 43a (11s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 09:12:31 (1747401151) Conflict PASS 43b (4s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 09:12:35 (1747401155) Conflict PASS 43c (4s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 09:12:39 (1747401159) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (4s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 09:12:43 (1747401163) Conflict PASS 43e (4s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 09:12:47 (1747401167) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (5s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 09:12:52 (1747401172) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (4s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 09:12:56 (1747401176) Conflict PASS 43h (4s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 09:13:00 (1747401180) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 09:13:01 (1747401181) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (35s) == sanityn test 43k: unlink vs create ==================== 09:13:36 (1747401216) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (356s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 09:19:32 (1747401572) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Fri May 16 09:19:36 EDT 2025 PASS 44a (5s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 09:19:37 (1747401577) open: File exists Conflict PASS 44b (3s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 09:19:41 (1747401581) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (4s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 09:19:44 (1747401584) Conflict PASS 44d (4s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 09:19:49 (1747401589) Conflict PASS 44e (4s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 09:19:53 (1747401593) Conflict PASS 44f (4s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 09:19:57 (1747401597) Conflict PASS 44g (4s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 09:20:01 (1747401601) Conflict PASS 44h (4s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 09:20:05 (1747401605) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 09:20:06 (1747401606) PASS 45a (20s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 09:20:26 (1747401626) Conflict PASS 45b (5s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 09:20:31 (1747401631) Conflict PASS 45c (4s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 09:20:35 (1747401635) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (4s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 09:20:39 (1747401639) Conflict PASS 45e (4s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 09:20:43 (1747401643) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (5s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 09:20:48 (1747401648) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (4s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 09:20:52 (1747401652) Conflict PASS 45h (3s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 09:20:55 (1747401655) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 09:20:56 (1747401656) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (354s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 09:26:50 (1747402010) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (5s) == sanityn test 46b: pdirops: link vs create ======================================================================== 09:26:55 (1747402015) open: File exists Conflict PASS 46b (4s) == sanityn test 46c: pdirops: link vs link ======================================================================== 09:26:59 (1747402019) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (4s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 09:27:03 (1747402023) Conflict PASS 46d (4s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 09:27:07 (1747402027) Conflict PASS 46e (5s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 09:27:12 (1747402032) Conflict PASS 46f (4s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 09:27:16 (1747402036) Conflict PASS 46g (4s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 09:27:20 (1747402040) Conflict PASS 46h (4s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 09:27:25 (1747402045) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (0s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 09:27:25 (1747402045) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 09:27:26 (1747402046) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 09:27:27 (1747402047) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 09:27:28 (1747402048) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (0s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 09:27:29 (1747402049) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (0s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 09:27:29 (1747402049) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 09:27:30 (1747402050) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 09:27:31 (1747402051) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.0044314 s, 2.3 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (6s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 09:27:37 (1747402057) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00340092 s, 46.8 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (24288) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 09:27:41 (1747402061) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00278594 s, 368 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181664 s, 57.7 MB/s 3145728 PASS 51b (14s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 09:27:55 (1747402075) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0033474 s, 306 kB/s PASS 51c (5s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 09:28:01 (1747402081) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00860587 s, 122 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (3s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 09:28:03 (1747402083) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2437 0x985 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2437 0x985 0 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 09:28:07 (1747402087) llite.lustre-ffff8800aa899000.xattr_cache=1 llite.lustre-ffff8800aab93800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504963 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:28:07.000000000 -0400 Modify: 2025-05-16 09:28:07.000000000 -0400 Change: 2025-05-16 09:28:07.000000000 -0400 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504966 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:28:12.000000000 -0400 Modify: 2025-05-16 09:28:12.000000000 -0400 Change: 2025-05-16 09:28:12.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504969 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:28:18.000000000 -0400 Modify: 2025-05-16 09:28:18.000000000 -0400 Change: 2025-05-16 09:28:18.000000000 -0400 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:28:18.000000000 -0400 Modify: 2025-05-16 09:28:18.000000000 -0400 Change: 2025-05-16 09:28:18.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504972 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:28:23.000000000 -0400 Modify: 2025-05-16 09:28:23.000000000 -0400 Change: 2025-05-16 09:28:23.000000000 -0400 Birth: - File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:28:23.000000000 -0400 Modify: 2025-05-16 09:28:23.000000000 -0400 Change: 2025-05-16 09:28:23.000000000 -0400 Birth: - PASS 54 (22s) == sanityn test 55a: rename vs unlink target dir ========= 09:28:29 (1747402109) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 09:28:36 (1747402116) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 09:28:43 (1747402123) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 09:28:55 (1747402135) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) == sanityn test 60: Verify data_version behaviour ======== 09:29:04 (1747402144) PASS 60 (2s) == sanityn test 70a: cd directory && rm directory ======== 09:29:06 (1747402146) PASS 70a (1s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 09:29:07 (1747402147) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 09:29:09 (1747402149) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00223886 s, 29.3 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (0s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 09:29:10 (1747402150) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00358306 s, 18.3 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (0s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 09:29:10 (1747402150) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 09:29:11 (1747402151) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00296577 s, 22.1 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 09:29:12 (1747402152) llite.lustre-ffff8800aa899000.xattr_cache=1 llite.lustre-ffff8800aab93800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (1s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 09:29:13 (1747402153) llite.lustre-ffff8800aa899000.xattr_cache=1 llite.lustre-ffff8800aab93800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (2s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 09:29:15 (1747402155) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00274084 s, 374 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00220103 s, 465 kB/s 2934: get lock2 2935: get lock1 2934: done 2934: get lock1 2935: done 2935: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 2935: cannot get lock2: Resource deadlock avoided 2935: exit rc=0 fcntl cmd 7 failed: Resource deadlock avoided 2934: cannot get lock1: Resource deadlock avoided 2934: done 2934: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 09:29:21 (1747402161) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0123123 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00775225 s, 135 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 3571 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (6s) == sanityn test 76: Verify MDT open_files listing ======== 09:29:27 (1747402167) opening files: ........ closing files: ........ PASS 76 (15s) == sanityn test 77a: check FIFO NRS policy =============== 09:29:42 (1747402182) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0984241 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156402 s, 67.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018881 s, 55.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246135 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270054 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258445 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0396016 s, 26.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0434419 s, 24.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0475574 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0425241 s, 24.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0490726 s, 21.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0393262 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0460233 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423651 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0470399 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0474756 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451946 s, 23.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa899000.lock_unused_count=7 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111615 s, 93.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109568 s, 95.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00961771 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00980379 s, 107 MB/s 1048576 bytes (1.0 MB) copied, 0.0111745 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00920546 s, 114 MB/s 1048576 bytes (1.0 MB) copied, 0.00956595 s, 110 MB/s 1048576 bytes (1.0 MB) copied, 0.00961665 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00969206 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110549 s, 94.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108608 s, 96.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101358 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111545 s, 94.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111241 s, 94.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010098 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125574 s, 83.5 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg329-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) == sanityn test 77b: check CRR-N NRS policy ============== 09:29:44 (1747402184) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0971727 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00931377 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010274 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184157 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199628 s, 52.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281921 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219931 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020455 s, 51.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318274 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196107 s, 53.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019666 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276722 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193616 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216598 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172378 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037807 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354432 s, 29.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa899000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00658115 s, 159 MB/s , 0.00629551 s, 167 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100096 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00646001 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102818 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00728891 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00859933 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101467 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00977226 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00988483 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00965053 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110554 s, 94.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109921 s, 95.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128941 s, 81.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119658 s, 87.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01316 s, 79.7 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg329-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0935644 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00782478 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00883769 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00628191 s, 167 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135621 s, 77.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241118 s, 43.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266088 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165749 s, 63.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267481 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138836 s, 75.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250855 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288341 s, 36.4 MB/s 1048576 bytes (1.0 MB) copied, 0.01954 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103849 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0287454 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212611 s, 49.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0264751 s, 39.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00937169 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00954015 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00802323 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0083604 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00179302 s, 585 MB/s , 0.00811531 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00831731 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00709896 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00149901 s, 700 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00147592 s, 710 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00710927 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114296 s, 91.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa899000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119538 s, 87.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00207557 s, 505 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148154 s, 70.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00189565 s, 553 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg329-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) == sanityn test 77c: check ORR NRS policy ================ 09:29:47 (1747402187) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.113455 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00764216 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176077 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220638 s, 47.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304174 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256369 s, 40.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0221462 s, 47.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0265067 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308096 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286928 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290087 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0221754 s, 47.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236119 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.024315 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01732 s, 60.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316073 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181908 s, 57.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa899000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00894394 s, 117 MB/s , 0.00816467 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00181898 s, 576 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103744 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00879545 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116182 s, 90.3 MB/s 1048576 bytes (1.0 MB) copied, 0.012629 s, 83.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00940214 s, 112 MB/s , 0.0110429 s, 95.0 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00238171 s, 440 MB/s , 0.00970435 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105322 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105256 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106757 s, 98.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153868 s, 68.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159607 s, 65.7 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg329-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.108881 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103858 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238575 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219385 s, 47.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235881 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.024472 s, 42.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0374323 s, 28.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0276248 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0417303 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0419645 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03787 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030603 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283049 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346262 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0257144 s, 40.8 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800aa899000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364608 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129334 s, 81.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106845 s, 98.1 MB/s 1048576 bytes (1.0 MB) copied, 0.00873728 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113574 s, 92.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100571 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136617 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115878 s, 90.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00966399 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00970831 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00708294 s, 148 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa899000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140041 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121787 s, 86.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110685 s, 94.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0124929 s, 83.9 MB/s , 0.0125735 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116721 s, 89.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141637 s, 74.0 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg329-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.104381 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00857032 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120932 s, 86.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152389 s, 68.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253484 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279336 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104947 s, 99.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179029 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322044 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175352 s, 59.8 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800aa899000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386662 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324472 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0362436 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0572324 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0412139 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0431599 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0580552 s, 18.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00805548 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00712932 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119153 s, 88.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00381265 s, 275 MB/s 1048576 bytes (1.0 MB) copied, 0.0134143 s, 78.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00418781 s, 250 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00994319 s, 105 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0124789 s, 84.0 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0120196 s, 87.2 MB/s , 0.0119111 s, 88.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.007398 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00317869 s, 330 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa899000.lock_unused_count=4 1048576 bytes (1.0 MB) copied, 0.00343038 s, 306 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00253527 s, 414 MB/s , 0.00302575 s, 347 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00191722 s, 547 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg329-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (4s) == sanityn test 77d: check TRR nrs policy ================ 09:29:51 (1747402191) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.112959 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131894 s, 79.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144255 s, 72.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211801 s, 49.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120489 s, 87.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0313577 s, 33.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354445 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0376586 s, 27.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0382802 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0314975 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0377611 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335419 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388562 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357218 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034125 s, 30.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa899000.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff8800aa899000.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0370426 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0417331 s, 25.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150689 s, 69.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142484 s, 73.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123099 s, 85.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0144007 s, 72.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0172724 s, 60.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0144645 s, 72.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151247 s, 69.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158154 s, 66.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00137387 s, 763 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa899000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108745 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0127769 s, 82.1 MB/s 1048576 bytes (1.0 MB) copied, 0.00146672 s, 715 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111798 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124397 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121923 s, 86.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130836 s, 80.1 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg329-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0858187 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00866963 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138575 s, 75.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020586 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0292534 s, 35.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0316937 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0230668 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0329535 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188351 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0330955 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276906 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361716 s, 29.0 MB/s , 0.0373371 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360765 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318689 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101231 s, 104 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa899000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff8800aa899000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.049922 s, 21.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa899000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00699605 s, 150 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00820156 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114374 s, 91.7 MB/s 1+0 records in 1+0 records out , 0.00940072 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00519874 s, 202 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00388195 s, 270 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00157741 s, 665 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00338666 s, 310 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100067 s, 105 MB/s 1048576 bytes (1.0 MB) copied, 0.00748696 s, 140 MB/s 1048576 bytes (1.0 MB) copied, 0.011241 s, 93.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011202 s, 93.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129905 s, 80.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0116999 s, 89.6 MB/s , 0.0114123 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161732 s, 64.8 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg329-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0958102 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00646492 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00999192 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174467 s, 60.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0257921 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0329557 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285402 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233401 s, 44.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390651 s, 26.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0298372 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0303054 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341031 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0264279 s, 39.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa899000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308192 s, 34.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0188913 s, 55.5 MB/s , 0.031134 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020674 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00722488 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00640809 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00932078 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00900739 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101611 s, 103 MB/s , 0.00916209 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00888204 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00849932 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116621 s, 89.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0164308 s, 63.8 MB/s , 0.0111278 s, 94.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00270384 s, 388 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00379706 s, 276 MB/s , 0.00141898 s, 739 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00174736 s, 600 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00155876 s, 673 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg329-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (4s) == sanityn test 77e: check TBF NID nrs policy ============ 09:29:55 (1747402195) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.203.29@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0902411 s, 186 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109527 s, 95.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013986 s, 75.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158434 s, 66.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105922 s, 99.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00703887 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0204574 s, 51.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00751195 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322229 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346574 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0305293 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00755501 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254345 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307413 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345013 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0325842 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322168 s, 32.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00740354 s, 142 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0101568 s, 103 MB/s , 0.00792246 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00887201 s, 118 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00941777 s, 111 MB/s , 0.0108448 s, 96.7 MB/s 1048576 bytes (1.0 MB) copied, 0.00978655 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0097724 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105621 s, 99.3 MB/s , 0.0103617 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103144 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137758 s, 76.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114651 s, 91.5 MB/s , 0.0125178 s, 83.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122295 s, 85.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137364 s, 76.3 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg329-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0960446 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105988 s, 98.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113667 s, 92.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237351 s, 44.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0366512 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392885 s, 26.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0294547 s, 35.6 MB/s , 0.0340699 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0382359 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354277 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0299972 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322656 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0407971 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306965 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338671 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295014 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.045308 s, 23.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00966626 s, 108 MB/s , 0.00691459 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111713 s, 93.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00784796 s, 134 MB/s 1048576 bytes (1.0 MB) copied, 0.00798505 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00693925 s, 151 MB/s , 0.0080633 s, 130 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100935 s, 104 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00897073 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0103949 s, 101 MB/s , 0.00885013 s, 118 MB/s 1048576 bytes (1.0 MB) copied, 0.0100573 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106295 s, 98.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00957398 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010786 s, 97.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103323 s, 101 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg329-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0929445 s, 181 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00973815 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165809 s, 63.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015302 s, 68.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119327 s, 87.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0224004 s, 46.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0334492 s, 31.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0375142 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308245 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0406627 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289588 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0347471 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360351 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.038839 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0410784 s, 25.5 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800aa899000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0359277 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0287635 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00859558 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00748394 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00816066 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00211719 s, 495 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115222 s, 91.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113431 s, 92.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109768 s, 95.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00384199 s, 273 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115234 s, 91.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00148337 s, 707 MB/s 1048576 bytes (1.0 MB) copied, 0.005366 s, 195 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0124862 s, 84.0 MB/s , 0.0134964 s, 77.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013257 s, 79.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146462 s, 71.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00170302 s, 616 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa899000.lock_unused_count=1 removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg329-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0990486 s, 169 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108498 s, 96.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121434 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263397 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0340225 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318688 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357428 s, 29.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa899000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff8800aa899000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.036237 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332732 s, 31.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0431142 s, 24.3 MB/s , 0.0430817 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0484393 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0479956 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400466 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0476624 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0477331 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0474904 s, 22.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00996645 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00805521 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00217042 s, 483 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00901128 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00153081 s, 685 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00184813 s, 567 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121447 s, 86.3 MB/s 1048576 bytes (1.0 MB) copied, 0.00624857 s, 168 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00131269 s, 799 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124048 s, 84.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00192404 s, 545 MB/s 1048576 bytes (1.0 MB) copied, 0.0129198 s, 81.2 MB/s 1048576 bytes (1.0 MB) copied, 0.013926 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129081 s, 81.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142862 s, 73.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142051 s, 73.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa899000.lock_unused_count=4 removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg329-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (6s) == sanityn test 77f: check TBF JobID nrs policy ========== 09:30:01 (1747402201) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg329-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.111433 s, 151 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00768646 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118221 s, 88.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143085 s, 73.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205928 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269618 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273317 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213751 s, 49.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279687 s, 37.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219413 s, 47.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0358884 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277598 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346456 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00890329 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0412912 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242659 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175459 s, 59.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00929308 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00804126 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00808688 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00924066 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118588 s, 88.4 MB/s , 0.00953306 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008173 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00188287 s, 557 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101371 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00961181 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00920557 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00732707 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00867119 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113493 s, 92.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101647 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102231 s, 103 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg329-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg329-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.10028 s, 167 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110363 s, 95.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00927169 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124459 s, 84.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.029579 s, 35.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0180812 s, 58.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03218 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217339 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0325882 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217712 s, 48.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0262578 s, 39.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0290383 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0403467 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266736 s, 39.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0470645 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288093 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0329243 s, 31.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00861108 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00763698 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00695946 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116828 s, 89.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00726227 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111578 s, 94.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00229335 s, 457 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00225876 s, 464 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00929408 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00987373 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00965108 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116164 s, 90.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103168 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0110819 s, 94.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0109232 s, 96.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115328 s, 90.9 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg329-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg329-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.093553 s, 179 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00745775 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123143 s, 85.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119795 s, 87.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161921 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272819 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186794 s, 56.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265659 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246387 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229081 s, 45.8 MB/s 1048576 bytes (1.0 MB) copied, 0.042204 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274904 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0224244 s, 46.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa899000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0420925 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0350151 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0474463 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170929 s, 61.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00613325 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100203 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115456 s, 90.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00771675 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0111749 s, 93.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0110021 s, 95.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00475061 s, 221 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00936374 s, 112 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0013813 s, 759 MB/s , 0.00369892 s, 283 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00916966 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101465 s, 103 MB/s 1048576 bytes (1.0 MB) copied, 0.00372322 s, 282 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103564 s, 101 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa899000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00179509 s, 584 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114401 s, 91.7 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg329-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg329-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0880575 s, 191 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106242 s, 98.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108075 s, 97.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141383 s, 74.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214902 s, 48.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294227 s, 35.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259475 s, 40.4 MB/s , 0.0222409 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233052 s, 45.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0282255 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020534 s, 51.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258253 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338811 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322448 s, 32.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.028114 s, 37.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0343209 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234353 s, 44.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00876379 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00156556 s, 670 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106897 s, 98.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00313389 s, 335 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00150425 s, 697 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00557243 s, 188 MB/s , 0.00166076 s, 631 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136295 s, 76.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120942 s, 86.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00912337 s, 115 MB/s , 0.0114143 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00939187 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00119029 s, 881 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102832 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0013505 s, 776 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00168878 s, 621 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg329-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (5s) == sanityn test 77g: Change TBF type directly ============ 09:30:07 (1747402207) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (2s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 09:30:09 (1747402209) oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg329-client: oleg329-server: ssh exited with exit code 2 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg329-client: oleg329-server: ssh exited with exit code 12 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg329-client: oleg329-server: ssh exited with exit code 12 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg329-client: oleg329-server: ssh exited with exit code 12 PASS 77h (4s) == sanityn test 77i: Change rank of TBF rule ============= 09:30:13 (1747402213) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (6s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 09:30:19 (1747402219) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0859061 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00899788 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00915275 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116587 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00922937 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0064906 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00966115 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170195 s, 61.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00876279 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00838781 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240843 s, 43.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241673 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0310635 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0382597 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0312351 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247809 s, 42.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250642 s, 41.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00716791 s, 146 MB/s , 0.0067846 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00981896 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00686282 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00871564 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105811 s, 99.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00861941 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00197604 s, 531 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103319 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00965341 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00979044 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115088 s, 91.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0104037 s, 101 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116698 s, 89.9 MB/s , 0.0113994 s, 92.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134774 s, 77.8 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg329-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93535 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.431 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (52s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 09:31:11 (1747402271) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg329-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.641021 s, 26.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.171314 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.163548 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.168528 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.173384 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.180831 s, 5.8 MB/s , 0.172837 s, 6.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.166075 s, 6.3 MB/s , 0.182477 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.371623 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.37563 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.372832 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.388075 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.385612 s, 2.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.390241 s, 2.7 MB/s , 0.377041 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.397769 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149704 s, 7.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.147135 s, 7.1 MB/s , 0.147847 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147067 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150287 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152667 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150697 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149698 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150821 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149518 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.150054 s, 7.0 MB/s 1048576 bytes (1.0 MB) copied, 0.149925 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150619 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150234 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150396 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152139 s, 6.9 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg329-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8296 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9685 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg329-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.640348 s, 26.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.170134 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.185943 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.182159 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.175856 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.191283 s, 5.5 MB/s , 0.191585 s, 5.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.186563 s, 5.6 MB/s , 0.192974 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.372207 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.385359 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.382881 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.387233 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.398183 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.400277 s, 2.6 MB/s , 0.399545 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.40117 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159302 s, 6.6 MB/s , 0.158446 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15684 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157051 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158963 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.160384 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158163 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158367 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159176 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159609 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.160716 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1602 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.16058 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159709 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.16006 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.161051 s, 6.5 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg329-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8296 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9683 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (135s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 09:33:26 (1747402406) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg329-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.193755 s, 86.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144341 s, 72.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124748 s, 84.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00849971 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182835 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225216 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240741 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293907 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00830043 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0611879 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0747114 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0767631 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0653044 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0779222 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0675709 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0625725 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0894081 s, 11.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00935129 s, 112 MB/s , 0.0104388 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00605947 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00647069 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00852786 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00778636 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00828187 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00830827 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00891106 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104255 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107003 s, 98.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110137 s, 95.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00944524 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113217 s, 92.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112556 s, 93.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116254 s, 90.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg329-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92511 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73633 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.29@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.29@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.187194 s, 89.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848774 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021421 s, 49.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275261 s, 38.1 MB/s , 0.0101896 s, 103 MB/s 1048576 bytes (1.0 MB) copied, 0.0172462 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135648 s, 77.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00652352 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195767 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0714905 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0744076 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169893 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.164599 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.27217 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.269293 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.364269 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.358157 s, 2.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105541 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00698308 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00873011 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112437 s, 93.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00986073 s, 106 MB/s 1048576 bytes (1.0 MB) copied, 0.00974959 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00989967 s, 106 MB/s , 0.0112561 s, 93.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115959 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106693 s, 98.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0105938 s, 99.0 MB/s , 0.0105105 s, 99.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011903 s, 88.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0135314 s, 77.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122959 s, 85.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123291 s, 85.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg329-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.91897 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73405 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.29@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg329-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.203273 s, 82.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00873272 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00947391 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112609 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0404226 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0416302 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0329225 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361061 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338899 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0703247 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0782275 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0896593 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0769566 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0941384 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.103531 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0942198 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.107154 s, 9.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00677808 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112064 s, 93.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00765594 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104009 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0088883 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00910451 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117522 s, 89.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00923715 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110487 s, 94.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00993888 s, 106 MB/s , 0.00947391 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00963122 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00156088 s, 672 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107827 s, 97.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106751 s, 98.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118899 s, 88.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg329-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92435 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.89325 s, 21.4 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg329-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.363296 s, 46.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327722 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286355 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109652 s, 95.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324492 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281935 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258754 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209022 s, 50.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0224565 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147605 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.176869 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.175681 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147133 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.320304 s, 3.3 MB/s , 0.335361 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.335093 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.354128 s, 3.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00909861 s, 115 MB/s , 0.0115952 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144373 s, 72.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00921024 s, 114 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158626 s, 66.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0145413 s, 72.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00237892 s, 441 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132498 s, 79.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125787 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145434 s, 72.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157268 s, 66.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138253 s, 75.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145117 s, 72.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150022 s, 69.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157807 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170431 s, 61.5 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg329-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.79927 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74437 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92553 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74606 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg329-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.642998 s, 26.1 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.16504 s, 6.4 MB/s , 0.160016 s, 6.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.162015 s, 6.5 MB/s , 0.160281 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.180496 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.175802 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159837 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.167374 s, 6.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.364548 s, 2.9 MB/s , 0.363006 s, 2.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.357786 s, 2.9 MB/s , 0.377495 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.3658 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.372023 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.362379 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.37105 s, 2.8 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out , 0.00886825 s, 118 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00891466 s, 118 MB/s , 0.00887594 s, 118 MB/s , 0.00898183 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134181 s, 78.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124115 s, 84.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0135215 s, 77.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111867 s, 93.7 MB/s , 0.0131024 s, 80.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125036 s, 83.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0051949 s, 202 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128236 s, 81.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0134675 s, 77.9 MB/s , 0.0125099 s, 83.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0133355 s, 78.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145349 s, 72.1 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg329-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.662 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4354 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg329-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.21884 s, 76.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163677 s, 64.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187642 s, 55.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00774139 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019394 s, 54.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0252052 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217236 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00982497 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184671 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0532248 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0593451 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0531571 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0723755 s, 14.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0722115 s, 14.5 MB/s , 0.0799966 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0748769 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0746534 s, 14.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00743853 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00941866 s, 111 MB/s 1048576 bytes (1.0 MB) copied, 0.0124524 s, 84.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00829085 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00962013 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010998 s, 95.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106553 s, 98.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138351 s, 75.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128027 s, 81.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013202 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139871 s, 75.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022386 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00213081 s, 492 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147232 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00200553 s, 523 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154988 s, 67.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg329-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92768 s, 21.3 MB/s Write runtime is 5 s, speed is 20.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73544 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg329-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.35043 s, 47.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152879 s, 68.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228853 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195288 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192422 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108881 s, 96.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00970118 s, 108 MB/s , 0.0127458 s, 82.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00836915 s, 125 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.152196 s, 6.9 MB/s , 0.150302 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166987 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.164726 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166726 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.187207 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.183795 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.198373 s, 5.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145619 s, 72.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159835 s, 65.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160572 s, 65.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154215 s, 68.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0176272 s, 59.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222885 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206956 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187389 s, 56.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214178 s, 49.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180734 s, 58.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233137 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020203 s, 51.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260771 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0191567 s, 54.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0191871 s, 54.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0054021 s, 194 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg329-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.79893 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73871 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77275 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73746 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (357s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 09:39:23 (1747402763) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (2s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 09:39:25 (1747402765) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg329-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.5807 s, 293 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg329-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0168 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (51s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 09:40:16 (1747402816) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.139786 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117007 s, 89.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146713 s, 71.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174509 s, 60.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0209059 s, 50.2 MB/s , 0.0255548 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0311181 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139784 s, 75.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388363 s, 27.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356403 s, 29.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0337502 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328561 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166366 s, 63.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263583 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346945 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279641 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275513 s, 38.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa899000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800aa899000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00864213 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00975731 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00723311 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00662979 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00267987 s, 391 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113108 s, 92.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00719242 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00730072 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00970866 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00815018 s, 129 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800aa899000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00855617 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00926149 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00899786 s, 117 MB/s 1048576 bytes (1.0 MB) copied, 0.00173048 s, 606 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112429 s, 93.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100329 s, 105 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg329-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95441 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97333 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.186228 s, 90.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226195 s, 46.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289313 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0373178 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0309709 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0730016 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0732372 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.114479 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.115577 s, 9.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.164697 s, 6.4 MB/s , 0.162604 s, 6.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.21893 s, 4.8 MB/s , 0.212683 s, 4.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.272689 s, 3.8 MB/s , 0.262912 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.357082 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.35913 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210606 s, 49.8 MB/s , 0.0202067 s, 51.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231387 s, 45.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211385 s, 49.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220598 s, 47.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0214452 s, 48.9 MB/s , 0.0235846 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220262 s, 47.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229592 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229632 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231268 s, 45.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236248 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240076 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239974 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234675 s, 44.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255587 s, 41.0 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg329-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg329-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92271 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96805 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (65s) == sanityn test 77o: Changing rank should not panic ====== 09:41:21 (1747402881) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (3s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 09:41:24 (1747402884) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (26s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 09:41:50 (1747402910) mds.MDS.mdt.nrs_policies=tbf oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg329-client: oleg329-server: ssh exited with exit code 36 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg329-client: oleg329-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (11s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 09:42:01 (1747402921) oleg329-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: Resource temporarily unavailable pdsh@oleg329-client: oleg329-server: ssh exited with exit code 11 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (2s) == sanityn test 79: xattr: intent error ================== 09:42:03 (1747402923) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505431 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:42:04.000000000 -0400 Modify: 2025-05-16 09:42:04.000000000 -0400 Change: 2025-05-16 09:42:04.000000000 -0400 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 09:42:15 (1747402935) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 09:42:16 (1747402936) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 09:42:17 (1747402937) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 09:42:18 (1747402938) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 09:42:19 (1747402939) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 09:42:20 (1747402940) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6857 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6857 PASS 82 (2s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 09:42:23 (1747402943) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 09:42:24 (1747402944) fail_loc=0x8000060b PASS 84 (8s) == sanityn test 90: open/create and unlink striped directory ========================================================== 09:42:31 (1747402951) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 09:42:32 (1747402952) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (0s) == sanityn test 92: create remote directory under orphan directory ========================================================== 09:42:32 (1747402952) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 09:42:33 (1747402953) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2627 0xa43 0 0 2647 0xa57 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2628 0xa44 0 0 2648 0xa58 0 rc1=0 and rc2=0 PASS 93 (8s) == sanityn test 94: signal vs CP callback race =========== 09:42:41 (1747402961) osc.lustre-OST0000-osc-ffff8800aa899000.idle_timeout=0 osc.lustre-OST0000-osc-ffff8800aab93800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800aa899000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800aab93800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0298562 s, 274 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 2.99711 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 4499 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0112283 s, 365 kB/s osc.lustre-OST0000-osc-ffff8800aa899000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800aab93800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aa899000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aab93800.idle_timeout=debug PASS 94 (10s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 09:42:51 (1747402971) llite.lustre-ffff8800aa899000.fast_read=0 llite.lustre-ffff8800aab93800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00162405 s, 7.6 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800aa899000.fast_read=1 llite.lustre-ffff8800aab93800.fast_read=1 PASS 95b (7s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 09:42:58 (1747402978) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 09:42:59 (1747402979) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0012754 s, 3.2 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 09:43:01 (1747402981) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0189566 s, 111 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 09:43:03 (1747402983) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0186155 s, 113 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (1s) == sanityn test 100e: DoM: read on open and file size ==== 09:43:04 (1747402984) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00167408 s, 7.2 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 09:43:06 (1747402986) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000498348 s, 8.2 MB/s PASS 101a (1s) == sanityn test 101b: Discard DoM data on rename ========= 09:43:07 (1747402987) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000592112 s, 6.9 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 09:43:09 (1747402989) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000534603 s, 7.7 MB/s PASS 101c (3s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 09:43:12 (1747402992) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1747402993 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1747402995 Finishing test test23 at 1747402995 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 09:43:19 (1747402999) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 09:43:20 (1747403000) fail_loc=0x416 PASS 105 (47s) == sanityn test 106a: Verify the btime via statx() ======= 09:44:07 (1747403047) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 09:44:08 (1747403048) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) == sanityn test 106c: Verify statx attributes mask ======= 09:44:09 (1747403049) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) == sanityn test 107a: Basic grouplock conflict =========== 09:44:10 (1747403050) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0816336 s, 128 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2633 0xa49 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6857 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6857 First grouplock blocks second one PASS 107a (5s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 09:44:15 (1747403055) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0796676 s, 132 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2654 0xa5e 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6857 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) == sanityn test 108a: lseek: parallel updates ============ 09:44:23 (1747403063) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00150544 s, 6.6 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.20307 s, 2.0 MB/s PASS 108a (6s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 09:44:29 (1747403069) 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 6 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 8 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 11 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 13 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 14 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 18 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 19 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 24 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 25 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 26 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 27 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 29 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 30 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 34 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 36 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 37 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 39 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 40 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions oleg329-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 43 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre 192.168.203.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg329-client.virtnet /mnt/lustre (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg329-server' oleg329-server: oleg329-server.virtnet: executing load_modules_local oleg329-server: Loading modules from /home/green/git/lustre-release/lustre oleg329-server: detected 4 online CPUs by sysfs oleg329-server: Force libcfs to create 2 CPU partitions Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre Starting client: oleg329-client.virtnet: -o user_xattr,flock oleg329-server@tcp:/lustre /mnt/lustre2 PASS 109 (434s) == sanityn test 110: do not grant another lock on resend ========================================================== 09:51:43 (1747403503) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:51:43.000000000 -0400 Modify: 2025-05-16 09:51:43.000000000 -0400 Change: 2025-05-16 09:51:43.000000000 -0400 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg329-client.virtnet oleg329-client.virtnet PASS 110 (43s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 09:52:26 (1747403546) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 09:52:27 (1747403547) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 09:52:28 (1747403548) lustre-OST0000-osc-ffff88012e3d4000 active. lustre-OST0001-osc-ffff88012e3d4000 active. lustre-OST0000-osc-ffff88012e3d4000 active. lustre-OST0001-osc-ffff88012e3d4000 active. MGC192.168.203.129@tcp active. PASS 113 (2s) cleanup: ====================================================== == sanityn test complete, duration 4141 sec ============== 09:52:31 (1747403551) Stopping clients: oleg329-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg329-client.virtnet /mnt/lustre2 opts: