-----============= acceptance-small: sanityn ============----- Wed Jul 17 08:00:28 EDT 2024 excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 Started clients oleg457-client.virtnet: 192.168.204.157@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt) oleg457-client.virtnet: executing check_config_client /mnt/lustre oleg457-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg457-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800a9d29800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800af5d3800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800a9d29800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800af5d3800.idle_timeout=debug disable quota as required oleg457-server: oleg457-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00953473 s, 110 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6821] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 08:00:40 (1721217640) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (1s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 08:00:41 (1721217641) -rw-r--r-- 1 root root 0 Jul 17 08:00 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (2s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 08:00:43 (1721217643) -rw-r--r-- 1 root root 0 Jul 17 08:00 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (1s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 08:00:44 (1721217644) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (1s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 08:00:45 (1721217645) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (2s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 08:00:47 (1721217647) total 1 -rwxrwxrwx 1 root root 0 Jul 17 08:00 f2a -rwxrwxrwx 1 root root 0 Jul 17 08:00 f2b total 1 -rwxrwxrwx 1 root root 0 Jul 17 08:00 f2a -rwxrwxrwx 1 root root 0 Jul 17 08:00 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00257977 s, 198 kB/s PASS 2e (1s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 08:00:48 (1721217648) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 08:00:49 (1721217649) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.08676 s, 24.2 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (1s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 08:00:50 (1721217650) PASS 3 (2s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 08:00:52 (1721217652) PASS 4 (2s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 08:00:54 (1721217654) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (2s) == sanityn test 6: remove of open file on other node ============================================================================ 08:00:56 (1721217656) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (1s) == sanityn test 7: remove of open directory on other node ======================================================================= 08:00:57 (1721217657) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (1s) == sanityn test 8: remove of open special file on other node ==================================================================== 08:00:58 (1721217658) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (2s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 08:01:00 (1721217660) PASS 9a (1s) == sanityn test 9b: append to striped sparse file ======== 08:01:01 (1721217661) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100215 s, 105 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.011157 s, 0.3 kB/s Data read (expecting 'foo'): foo PASS 9b (2s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 08:01:03 (1721217663) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00159671 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00144671 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00159192 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00145099 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00159604 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00149141 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00151808 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00141583 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00162028 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0013261 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.001566 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00138967 s, 0.7 kB/s PASS 10a (1s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 08:01:04 (1721217664) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00252321 s, 1.2 MB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0059793 s, 685 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000260275 s, 11.8 MB/s PASS 10b (2s) == sanityn test 11: execution of file opened for write should return error ============================================================== 08:01:06 (1721217666) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6821 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (1s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 08:01:07 (1721217667) warning: '-runtime' deprecated, use '-t runtime' instead - link 3203 (time 1721217678.17 total 10.00 last 320.23) - link 5351 (time 1721217688.17 total 20.01 last 214.68) - link 6948 (time 1721217698.17 total 30.01 last 159.68) - link 8821 (time 1721217708.18 total 40.01 last 187.24) - link 10000 (time 1721217715.60 total 47.44 last 158.79) - link 11724 (time 1721217725.60 total 57.44 last 172.34) - link 13844 (time 1721217735.61 total 67.44 last 211.94) - link 15875 (time 1721217745.61 total 77.45 last 203.04) - link 17883 (time 1721217755.61 total 87.45 last 200.77) - link 19593 (time 1721217765.62 total 97.45 last 170.95) total: 19945 link in 100.00 seconds: 199.45 ops/second using seed 88942268 running for 100 seconds - stat 10000 (time 1721217671 ; total 2 ; last 2) - stat 20000 (time 1721217673 ; total 4 ; last 2) - stat 30000 (time 1721217676 ; total 7 ; last 3) - stat 40000 (time 1721217679 ; total 10 ; last 3) - stat 50000 (time 1721217681 ; total 12 ; last 2) - stat 60000 (time 1721217685 ; total 16 ; last 4) - stat 70000 (time 1721217690 ; total 21 ; last 5) - stat 80000 (time 1721217694 ; total 25 ; last 4) - stat 90000 (time 1721217698 ; total 29 ; last 4) - stat 100000 (time 1721217702 ; total 33 ; last 4) - stat 110000 (time 1721217706 ; total 37 ; last 4) - stat 120000 (time 1721217710 ; total 41 ; last 4) - stat 130000 (time 1721217715 ; total 46 ; last 5) - stat 140000 (time 1721217719 ; total 50 ; last 4) - stat 149052 (time 1721217730 ; total 61 ; last 11) - stat 150000 (time 1721217731 ; total 62 ; last 1) - stat 160000 (time 1721217739 ; total 70 ; last 8) - stat 170000 (time 1721217748 ; total 79 ; last 9) - stat 179709 (time 1721217759 ; total 90 ; last 11) - stat 180000 (time 1721217759 ; total 90 ; last 0) - stat 190000 (time 1721217768 ; total 99 ; last 9) total: 190267 stats in 100 seconds: 1902.670044 stats/second - unlinked 0 (time 1721217719 ; total 0 ; last 0) - unlinked 10000 (time 1721217783 ; total 64 ; last 64) unlink(/mnt/lustre2/lockdir/lockfile19945) error: No such file or directory total: 19944 unlinks in 109 seconds: 182.972473 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (20415) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (20418) - No such process PASS 12 (163s) == sanityn test 13: test directory page revocation ======= 08:03:50 (1721217830) f13 f13 PASS 13 (2s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 08:03:52 (1721217832) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6821 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4253: 21811 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 08:03:55 (1721217835) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4253: 22439 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (2s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 08:03:57 (1721217837) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 23066 Terminated $DIR1/$tdir/sleep 60 PASS 14b (2s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 08:03:59 (1721217839) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 23695 Terminated $DIR1/$tdir/sleep 60 PASS 14c (2s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 08:04:01 (1721217841) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 24324 Terminated $DIR1/$tdir/sleep 60 PASS 14d (2s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 08:04:03 (1721217843) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7523328 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7523328kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (12s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 08:04:15 (1721217855) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.119951 s, 17.5 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0855575 s, 24.5 MB/s Chance of close/open is 1 in 50 Seed set to 7856 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read 000002[0] 1721217857.163709 write 0x15597b thru 0x15f0c7 (0x974d bytes) 000003[0] 1721217857.176806 mapread 0x03cf24 thru 0x0419d6 (0x4ab3 bytes) 000004[1] 1721217857.201449 write 0x9bb3cd thru 0x9c733d (0xbf71 bytes) 000007[0] 1721217857.224068 mapread 0x6a685c thru 0x6abb18 (0x52bd bytes) 000012[0] 1721217857.272451 read 0x512eb2 thru 0x51e1fd (0xb34c bytes) 000014[1] 1721217857.287315 mapwrite 0x721f96 thru 0x72755d (0x55c8 bytes) 000015[1] 1721217857.340793 write 0x2547c9 thru 0x25f269 (0xaaa1 bytes) 000017[0] 1721217857.357036 read 0x7e33c6 thru 0x7e369b (0x2d6 bytes) 000020[0] 1721217857.387943 write 0x9a40b8 thru 0x9b007d (0xbfc6 bytes) 000021[0] 1721217857.404621 write 0x8aeaef thru 0x8b2045 (0x3557 bytes) 000027[0] 1721217857.440462 mapread 0x640e53 thru 0x6500e7 (0xf295 bytes) 000029[1] 1721217857.496056 write 0x1e12e6 thru 0x1e83d2 (0x70ed bytes) 000031[1] 1721217857.525010 read 0x60765e thru 0x60fe33 (0x87d6 bytes) 000032[0] 1721217857.538171 read 0x943de0 thru 0x953985 (0xfba6 bytes) 000033[1] 1721217857.553719 mapread 0x29f8db thru 0x2a8123 (0x8849 bytes) 000035[1] 1721217857.564634 mapread 0x70eb91 thru 0x718d9c (0xa20c bytes) 000038[1] 1721217857.572493 mapwrite 0x75fe3f thru 0x76f232 (0xf3f4 bytes) 000042[0] 1721217857.630432 read 0x96c866 thru 0x96e61e (0x1db9 bytes) 000043[0] 1721217857.645782 mapread 0x03676d thru 0x03cbb3 (0x6447 bytes) 000044[0] 1721217857.663758 write 0x1d36fa thru 0x1dfa6d (0xc374 bytes) 000046[1] 1721217857.681924 read 0x9848ed thru 0x985faa (0x16be bytes) 000049[1] 1721217857.702118 write 0x007efe thru 0x00e482 (0x6585 bytes) truncating to largest ever: 0x4c8f10 000050[1] 1721217857.719450 trunc from 0x9c733e to 0x4c8f0f (0x4fe42e bytes) 000053[1] 1721217857.745984 read 0x4092d9 thru 0x410916 (0x763e bytes) 000054[0] 1721217857.754424 write 0x3fe2a5 thru 0x4083f5 (0xa151 bytes) 000056[0] 1721217857.768954 read 0x2748f9 thru 0x278b02 (0x420a bytes) 000057[0] 1721217857.787214 mapread 0x40d53e thru 0x40d998 (0x45b bytes) 000058[0] 1721217857.788229 write 0x812348 thru 0x81f0d3 (0xcd8c bytes) 000059[1] 1721217857.795914 mapwrite 0x0f7228 thru 0x0fe017 (0x6df0 bytes) 000060[0] 1721217857.849275 mapwrite 0x2f7f16 thru 0x303da4 (0xbe8f bytes) 000061[0] 1721217857.900756 read 0x032787 thru 0x039427 (0x6ca1 bytes) 000062[1] 1721217857.926531 mapwrite 0x0308d5 thru 0x0329ab (0x20d7 bytes) 000063[1] 1721217857.980617 write 0x725836 thru 0x734251 (0xea1c bytes) 000064[1] 1721217858.012171 mapread 0x70ce1f thru 0x70e2fa (0x14dc bytes) 000066[1] 1721217858.020501 mapwrite 0x448a1a thru 0x44b48d (0x2a74 bytes) 000068[0] 1721217858.060280 mapwrite 0x3c91cd thru 0x3d6472 (0xd2a6 bytes) 000069[0] 1721217858.146346 mapread 0x32caca thru 0x33707a (0xa5b1 bytes) 000071[0] 1721217858.195109 write 0x6ad659 thru 0x6b712e (0x9ad6 bytes) 000072[1] 1721217858.207106 mapread 0x47d765 thru 0x488fd2 (0xb86e bytes) 000073[1] 1721217858.246205 mapread 0x5f6216 thru 0x5fdbad (0x7998 bytes) 000075[0] 1721217858.269093 mapwrite 0x9ad65a thru 0x9ae36d (0xd14 bytes) 000076[0] 1721217858.316683 read 0x81b95e thru 0x829816 (0xdeb9 bytes) 000078[0] 1721217858.329016 mapwrite 0x0a46a9 thru 0x0b2efc (0xe854 bytes) 000079[1] 1721217858.384131 write 0x23f53f thru 0x24eac1 (0xf583 bytes) 000081[0] 1721217858.409577 mapwrite 0x404373 thru 0x408c3d (0x48cb bytes) 000082[0] 1721217858.455619 read 0x36233c thru 0x36ac1b (0x88e0 bytes) 000083[0] 1721217858.480019 mapwrite 0x19e3c0 thru 0x1a2347 (0x3f88 bytes) 000084[1] 1721217858.535133 mapread 0x7c573b thru 0x7d0601 (0xaec7 bytes) 000085[0] 1721217858.584094 mapwrite 0x3583ff thru 0x363e7d (0xba7f bytes) 000088[0] 1721217858.660648 mapwrite 0x293012 thru 0x294fe8 (0x1fd7 bytes) 000089[0] 1721217858.700199 mapwrite 0x65bc31 thru 0x66407e (0x844e bytes) truncating to largest ever: 0x51cd70 000090[0] 1721217858.727176 trunc from 0x9ae36e to 0x51cd6f (0x4915fe bytes) 000091[0] 1721217858.739233 mapwrite 0x43b80a thru 0x446c03 (0xb3fa bytes) 000092[1] 1721217858.776973 read 0x3659f1 thru 0x370499 (0xaaa9 bytes) 000093[1] 1721217858.791480 mapwrite 0x19df46 thru 0x1a3c3b (0x5cf6 bytes) 000097[0] 1721217858.871390 mapread 0x01a93c thru 0x02307c (0x8741 bytes) 000098[0] 1721217858.906047 mapwrite 0x687ddf thru 0x68cd4a (0x4f6c bytes) 000099[0] 1721217858.956971 mapread 0x03c9a1 thru 0x04c8da (0xff3a bytes) 000100[1] 1721217858.993590 write 0x907983 thru 0x915125 (0xd7a3 bytes) 000102[1] 1721217859.015475 trunc from 0x915126 to 0x3c70e2 (0x54e043 bytes) 000103[1] 1721217859.035625 write 0x80173b thru 0x81133f (0xfc05 bytes) 000104[1] 1721217859.040051 mapwrite 0x20504e thru 0x20df02 (0x8eb5 bytes) 000105[0] 1721217859.100568 mapwrite 0x149efc thru 0x1546aa (0xa7af bytes) 000108[1] 1721217859.185344 read 0x7da035 thru 0x7da7a4 (0x770 bytes) 000109[0] 1721217859.200839 mapread 0x4f74d3 thru 0x4fef60 (0x7a8e bytes) 000112[1] 1721217859.232180 mapwrite 0x5f488d thru 0x602a3b (0xe1af bytes) 000114[1] 1721217859.318693 read 0x0499b9 thru 0x04b42a (0x1a72 bytes) 000116[1] 1721217859.374491 write 0x823f4e thru 0x8260c7 (0x217a bytes) 000117[1] 1721217859.382756 mapwrite 0x7fdcae thru 0x805f9f (0x82f2 bytes) 000121[1] 1721217859.429334 trunc from 0x8260c8 to 0x36a8c8 (0x4bb7ff bytes) truncating to largest ever: 0x8c9a80 000123[0] 1721217859.445114 trunc from 0x36a8c9 to 0x8c9a7f (0x55f1b7 bytes) truncating to largest ever: 0x9c9769 000125[1] 1721217859.482028 trunc from 0x8c9a80 to 0x9c9768 (0xffce9 bytes) 000127[1] 1721217859.505386 read 0x52338d thru 0x525773 (0x23e7 bytes) 000128[0] 1721217859.540600 read 0x2d4bd7 thru 0x2e1bba (0xcfe4 bytes) 000129[1] 1721217859.560171 write 0x5448eb thru 0x54cf8d (0x86a3 bytes) 000130[1] 1721217859.568446 read 0x6d6e9f thru 0x6df06c (0x81ce bytes) 000131[0] 1721217859.590228 write 0x9d4f32 thru 0x9d9343 (0x4412 bytes) 000132[0] 1721217859.609923 mapwrite 0x337840 thru 0x33c899 (0x505a bytes) 000134[0] 1721217859.701202 mapwrite 0x6ee8cc thru 0x6f165b (0x2d90 bytes) 000138[1] 1721217859.781940 read 0x2a1970 thru 0x2a40a1 (0x2732 bytes) 000141[1] 1721217859.801619 write 0x4c422d thru 0x4c874f (0x4523 bytes) 000143[1] 1721217859.826760 read 0x58637a thru 0x594401 (0xe088 bytes) 000144[1] 1721217859.862498 mapread 0x64ec97 thru 0x6586be (0x9a28 bytes) 000146[0] 1721217859.871113 mapread 0x6d6547 thru 0x6dc27f (0x5d39 bytes) 000147[1] 1721217859.905254 read 0x0c8ee4 thru 0x0ce2ab (0x53c8 bytes) 000148[1] 1721217859.936254 mapwrite 0x78f8b5 thru 0x791520 (0x1c6c bytes) 000153[0] 1721217860.013623 mapwrite 0x5cdb02 thru 0x5d963f (0xbb3e bytes) 000154[0] 1721217860.079877 read 0x2d2772 thru 0x2e154b (0xedda bytes) 000157[1] 1721217860.142521 trunc from 0x9d9344 to 0x67b031 (0x35e312 bytes) 000159[0] 1721217860.179864 write 0x65838e thru 0x65d360 (0x4fd3 bytes) 000160[0] 1721217860.196324 mapwrite 0x754211 thru 0x75ec7f (0xaa6f bytes) 000162[1] 1721217860.270620 write 0x116d09 thru 0x12090d (0x9c05 bytes) 000163[0] 1721217860.286815 read 0x72899b thru 0x7371c7 (0xe82d bytes) 000164[0] 1721217860.304825 write 0x3767bf thru 0x386500 (0xfd42 bytes) 000165[1] 1721217860.314840 mapread 0x6684a0 thru 0x67051e (0x807f bytes) 000167[0] 1721217860.350232 read 0x1adf1b thru 0x1b8aae (0xab94 bytes) 000170[1] 1721217860.403370 write 0x1c2569 thru 0x1d18b2 (0xf34a bytes) 000171[0] 1721217860.427544 write 0x79e185 thru 0x7a61fe (0x807a bytes) 000172[1] 1721217860.444271 trunc from 0x7a61ff to 0x3971a5 (0x40f059 bytes) 000173[0] 1721217860.468701 write 0x76fd0c thru 0x77e1dd (0xe4d2 bytes) 000174[1] 1721217860.482309 write 0x687fe9 thru 0x68de8f (0x5ea7 bytes) 000175[0] 1721217860.490649 write 0x11780d thru 0x11968c (0x1e80 bytes) 000176[0] 1721217860.506681 mapwrite 0x515d21 thru 0x51c09c (0x637c bytes) 000178[1] 1721217860.570327 mapwrite 0x66fb89 thru 0x678fdf (0x9457 bytes) 000179[0] 1721217860.627026 write 0x964d9e thru 0x96a471 (0x56d4 bytes) 000180[1] 1721217860.638477 read 0x56e43d thru 0x5733f3 (0x4fb7 bytes) 000181[0] 1721217860.663050 trunc from 0x96a472 to 0x972f70 (0x8aff bytes) 000182[1] 1721217860.681512 trunc from 0x972f71 to 0x2f525e (0x67dd12 bytes) 000183[0] 1721217860.702885 trunc from 0x2f525f to 0x3c0a6a (0xcb80c bytes) 000184[0] 1721217860.720838 write 0x93bf30 thru 0x9421b0 (0x6281 bytes) 000185[1] 1721217860.729479 mapread 0x79e66d thru 0x79f13b (0xacf bytes) 000186[0] 1721217860.748013 write 0x4cb8ca thru 0x4da828 (0xef5f bytes) 000187[0] 1721217860.755094 mapread 0x64ace6 thru 0x64b6dc (0x9f7 bytes) 000188[0] 1721217860.771052 mapwrite 0x60f81f thru 0x61ed8e (0xf570 bytes) 000191[0] 1721217860.828009 mapwrite 0x1e2f44 thru 0x1edc70 (0xad2d bytes) 000192[1] 1721217860.872246 read 0x16141f thru 0x16817a (0x6d5c bytes) 000193[1] 1721217860.883195 trunc from 0x9421b1 to 0x435b93 (0x50c61d bytes) 000194[0] 1721217860.902581 mapwrite 0x6f4469 thru 0x6f7b1c (0x36b4 bytes) 000197[1] 1721217860.932894 read 0x59d6c4 thru 0x59e2c3 (0xc00 bytes) 000199[1] 1721217860.949684 mapread 0x302542 thru 0x30d846 (0xb305 bytes) 000200[0] 1721217860.977755 write 0x3bb690 thru 0x3c04bb (0x4e2c bytes) 000202[0] 1721217860.990245 read 0x5e1aed thru 0x5e858a (0x6a9e bytes) 000205[0] 1721217861.007648 read 0x019bbb thru 0x020f55 (0x739b bytes) 000214[1] 1721217861.068620 write 0x8c6784 thru 0x8d361e (0xce9b bytes) 000216[0] 1721217861.088343 write 0x04734c thru 0x04e863 (0x7518 bytes) 000217[0] 1721217861.107943 read 0x017db3 thru 0x01d33c (0x558a bytes) 000218[0] 1721217861.129715 trunc from 0x8d361f to 0x23f194 (0x69448a bytes) 000220[1] 1721217861.145616 write 0x53b51e thru 0x5405ab (0x508e bytes) 000223[0] 1721217861.173792 read 0x253037 thru 0x25a965 (0x792f bytes) 000228[1] 1721217861.221771 mapwrite 0x58a556 thru 0x593bcd (0x9678 bytes) 000229[0] 1721217861.269613 mapread 0x2c7bda thru 0x2d5447 (0xd86e bytes) 000230[0] 1721217861.292293 trunc from 0x593bce to 0x5c4df8 (0x3122b bytes) 000231[1] 1721217861.312716 trunc from 0x5c4df9 to 0x8ab366 (0x2e656e bytes) 000232[0] 1721217861.323322 trunc from 0x8ab367 to 0x3e2a9b (0x4c88cb bytes) 000234[0] 1721217861.340940 mapwrite 0x5dd7bb thru 0x5e7bef (0xa435 bytes) 000235[0] 1721217861.385999 mapwrite 0x85ab0a thru 0x85c0dc (0x15d3 bytes) 000236[1] 1721217861.412211 trunc from 0x85c0dd to 0x207c8c (0x654450 bytes) 000238[1] 1721217861.429371 write 0x9cf389 thru 0x9df351 (0xffc9 bytes) truncating to largest ever: 0x9d4605 000241[1] 1721217861.438073 trunc from 0x9df352 to 0x9d4604 (0xad4d bytes) 000246[0] 1721217861.460727 read 0x835e11 thru 0x83682f (0xa1f bytes) 000247[0] 1721217861.472116 mapwrite 0x9d5e90 thru 0x9dc465 (0x65d6 bytes) 000249[1] 1721217861.518086 read 0x30cab7 thru 0x30e873 (0x1dbd bytes) 000251[0] 1721217861.538725 write 0x0c7946 thru 0x0cd68d (0x5d48 bytes) 000252[1] 1721217861.542356 write 0x4520be thru 0x45895e (0x68a1 bytes) 000254[0] 1721217861.575770 write 0x38488a thru 0x38aca6 (0x641d bytes) 000255[1] 1721217861.592335 mapwrite 0x99d572 thru 0x9ad0f2 (0xfb81 bytes) 000257[0] 1721217861.648176 mapwrite 0x13b8fa thru 0x14278b (0x6e92 bytes) 000261[0] 1721217861.700487 mapwrite 0x6bd8c2 thru 0x6c15ad (0x3cec bytes) 000264[1] 1721217861.740861 trunc from 0x9dc466 to 0x704239 (0x2d822c bytes) 000265[0] 1721217861.755756 read 0x14c9b3 thru 0x14ed1d (0x236b bytes) 000266[1] 1721217861.767858 read 0x1a2598 thru 0x1ac3e8 (0x9e51 bytes) 000270[1] 1721217861.804938 mapread 0x2c85e7 thru 0x2ca60f (0x2029 bytes) 000272[0] 1721217861.811133 read 0x454894 thru 0x45b298 (0x6a05 bytes) 000273[0] 1721217861.822179 read 0x2aecb3 thru 0x2af72b (0xa79 bytes) 000276[0] 1721217861.841108 write 0x345ff5 thru 0x346eab (0xeb7 bytes) 000277[0] 1721217861.856507 write 0x306f5c thru 0x30c2d6 (0x537b bytes) 000278[1] 1721217861.860165 mapwrite 0x51a21c thru 0x522084 (0x7e69 bytes) 000281[1] 1721217861.898181 mapread 0x1ce855 thru 0x1d1407 (0x2bb3 bytes) 000283[1] 1721217861.904848 write 0x57f9ec thru 0x58d61b (0xdc30 bytes) 000288[1] 1721217861.925615 trunc from 0x70423a to 0x77d33d (0x79104 bytes) 000291[0] 1721217861.949373 trunc from 0x77d33e to 0x12d7b6 (0x64fb87 bytes) 000294[1] 1721217861.970386 mapwrite 0x3e3697 thru 0x3e7679 (0x3fe3 bytes) 000295[1] 1721217862.009454 mapwrite 0x0a31be thru 0x0ab8e4 (0x8727 bytes) 000296[1] 1721217862.043538 trunc from 0x3e767a to 0x359fe3 (0x8d696 bytes) 000297[1] 1721217862.054920 mapread 0x0ce14c thru 0x0d209b (0x3f50 bytes) 000301[0] 1721217862.065196 read 0x2c4070 thru 0x2cf25b (0xb1ec bytes) 000304[0] 1721217862.084232 trunc from 0x359fe4 to 0x1772db (0x1e2d08 bytes) 000305[1] 1721217862.099889 write 0x55cc4c thru 0x5653d1 (0x8786 bytes) 000306[1] 1721217862.110891 read 0x497e3a thru 0x4a7028 (0xf1ef bytes) 000308[0] 1721217862.129148 mapread 0x5205d1 thru 0x52a9ce (0xa3fe bytes) 000310[0] 1721217862.150347 mapread 0x4fc7a5 thru 0x505af3 (0x934f bytes) 000313[0] 1721217862.152990 read 0x511c64 thru 0x51fd8c (0xe129 bytes) 000314[0] 1721217862.153658 write 0x12a5e8 thru 0x12fa96 (0x54af bytes) 000315[1] 1721217862.164831 write 0x79cfe1 thru 0x7a4f54 (0x7f74 bytes) 000317[0] 1721217862.177303 read 0x4daa4e thru 0x4e261d (0x7bd0 bytes) 000318[1] 1721217862.180481 mapread 0x353746 thru 0x362eaa (0xf765 bytes) 000319[0] 1721217862.218011 mapwrite 0x0406f0 thru 0x04af57 (0xa868 bytes) 000322[0] 1721217862.274318 trunc from 0x7a4f55 to 0x586c37 (0x21e31d bytes) 000323[0] 1721217862.290771 trunc from 0x586c38 to 0x1b0bb2 (0x3d6085 bytes) 000325[1] 1721217862.304591 mapwrite 0x3130fe thru 0x31bfb4 (0x8eb7 bytes) 000326[0] 1721217862.358666 trunc from 0x31bfb5 to 0x28f392 (0x8cc22 bytes) 000327[0] 1721217862.374166 read 0x01d835 thru 0x02c400 (0xebcc bytes) 000329[1] 1721217862.401617 write 0x47bdd3 thru 0x47e255 (0x2483 bytes) 000330[1] 1721217862.415779 mapwrite 0x3195bf thru 0x324fe7 (0xba29 bytes) 000332[1] 1721217862.487209 read 0x2f7b1c thru 0x300088 (0x856d bytes) 000334[0] 1721217862.503604 trunc from 0x47e256 to 0x6b5408 (0x2371b3 bytes) 000336[0] 1721217862.534327 mapwrite 0x236d12 thru 0x24261c (0xb90b bytes) 000337[0] 1721217862.614782 write 0x1290e0 thru 0x13871a (0xf63b bytes) 000339[1] 1721217862.628183 trunc from 0x6b5409 to 0x19925c (0x51c1ac bytes) 000340[0] 1721217862.653991 write 0x7e6fe9 thru 0x7ec5cb (0x55e3 bytes) 000341[0] 1721217862.668599 read 0x260121 thru 0x264742 (0x4622 bytes) 000342[1] 1721217862.691903 mapread 0x469327 thru 0x475278 (0xbf52 bytes) 000343[1] 1721217862.780430 read 0x3a74b8 thru 0x3b16d8 (0xa221 bytes) 000344[1] 1721217862.816084 mapread 0x0263ff thru 0x0289be (0x25c0 bytes) 000345[1] 1721217862.823569 write 0x86a099 thru 0x86b783 (0x16eb bytes) 000348[0] 1721217862.846946 read 0x2f798b thru 0x2f8126 (0x79c bytes) 000349[1] 1721217862.884138 read 0x7d34e0 thru 0x7d8784 (0x52a5 bytes) 000351[1] 1721217862.886528 mapread 0x301e4c thru 0x30a9d8 (0x8b8d bytes) 000357[1] 1721217862.914609 read 0x0e8c17 thru 0x0f0fd4 (0x83be bytes) 000358[1] 1721217862.941819 mapread 0x6f6a8f thru 0x704c69 (0xe1db bytes) 000359[1] 1721217862.942510 read 0x0b9e08 thru 0x0bdb42 (0x3d3b bytes) 000360[1] 1721217862.949224 trunc from 0x86b784 to 0x7c21ff (0xa9584 bytes) 000362[1] 1721217862.983169 write 0x23fadd thru 0x24e26c (0xe790 bytes) 000364[0] 1721217862.995749 mapwrite 0x4452be thru 0x4529d7 (0xd71a bytes) 000365[0] 1721217863.080610 read 0x663ed8 thru 0x66ea23 (0xab4c bytes) 000367[0] 1721217863.104827 trunc from 0x7c2200 to 0x985d26 (0x1c3b27 bytes) 000368[1] 1721217863.117829 mapread 0x209bf7 thru 0x215308 (0xb712 bytes) 000370[1] 1721217863.175712 mapread 0x25453a thru 0x256d10 (0x27d7 bytes) 000373[1] 1721217863.194912 read 0x5e2c95 thru 0x5ead3e (0x80aa bytes) 000374[1] 1721217863.219029 read 0x1f94a4 thru 0x1febd6 (0x5733 bytes) 000375[1] 1721217863.240983 mapread 0x162572 thru 0x16bda8 (0x9837 bytes) 000376[0] 1721217863.266944 mapread 0x7a3ae6 thru 0x7a40a3 (0x5be bytes) 000377[0] 1721217863.282821 read 0x6368d5 thru 0x63e090 (0x77bc bytes) 000378[1] 1721217863.292799 write 0x44e898 thru 0x454c13 (0x637c bytes) 000380[1] 1721217863.314552 trunc from 0x985d27 to 0x53fc2b (0x4460fb bytes) 000382[0] 1721217863.332575 read 0x035439 thru 0x03d1da (0x7da2 bytes) 000384[1] 1721217863.361955 read 0x324a55 thru 0x32a8ce (0x5e7a bytes) 000385[1] 1721217863.383097 mapread 0x19cd5e thru 0x19f061 (0x2304 bytes) 000386[0] 1721217863.392508 trunc from 0x53fc2c to 0x72ee6a (0x1ef23f bytes) 000387[0] 1721217863.419493 mapwrite 0x39f60b thru 0x3a0c4b (0x1641 bytes) 000389[1] 1721217863.492641 read 0x2ac319 thru 0x2ad513 (0x11fb bytes) 000391[1] 1721217863.511962 write 0x33e441 thru 0x349642 (0xb202 bytes) 000392[0] 1721217863.549197 mapwrite 0x12b320 thru 0x139564 (0xe245 bytes) 000396[0] 1721217863.629444 mapwrite 0x9ee2f9 thru 0x9fb158 (0xce60 bytes) 000398[1] 1721217863.715809 read 0x451c7d thru 0x4537a6 (0x1b2a bytes) 000402[0] 1721217863.746330 write 0x676ca9 thru 0x67e2cd (0x7625 bytes) 000404[0] 1721217863.763936 trunc from 0x9fb159 to 0x9bcbdc (0x3e57c bytes) 000405[0] 1721217863.772822 trunc from 0x9bcbdd to 0x1daae1 (0x7e20fb bytes) 000407[0] 1721217863.789572 trunc from 0x1daae2 to 0x57fd2c (0x3a524b bytes) 000408[1] 1721217863.806095 mapread 0x0cbc58 thru 0x0d6d8d (0xb136 bytes) 000409[1] 1721217863.844513 trunc from 0x57fd2d to 0x360646 (0x21f6e6 bytes) 000410[1] 1721217863.860243 read 0x31c110 thru 0x325137 (0x9028 bytes) 000412[0] 1721217863.866658 write 0x6080c3 thru 0x611897 (0x97d5 bytes) 000413[0] 1721217863.878926 mapread 0x49cc36 thru 0x4abc2b (0xeff6 bytes) 000415[1] 1721217863.930486 mapwrite 0x411b9b thru 0x41afdb (0x9441 bytes) 000417[1] 1721217863.986678 trunc from 0x611898 to 0x6665bd (0x54d26 bytes) 000419[0] 1721217864.010064 mapread 0x001b98 thru 0x00f97c (0xdde5 bytes) 000421[1] 1721217864.069429 write 0x1527ad thru 0x1616c1 (0xef15 bytes) 000422[1] 1721217864.082580 trunc from 0x6665be to 0x197e9b (0x4ce722 bytes) 000423[1] 1721217864.105495 write 0x7ca1d1 thru 0x7d46d9 (0xa509 bytes) 000428[0] 1721217864.132806 write 0x058635 thru 0x05c25c (0x3c28 bytes) 000429[0] 1721217864.149380 mapread 0x3e7523 thru 0x3f47a7 (0xd285 bytes) 000438[1] 1721217864.251315 mapread 0x4f5fd2 thru 0x4fb45d (0x548c bytes) 000440[0] 1721217864.281834 trunc from 0x7d46da to 0x543530 (0x2911a9 bytes) 000441[1] 1721217864.296834 mapwrite 0x209133 thru 0x211f52 (0x8e20 bytes) 000443[1] 1721217864.370268 mapread 0x1b25fa thru 0x1b840b (0x5e12 bytes) 000444[1] 1721217864.402156 trunc from 0x543531 to 0x323a8c (0x21faa4 bytes) 000445[1] 1721217864.422454 write 0x9084ab thru 0x90a4d8 (0x202e bytes) 000447[0] 1721217864.434344 mapread 0x3a9627 thru 0x3b8f54 (0xf92e bytes) 000450[1] 1721217864.504659 write 0x111909 thru 0x11c2b8 (0xa9b0 bytes) 000456[1] 1721217864.543951 trunc from 0x90a4d9 to 0x61565c (0x2f4e7c bytes) 000458[0] 1721217864.568328 mapwrite 0x63a5d9 thru 0x6432e9 (0x8d11 bytes) 000460[0] 1721217864.647055 read 0x4dd73b thru 0x4e1dca (0x4690 bytes) 000461[0] 1721217864.663231 read 0x2f4590 thru 0x2f8cdd (0x474e bytes) 000462[1] 1721217864.685092 trunc from 0x6432ea to 0x9c565a (0x382371 bytes) 000463[1] 1721217864.714941 mapread 0x1fd282 thru 0x201b24 (0x48a3 bytes) 000464[0] 1721217864.734007 mapwrite 0x1a96df thru 0x1b769c (0xdfbe bytes) 000465[0] 1721217864.814613 mapread 0x9723b6 thru 0x9754eb (0x3136 bytes) 000466[0] 1721217864.835064 mapwrite 0x9f65e3 thru 0x9fffff (0x9a1d bytes) 000469[1] 1721217864.918386 mapwrite 0x616f15 thru 0x61b3fe (0x44ea bytes) 000472[0] 1721217864.993818 write 0x97f3b7 thru 0x98c9e4 (0xd62e bytes) 000474[1] 1721217865.009387 trunc from 0xa00000 to 0x0fc7e8 (0x903817 bytes) 000476[1] 1721217865.029291 mapwrite 0x09e50a thru 0x0ae2b2 (0xfda9 bytes) 000477[1] 1721217865.104919 write 0x1271b3 thru 0x135ac0 (0xe90e bytes) 000478[0] 1721217865.112260 read 0x070818 thru 0x0798a0 (0x9089 bytes) 000479[1] 1721217865.131573 write 0x250d25 thru 0x2581d0 (0x74ac bytes) 000480[0] 1721217865.145124 mapread 0x2114c5 thru 0x212852 (0x138e bytes) 000481[0] 1721217865.164936 trunc from 0x2581d1 to 0x74bd70 (0x4f3ba0 bytes) 000482[1] 1721217865.186823 mapwrite 0x32ee22 thru 0x3315ba (0x2799 bytes) 000483[0] 1721217865.242505 mapread 0x0ff9e5 thru 0x10c505 (0xcb21 bytes) 000486[1] 1721217865.303648 mapwrite 0x3af561 thru 0x3b1674 (0x2114 bytes) 000487[0] 1721217865.358034 trunc from 0x74bd71 to 0x55ca5e (0x1ef312 bytes) 000493[0] 1721217865.389594 trunc from 0x55ca5f to 0x4f1b42 (0x6af1c bytes) 000495[0] 1721217865.407949 mapread 0x2e3f3d thru 0x2e8545 (0x4609 bytes) 000497[0] 1721217865.408880 trunc from 0x4f1b43 to 0x252f5f (0x29ebe3 bytes) 000498[0] 1721217865.427528 read 0x20120a thru 0x203824 (0x261b bytes) 000500[1] 1721217865.437261 write 0x28c225 thru 0x28f3df (0x31bb bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 7865 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[1] 1721217865.708928 write 0x137000 thru 0x13a1bf (0x31c0 bytes) 000002[1] 1721217865.716300 write 0x4b0000 thru 0x4ba520 (0xa521 bytes) 000003[1] 1721217865.717987 mapwrite 0x8c0000 thru 0x8c7980 (0x7981 bytes) 000004[0] 1721217865.794823 mapwrite 0x81f000 thru 0x82ca52 (0xda53 bytes) truncating to largest ever: 0x3e58b7 000005[1] 1721217865.855616 trunc from 0x8c7981 to 0x3e58b6 (0x4e20ca bytes) 000009[0] 1721217865.877387 read 0x27b000 thru 0x27f4d2 (0x44d3 bytes) 000011[1] 1721217865.887921 read 0x3a2000 thru 0x3a215e (0x15f bytes) 000013[1] 1721217865.891015 trunc from 0x3e58b7 to 0x172385 (0x273531 bytes) 000014[1] 1721217865.903470 read 0x108000 thru 0x108882 (0x883 bytes) 000017[1] 1721217865.919956 mapread 0x170000 thru 0x171a14 (0x1a15 bytes) 000018[0] 1721217865.927335 mapread 0x132000 thru 0x140457 (0xe458 bytes) 000019[1] 1721217865.946065 read 0x03d000 thru 0x043467 (0x6468 bytes) 000022[1] 1721217865.956906 write 0x964000 thru 0x967039 (0x303a bytes) 000023[1] 1721217865.964170 mapwrite 0x222000 thru 0x231adf (0xfae0 bytes) 000024[1] 1721217866.033443 mapread 0x29a000 thru 0x2a1de6 (0x7de7 bytes) truncating to largest ever: 0x9da16a 000025[1] 1721217866.034171 trunc from 0x96703a to 0x9da169 (0x73130 bytes) 000026[0] 1721217866.042778 read 0x6a5000 thru 0x6b110c (0xc10d bytes) 000032[1] 1721217866.080389 trunc from 0x9da16a to 0x029dfd (0x9b036c bytes) 000034[1] 1721217866.100054 write 0x948000 thru 0x94cebc (0x4ebd bytes) 000036[0] 1721217866.119804 mapread 0x61a000 thru 0x61efb1 (0x4fb2 bytes) 000040[0] 1721217866.147641 read 0x118000 thru 0x123eec (0xbeed bytes) 000043[1] 1721217866.187700 read 0x134000 thru 0x137961 (0x3962 bytes) 000047[0] 1721217866.228667 mapread 0x7ba000 thru 0x7c0aca (0x6acb bytes) 000048[1] 1721217866.229287 mapread 0x77b000 thru 0x77b9b5 (0x9b6 bytes) 000049[1] 1721217866.229674 trunc from 0x94cebd to 0x240b4a (0x70c372 bytes) 000050[0] 1721217866.254751 trunc from 0x240b4b to 0x130d4a (0x10fe00 bytes) 000053[0] 1721217866.268270 mapwrite 0x80c000 thru 0x80e5be (0x25bf bytes) 000054[1] 1721217866.322867 read 0x62c000 thru 0x639f30 (0xdf31 bytes) 000056[1] 1721217866.342667 read 0x090000 thru 0x09b37b (0xb37c bytes) 000058[1] 1721217866.376921 write 0x627000 thru 0x62fe7b (0x8e7c bytes) 000059[0] 1721217866.397218 mapread 0x531000 thru 0x540b37 (0xfb38 bytes) 000060[1] 1721217866.443839 trunc from 0x80e5bf to 0x33b740 (0x4d2e7e bytes) 000061[0] 1721217866.461551 read 0x03a000 thru 0x03e64a (0x464b bytes) 000062[0] 1721217866.485827 trunc from 0x33b741 to 0x893db9 (0x558679 bytes) 000064[0] 1721217866.507117 read 0x3c8000 thru 0x3cc652 (0x4653 bytes) 000066[0] 1721217866.537389 trunc from 0x893dba to 0x4ee64e (0x3a576b bytes) 000068[0] 1721217866.561596 mapread 0x0ae000 thru 0x0b9b07 (0xbb08 bytes) 000069[0] 1721217866.600665 read 0x30a000 thru 0x317756 (0xd757 bytes) 000070[1] 1721217866.616594 trunc from 0x4ee64f to 0x11a4e4 (0x3d416a bytes) 000071[1] 1721217866.638937 read 0x048000 thru 0x055810 (0xd811 bytes) 000074[1] 1721217866.664971 read 0x04f000 thru 0x059116 (0xa117 bytes) 000075[0] 1721217866.665361 trunc from 0x11a4e5 to 0x370d7b (0x256897 bytes) 000076[1] 1721217866.680000 read 0x20a000 thru 0x20dd01 (0x3d02 bytes) 000078[1] 1721217866.702116 read 0x180000 thru 0x185ba0 (0x5ba1 bytes) 000079[0] 1721217866.710651 write 0x316000 thru 0x32520d (0xf20e bytes) 000080[0] 1721217866.722039 read 0x2b7000 thru 0x2bd067 (0x6068 bytes) 000081[0] 1721217866.739513 mapread 0x2f2000 thru 0x301926 (0xf927 bytes) 000083[1] 1721217866.747836 read 0x1d8000 thru 0x1e74c8 (0xf4c9 bytes) 000085[1] 1721217866.771191 write 0x48a000 thru 0x495a21 (0xba22 bytes) 000086[0] 1721217866.784371 mapwrite 0x8d9000 thru 0x8e65ec (0xd5ed bytes) 000087[1] 1721217866.872524 mapread 0x2d9000 thru 0x2db3ba (0x23bb bytes) 000088[1] 1721217866.912839 write 0x6c9000 thru 0x6cd10f (0x4110 bytes) 000089[1] 1721217866.930379 trunc from 0x8e65ed to 0x7fab94 (0xeba58 bytes) 000095[0] 1721217866.957299 read 0x45a000 thru 0x45bd91 (0x1d92 bytes) 000098[1] 1721217866.992470 mapwrite 0x814000 thru 0x81c205 (0x8206 bytes) 000099[1] 1721217867.069921 trunc from 0x81c206 to 0x366003 (0x4b6202 bytes) 000101[1] 1721217867.089462 read 0x185000 thru 0x192e38 (0xde39 bytes) 000102[0] 1721217867.101689 mapread 0x358000 thru 0x35f91b (0x791c bytes) 000103[0] 1721217867.123674 write 0x999000 thru 0x999759 (0x75a bytes) 000106[1] 1721217867.129642 mapwrite 0x1c1000 thru 0x1c3162 (0x2163 bytes) 000107[0] 1721217867.198335 mapread 0x62f000 thru 0x639e7e (0xae7f bytes) 000108[1] 1721217867.250941 write 0x558000 thru 0x562b6c (0xab6d bytes) 000117[0] 1721217867.277126 write 0x1f6000 thru 0x2012f8 (0xb2f9 bytes) 000118[1] 1721217867.289308 trunc from 0x99975a to 0x96615d (0x335fc bytes) 000119[1] 1721217867.305084 mapwrite 0x960000 thru 0x96e8e2 (0xe8e3 bytes) 000121[0] 1721217867.391511 mapwrite 0x5e6000 thru 0x5ea270 (0x4271 bytes) 000122[0] 1721217867.436712 trunc from 0x96e8e3 to 0x0014ba (0x96d428 bytes) 000136[0] 1721217867.506752 mapwrite 0x74b000 thru 0x75446b (0x946c bytes) 000137[0] 1721217867.581270 read 0x590000 thru 0x5907b3 (0x7b4 bytes) 000138[1] 1721217867.597517 mapwrite 0x6a9000 thru 0x6b6f00 (0xdf01 bytes) 000139[0] 1721217867.673585 mapwrite 0x33e000 thru 0x34b53f (0xd540 bytes) 000141[0] 1721217867.755678 trunc from 0x75446c to 0x84a27d (0xf5e12 bytes) 000142[0] 1721217867.772489 mapread 0x576000 thru 0x58146e (0xb46f bytes) 000143[0] 1721217867.820243 write 0x792000 thru 0x798114 (0x6115 bytes) 000151[0] 1721217867.854353 read 0x49a000 thru 0x4a31b9 (0x91ba bytes) 000156[0] 1721217867.878308 mapwrite 0x171000 thru 0x1713b7 (0x3b8 bytes) 000159[1] 1721217867.913452 read 0x38d000 thru 0x397c66 (0xac67 bytes) 000160[0] 1721217867.948788 mapwrite 0x320000 thru 0x32e347 (0xe348 bytes) 000163[1] 1721217868.034573 mapwrite 0x8ae000 thru 0x8bd399 (0xf39a bytes) 000165[1] 1721217868.091389 trunc from 0x8bd39a to 0x3370e6 (0x5862b3 bytes) 000166[1] 1721217868.101841 mapread 0x286000 thru 0x2951a8 (0xf1a9 bytes) 000167[1] 1721217868.143542 write 0x738000 thru 0x7426b5 (0xa6b6 bytes) 000168[0] 1721217868.145664 mapread 0x4bb000 thru 0x4c223c (0x723d bytes) 000171[1] 1721217868.180932 mapread 0x263000 thru 0x26dbca (0xabcb bytes) 000174[0] 1721217868.211225 mapread 0x5aa000 thru 0x5b28b7 (0x88b8 bytes) 000179[0] 1721217868.234488 mapread 0x59f000 thru 0x5a722c (0x822d bytes) 000180[1] 1721217868.257756 trunc from 0x7426b6 to 0x1e4fcd (0x55d6e8 bytes) 000182[1] 1721217868.283469 trunc from 0x1e4fce to 0x938819 (0x75384c bytes) 000184[1] 1721217868.302354 mapwrite 0x732000 thru 0x734f45 (0x2f46 bytes) 000185[1] 1721217868.342938 write 0x37f000 thru 0x37f0aa (0xab bytes) 000190[1] 1721217868.367967 trunc from 0x93881a to 0x3c2ba7 (0x575c72 bytes) 000191[1] 1721217868.376912 mapwrite 0x365000 thru 0x36a5db (0x55dc bytes) 000192[0] 1721217868.428837 trunc from 0x3c2ba8 to 0x199ed4 (0x228cd3 bytes) 000193[0] 1721217868.448228 trunc from 0x199ed5 to 0x44e3da (0x2b4506 bytes) 000196[1] 1721217868.466259 mapread 0x142000 thru 0x1509c0 (0xe9c1 bytes) 000198[0] 1721217868.505485 write 0x871000 thru 0x87d613 (0xc614 bytes) 000204[0] 1721217868.534737 mapwrite 0x201000 thru 0x20ea84 (0xda85 bytes) 000206[0] 1721217868.615855 trunc from 0x87d614 to 0x5d1b21 (0x2abaf2 bytes) 000210[1] 1721217868.656667 mapwrite 0x76c000 thru 0x776108 (0xa109 bytes) 000211[1] 1721217868.736247 write 0x1dc000 thru 0x1ddd5b (0x1d5c bytes) 000213[0] 1721217868.749151 write 0x619000 thru 0x6197c4 (0x7c5 bytes) 000214[1] 1721217868.766119 read 0x3c2000 thru 0x3c40b8 (0x20b9 bytes) 000215[1] 1721217868.793978 write 0x4ba000 thru 0x4bf47e (0x547f bytes) 000218[0] 1721217868.821553 mapread 0x582000 thru 0x583cf3 (0x1cf4 bytes) 000219[0] 1721217868.839183 trunc from 0x776109 to 0x080bc5 (0x6f5543 bytes) 000220[0] 1721217868.860460 read 0x047000 thru 0x0479d0 (0x9d1 bytes) 000224[0] 1721217868.883422 trunc from 0x080bc6 to 0x1ae65d (0x12da98 bytes) 000226[1] 1721217868.892111 mapread 0x04a000 thru 0x056da9 (0xcdaa bytes) 000230[0] 1721217868.937597 write 0x472000 thru 0x47d459 (0xb45a bytes) 000233[0] 1721217868.958371 trunc from 0x47d45a to 0x51493e (0x974e5 bytes) 000234[1] 1721217868.975248 mapwrite 0x245000 thru 0x2461e3 (0x11e4 bytes) 000237[0] 1721217869.031448 write 0x34e000 thru 0x3572ed (0x92ee bytes) 000238[1] 1721217869.059155 mapwrite 0x523000 thru 0x52ea19 (0xba1a bytes) 000239[0] 1721217869.145041 mapwrite 0x3a5000 thru 0x3ab62f (0x6630 bytes) 000242[1] 1721217869.200618 read 0x49c000 thru 0x4a4799 (0x879a bytes) 000243[0] 1721217869.216255 write 0x1a0000 thru 0x1aef20 (0xef21 bytes) 000244[0] 1721217869.225178 write 0x932000 thru 0x934e51 (0x2e52 bytes) 000245[1] 1721217869.234389 trunc from 0x934e52 to 0x9b0729 (0x7b8d8 bytes) 000246[1] 1721217869.260311 write 0x8ed000 thru 0x8fcc36 (0xfc37 bytes) 000249[0] 1721217869.284176 mapread 0x2fa000 thru 0x2fe2ad (0x42ae bytes) 000250[0] 1721217869.312819 trunc from 0x9b072a to 0x429827 (0x586f02 bytes) 000251[1] 1721217869.331904 read 0x3fa000 thru 0x3ff5e8 (0x55e9 bytes) 000252[0] 1721217869.340438 mapwrite 0x0c5000 thru 0x0cf1ef (0xa1f0 bytes) 000260[1] 1721217869.432694 read 0x2de000 thru 0x2eb288 (0xd289 bytes) 000264[0] 1721217869.450346 read 0x068000 thru 0x06e8ca (0x68cb bytes) 000265[1] 1721217869.470975 trunc from 0x429828 to 0x85f4da (0x435cb3 bytes) 000266[0] 1721217869.497183 write 0x969000 thru 0x969b17 (0xb18 bytes) 000269[0] 1721217869.511827 read 0x5b3000 thru 0x5b85a8 (0x55a9 bytes) 000270[1] 1721217869.537143 mapread 0x81e000 thru 0x82cd21 (0xed22 bytes) 000272[1] 1721217869.581956 mapwrite 0x76e000 thru 0x779957 (0xb958 bytes) 000273[0] 1721217869.675156 trunc from 0x969b18 to 0x8ff3d1 (0x6a746 bytes) 000275[0] 1721217869.703335 read 0x3c2000 thru 0x3d0314 (0xe315 bytes) 000277[0] 1721217869.737776 trunc from 0x8ff3d2 to 0x8846d2 (0x7acff bytes) 000278[0] 1721217869.754844 write 0x46a000 thru 0x46f357 (0x5358 bytes) 000280[1] 1721217869.764988 mapwrite 0x1ad000 thru 0x1b0b43 (0x3b44 bytes) 000281[0] 1721217869.826656 read 0x569000 thru 0x575263 (0xc264 bytes) 000282[0] 1721217869.847824 mapread 0x81c000 thru 0x8244dc (0x84dd bytes) 000285[1] 1721217869.852648 trunc from 0x8846d3 to 0x4c8399 (0x3bc339 bytes) 000286[0] 1721217869.877552 read 0x043000 thru 0x045a61 (0x2a62 bytes) 000287[0] 1721217869.904679 mapwrite 0x352000 thru 0x35e2aa (0xc2ab bytes) 000288[1] 1721217869.997876 read 0x365000 thru 0x372052 (0xd053 bytes) 000289[1] 1721217870.013577 write 0x108000 thru 0x11200f (0xa010 bytes) 000291[0] 1721217870.030499 write 0x2bd000 thru 0x2c0afb (0x3afc bytes) 000292[0] 1721217870.040778 read 0x453000 thru 0x45ccae (0x9caf bytes) 000295[1] 1721217870.055462 write 0x0e3000 thru 0x0eb3b8 (0x83b9 bytes) 000296[0] 1721217870.065782 read 0x08e000 thru 0x094ed8 (0x6ed9 bytes) 000297[0] 1721217870.093792 mapwrite 0x030000 thru 0x0363ad (0x63ae bytes) 000299[1] 1721217870.152158 trunc from 0x4c839a to 0x3d9080 (0xef319 bytes) 000304[0] 1721217870.189004 read 0x079000 thru 0x0802fc (0x72fd bytes) 000305[0] 1721217870.211052 mapread 0x164000 thru 0x16d898 (0x9899 bytes) 000306[0] 1721217870.211749 mapwrite 0x2a4000 thru 0x2a8fa0 (0x4fa1 bytes) 000307[0] 1721217870.273716 write 0x872000 thru 0x881630 (0xf631 bytes) 000308[1] 1721217870.283775 trunc from 0x881631 to 0x8552cb (0x2c365 bytes) 000309[1] 1721217870.301003 trunc from 0x8552cc to 0x6912a8 (0x1c4023 bytes) 000310[0] 1721217870.309005 read 0x4bd000 thru 0x4bfcc7 (0x2cc8 bytes) 000311[0] 1721217870.328637 mapread 0x38e000 thru 0x39d258 (0xf259 bytes) 000313[0] 1721217870.365056 read 0x68d000 thru 0x690b20 (0x3b21 bytes) 000315[0] 1721217870.365591 write 0x703000 thru 0x710b5d (0xdb5e bytes) 000316[1] 1721217870.373890 mapwrite 0x9a4000 thru 0x9a6971 (0x2972 bytes) 000317[1] 1721217870.444385 mapwrite 0x11a000 thru 0x1290d2 (0xf0d3 bytes) 000318[0] 1721217870.513131 mapread 0x40a000 thru 0x40d57c (0x357d bytes) 000320[1] 1721217870.541777 mapwrite 0x7eb000 thru 0x7eb320 (0x321 bytes) 000322[1] 1721217870.581630 read 0x37b000 thru 0x37e8c4 (0x38c5 bytes) 000323[1] 1721217870.615341 mapwrite 0x957000 thru 0x95bf2a (0x4f2b bytes) 000325[1] 1721217870.653360 mapwrite 0x582000 thru 0x58d28d (0xb28e bytes) 000328[1] 1721217870.692044 write 0x3b1000 thru 0x3b57dd (0x47de bytes) 000329[1] 1721217870.700361 write 0x5d4000 thru 0x5dece1 (0xace2 bytes) 000330[0] 1721217870.709111 read 0x818000 thru 0x827f79 (0xff7a bytes) 000331[1] 1721217870.739053 write 0x262000 thru 0x26d140 (0xb141 bytes) 000333[0] 1721217870.754739 mapwrite 0x71c000 thru 0x728254 (0xc255 bytes) 000334[1] 1721217870.824244 read 0x831000 thru 0x834929 (0x392a bytes) 000335[0] 1721217870.840435 trunc from 0x9a6972 to 0x48cd4b (0x519c26 bytes) 000336[0] 1721217870.862840 trunc from 0x48cd4c to 0x3b2a6a (0xda2e1 bytes) 000337[1] 1721217870.878849 mapwrite 0x67e000 thru 0x685134 (0x7135 bytes) 000340[0] 1721217870.965879 mapwrite 0x941000 thru 0x9505a7 (0xf5a8 bytes) 000342[0] 1721217871.048625 trunc from 0x9505a8 to 0x9d44f5 (0x83f4e bytes) 000344[1] 1721217871.069930 read 0x819000 thru 0x82859b (0xf59c bytes) 000345[0] 1721217871.092526 write 0x21e000 thru 0x22410f (0x6110 bytes) 000349[1] 1721217871.128312 read 0x424000 thru 0x42f5b7 (0xb5b8 bytes) 000350[0] 1721217871.167541 write 0x4ed000 thru 0x4f29ec (0x59ed bytes) 000351[1] 1721217871.182692 mapwrite 0x4a7000 thru 0x4b07d4 (0x97d5 bytes) 000352[0] 1721217871.255225 trunc from 0x9d44f6 to 0x418e40 (0x5bb6b5 bytes) 000354[0] 1721217871.268596 mapread 0x077000 thru 0x086fcd (0xffce bytes) 000356[0] 1721217871.305477 mapread 0x339000 thru 0x3392d9 (0x2da bytes) 000357[1] 1721217871.317617 mapread 0x30e000 thru 0x31c96c (0xe96d bytes) 000358[0] 1721217871.361056 trunc from 0x418e41 to 0x162423 (0x2b6a1d bytes) 000359[1] 1721217871.373868 mapwrite 0x1be000 thru 0x1c358e (0x558f bytes) 000360[1] 1721217871.436327 read 0x10b000 thru 0x11a454 (0xf455 bytes) 000362[0] 1721217871.454030 mapread 0x0df000 thru 0x0e843c (0x943d bytes) 000365[1] 1721217871.488997 trunc from 0x1c358f to 0x41e781 (0x25b1f3 bytes) 000367[0] 1721217871.510512 read 0x12d000 thru 0x131110 (0x4111 bytes) 000368[1] 1721217871.532814 mapread 0x3e0000 thru 0x3edeb7 (0xdeb8 bytes) 000369[1] 1721217871.550617 mapread 0x1c1000 thru 0x1c8d46 (0x7d47 bytes) 000374[0] 1721217871.557987 read 0x351000 thru 0x35bd4c (0xad4d bytes) 000376[0] 1721217871.558532 mapread 0x2bb000 thru 0x2c19a9 (0x69aa bytes) 000379[0] 1721217871.559240 read 0x1dc000 thru 0x1e2379 (0x637a bytes) 000381[1] 1721217871.569957 write 0x6d0000 thru 0x6de102 (0xe103 bytes) 000382[1] 1721217871.581526 mapread 0x401000 thru 0x40d828 (0xc829 bytes) 000383[0] 1721217871.594611 mapwrite 0x28a000 thru 0x29624e (0xc24f bytes) 000385[0] 1721217871.669251 mapread 0x05d000 thru 0x05d6cc (0x6cd bytes) 000386[0] 1721217871.671606 mapread 0x50e000 thru 0x51d8bc (0xf8bd bytes) 000388[0] 1721217871.711322 mapwrite 0x59c000 thru 0x5a7678 (0xb679 bytes) 000389[0] 1721217871.779133 mapread 0x6b4000 thru 0x6bbd79 (0x7d7a bytes) 000390[0] 1721217871.797215 write 0x53e000 thru 0x53ee8e (0xe8f bytes) 000391[1] 1721217871.799771 trunc from 0x6de103 to 0x339cd3 (0x3a442f bytes) 000392[0] 1721217871.822832 trunc from 0x339cd4 to 0x4d9f72 (0x1a029f bytes) 000395[0] 1721217871.850139 read 0x4c3000 thru 0x4c5e45 (0x2e46 bytes) 000396[1] 1721217871.863345 trunc from 0x4d9f73 to 0x6534e2 (0x179570 bytes) 000400[1] 1721217871.885431 write 0x1d8000 thru 0x1e6670 (0xe671 bytes) 000401[0] 1721217871.893071 trunc from 0x6534e3 to 0x4b33a2 (0x1a0140 bytes) 000402[0] 1721217871.904249 mapread 0x177000 thru 0x18460e (0xd60f bytes) 000403[0] 1721217871.931345 mapwrite 0x56b000 thru 0x578b1e (0xdb1f bytes) 000405[1] 1721217871.970481 trunc from 0x578b1f to 0x838921 (0x2bfe03 bytes) 000406[1] 1721217871.982993 trunc from 0x838922 to 0x72b800 (0x10d121 bytes) 000407[0] 1721217871.997380 write 0x37e000 thru 0x3804e3 (0x24e4 bytes) 000408[0] 1721217872.012683 write 0x96c000 thru 0x973fd5 (0x7fd6 bytes) 000409[0] 1721217872.014229 read 0x76f000 thru 0x770be3 (0x1be4 bytes) 000410[1] 1721217872.028658 mapread 0x0ea000 thru 0x0f99aa (0xf9ab bytes) 000412[1] 1721217872.055227 mapread 0x5b8000 thru 0x5bc3f3 (0x43f4 bytes) 000413[0] 1721217872.070277 read 0x817000 thru 0x81addd (0x3dde bytes) 000415[1] 1721217872.082548 mapread 0x202000 thru 0x2020c4 (0xc5 bytes) 000417[0] 1721217872.085539 mapread 0x57c000 thru 0x5850ac (0x90ad bytes) 000418[1] 1721217872.108746 mapread 0x782000 thru 0x7863e3 (0x43e4 bytes) 000421[1] 1721217872.118996 read 0x94a000 thru 0x94f8c0 (0x58c1 bytes) 000423[1] 1721217872.123146 read 0x95e000 thru 0x96d193 (0xf194 bytes) 000426[0] 1721217872.127811 mapread 0x8bf000 thru 0x8c37d3 (0x47d4 bytes) 000427[0] 1721217872.128246 mapwrite 0x075000 thru 0x07f716 (0xa717 bytes) 000429[0] 1721217872.204680 mapwrite 0x0fc000 thru 0x104c6c (0x8c6d bytes) 000433[1] 1721217872.279578 trunc from 0x973fd6 to 0x89f954 (0xd4681 bytes) 000436[1] 1721217872.302526 write 0x138000 thru 0x141d8c (0x9d8d bytes) 000440[1] 1721217872.319969 trunc from 0x89f955 to 0x950e15 (0xb14c1 bytes) 000441[1] 1721217872.329198 mapwrite 0x2c1000 thru 0x2c82ff (0x7300 bytes) 000442[1] 1721217872.384720 mapwrite 0x9dd000 thru 0x9ec693 (0xf694 bytes) 000444[0] 1721217872.432287 read 0x1c0000 thru 0x1cee50 (0xee51 bytes) 000446[0] 1721217872.479510 mapread 0x5fa000 thru 0x5fd6c0 (0x36c1 bytes) 000452[1] 1721217872.480985 mapwrite 0x362000 thru 0x364f49 (0x2f4a bytes) 000459[1] 1721217872.544567 write 0x440000 thru 0x44060e (0x60f bytes) 000460[1] 1721217872.555275 write 0x77d000 thru 0x78244f (0x5450 bytes) 000462[0] 1721217872.566287 mapwrite 0x284000 thru 0x28f82a (0xb82b bytes) 000463[0] 1721217872.636951 trunc from 0x9ec694 to 0x839aec (0x1b2ba7 bytes) 000464[0] 1721217872.651941 mapread 0x46c000 thru 0x47ada4 (0xeda5 bytes) 000466[0] 1721217872.694294 mapread 0x632000 thru 0x638d77 (0x6d78 bytes) 000468[0] 1721217872.713776 trunc from 0x839aed to 0x1661b3 (0x6d3939 bytes) 000469[0] 1721217872.723381 write 0x077000 thru 0x07826c (0x126d bytes) 000470[1] 1721217872.726168 mapwrite 0x81f000 thru 0x829d6b (0xad6c bytes) 000471[0] 1721217872.806034 read 0x3fe000 thru 0x405264 (0x7265 bytes) 000473[1] 1721217872.834586 write 0x81d000 thru 0x8245b9 (0x75ba bytes) 000474[1] 1721217872.858735 mapread 0x2b7000 thru 0x2c446a (0xd46b bytes) 000478[1] 1721217872.911371 mapwrite 0x813000 thru 0x822330 (0xf331 bytes) 000479[0] 1721217872.976270 write 0x04d000 thru 0x057021 (0xa022 bytes) 000481[0] 1721217872.997222 mapwrite 0x392000 thru 0x3a1a53 (0xfa54 bytes) 000483[1] 1721217873.072243 read 0x632000 thru 0x63e385 (0xc386 bytes) 000484[1] 1721217873.092869 read 0x1a5000 thru 0x1abe8e (0x6e8f bytes) 000486[0] 1721217873.114617 read 0x720000 thru 0x72c559 (0xc55a bytes) 000487[1] 1721217873.122775 mapwrite 0x3de000 thru 0x3eb20f (0xd210 bytes) 000489[1] 1721217873.215546 trunc from 0x829d6c to 0x2b2e1b (0x576f50 bytes) 000490[0] 1721217873.223775 mapwrite 0x22a000 thru 0x22bb42 (0x1b43 bytes) 000491[0] 1721217873.266372 write 0x2ba000 thru 0x2c17e9 (0x77ea bytes) 000497[1] 1721217873.290190 mapwrite 0x4d2000 thru 0x4dbc16 (0x9c17 bytes) 000500[1] 1721217873.387465 write 0x043000 thru 0x044021 (0x1022 bytes) All operations completed A-OK! PASS 16a (19s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 08:04:35 (1721217875) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.152087 s, 13.8 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.10542 s, 19.9 MB/s Chance of close/open is 1 in 50 Seed set to 7875 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[1] 1721217876.175663 write 0x4b2489 thru 0x4b41f6 (0x1d6e bytes) truncating to largest ever: 0x4f9016 000004[1] 1721217876.202776 trunc from 0x4b41f7 to 0x4f9015 (0x44e1f bytes) 000005[0] 1721217876.222148 mapwrite 0x5bce0e thru 0x5be400 (0x15f3 bytes) 000006[0] 1721217876.296230 mapread 0x13e658 thru 0x13f82d (0x11d6 bytes) 000007[1] 1721217876.308035 mapwrite 0x516165 thru 0x516868 (0x704 bytes) 000015[1] 1721217876.407190 mapwrite 0x156c65 thru 0x1574a2 (0x83e bytes) 000016[1] 1721217876.437349 write 0x4946ec thru 0x4949e7 (0x2fc bytes) 000018[1] 1721217876.449336 write 0x612e95 thru 0x613cea (0xe56 bytes) 000019[1] 1721217876.450500 mapwrite 0x8e89e1 thru 0x8e948e (0xaae bytes) 000021[1] 1721217876.508457 write 0x08c60d thru 0x08e3fc (0x1df0 bytes) 000024[1] 1721217876.524156 write 0x737128 thru 0x738aed (0x19c6 bytes) 000025[1] 1721217876.529240 mapwrite 0x2a0a46 thru 0x2a0d02 (0x2bd bytes) 000027[0] 1721217876.566761 mapread 0x515cfe thru 0x51600e (0x311 bytes) truncating to largest ever: 0x69223e 000031[1] 1721217876.600491 trunc from 0x8e948f to 0x69223d (0x257251 bytes) 000032[0] 1721217876.612814 trunc from 0x69223e to 0x3cf5bd (0x2c2c80 bytes) 000034[0] 1721217876.641223 mapread 0x239e97 thru 0x23abd1 (0xd3b bytes) 000036[1] 1721217876.648476 read 0x35bdbe thru 0x35d4c5 (0x1708 bytes) 000037[0] 1721217876.656983 write 0x9d188c thru 0x9d2295 (0xa0a bytes) 000044[0] 1721217876.691196 trunc from 0x9d2296 to 0x46d4f1 (0x564da4 bytes) 000045[1] 1721217876.704710 trunc from 0x46d4f2 to 0x062dba (0x40a737 bytes) 000048[1] 1721217876.732833 mapwrite 0x33422a thru 0x334aed (0x8c4 bytes) 000050[0] 1721217876.771885 mapwrite 0x6c3e54 thru 0x6c5242 (0x13ef bytes) truncating to largest ever: 0x69e4a2 000051[1] 1721217876.822009 trunc from 0x6c5243 to 0x69e4a1 (0x26da1 bytes) 000052[0] 1721217876.843918 write 0x453410 thru 0x453d57 (0x948 bytes) 000053[1] 1721217876.856832 trunc from 0x69e4a2 to 0x5f8701 (0xa5da0 bytes) 000054[1] 1721217876.868784 read 0x54b3f8 thru 0x54c918 (0x1521 bytes) 000056[0] 1721217876.880247 mapread 0x5e19ef thru 0x5e33dc (0x19ee bytes) 000058[1] 1721217876.909605 trunc from 0x5f8702 to 0x4b6cfb (0x141a06 bytes) 000060[0] 1721217876.922190 mapwrite 0x549030 thru 0x54a681 (0x1652 bytes) 000062[1] 1721217876.973848 mapwrite 0x61ef5b thru 0x61efd3 (0x79 bytes) 000063[1] 1721217877.023776 mapwrite 0x773f75 thru 0x775689 (0x1715 bytes) truncating to largest ever: 0x6c595c 000065[0] 1721217877.083521 trunc from 0x77568a to 0x6c595b (0xafd2e bytes) 000066[0] 1721217877.109781 trunc from 0x6c595c to 0x4b2ac5 (0x212e96 bytes) 000067[0] 1721217877.125246 write 0x702107 thru 0x702eac (0xda6 bytes) 000068[1] 1721217877.133361 write 0x1cba5a thru 0x1cd080 (0x1627 bytes) 000069[1] 1721217877.153298 trunc from 0x702ead to 0x447b28 (0x2bb384 bytes) 000070[0] 1721217877.170387 mapread 0x323f94 thru 0x324341 (0x3ae bytes) 000075[0] 1721217877.211154 mapwrite 0x5893a9 thru 0x589cf1 (0x949 bytes) 000078[1] 1721217877.273706 read 0x0e36d6 thru 0x0e3765 (0x90 bytes) 000080[1] 1721217877.292481 mapread 0x1c6848 thru 0x1c80ba (0x1873 bytes) 000082[0] 1721217877.330537 read 0x07be93 thru 0x07d41d (0x158b bytes) 000084[0] 1721217877.333826 read 0x3b1057 thru 0x3b2a75 (0x1a1f bytes) 000086[1] 1721217877.348182 mapwrite 0x204b3d thru 0x2050d9 (0x59d bytes) 000087[0] 1721217877.391133 read 0x52c445 thru 0x52cc29 (0x7e5 bytes) 000088[1] 1721217877.391512 read 0x3f9073 thru 0x3fb03c (0x1fca bytes) 000089[1] 1721217877.391817 trunc from 0x589cf2 to 0x0f1fcb (0x497d26 bytes) 000091[1] 1721217877.417949 mapread 0x0809d6 thru 0x0810aa (0x6d5 bytes) 000094[1] 1721217877.437965 mapread 0x03fd8a thru 0x041731 (0x19a8 bytes) 000097[1] 1721217877.451335 mapwrite 0x287e75 thru 0x28840a (0x596 bytes) 000098[1] 1721217877.471522 mapwrite 0x9b2e2d thru 0x9b397d (0xb51 bytes) 000099[0] 1721217877.498392 mapread 0x81282b thru 0x8147eb (0x1fc1 bytes) 000107[1] 1721217877.525737 read 0x22c005 thru 0x22cebe (0xeba bytes) 000108[0] 1721217877.527460 read 0x3f1a09 thru 0x3f225f (0x857 bytes) 000110[1] 1721217877.555030 mapread 0x8aa8d8 thru 0x8ab0ab (0x7d4 bytes) 000113[0] 1721217877.558801 read 0x061f35 thru 0x0621da (0x2a6 bytes) 000114[0] 1721217877.575292 mapread 0x137c58 thru 0x1393be (0x1767 bytes) truncating to largest ever: 0x71f08b 000116[1] 1721217877.577874 trunc from 0x9b397e to 0x71f08a (0x2948f3 bytes) 000122[1] 1721217877.613468 mapwrite 0x701603 thru 0x703396 (0x1d94 bytes) 000123[1] 1721217877.632516 trunc from 0x71f08b to 0x69df10 (0x8117a bytes) 000124[1] 1721217877.644892 write 0x4c2d2c thru 0x4c459a (0x186f bytes) 000125[1] 1721217877.650573 mapread 0x471828 thru 0x471f96 (0x76f bytes) 000127[0] 1721217877.665132 mapwrite 0x268786 thru 0x269697 (0xf12 bytes) 000130[1] 1721217877.712758 trunc from 0x69df11 to 0x5c3088 (0xdae88 bytes) 000134[1] 1721217877.741234 write 0x47a751 thru 0x47ab9f (0x44f bytes) 000135[1] 1721217877.744854 mapread 0x4e6f77 thru 0x4e8491 (0x151b bytes) 000138[1] 1721217877.765293 mapread 0x2e464e thru 0x2e5ebc (0x186f bytes) 000141[0] 1721217877.787864 read 0x4a773e thru 0x4a8d47 (0x160a bytes) 000142[0] 1721217877.805938 write 0x21d9c1 thru 0x21f35d (0x199d bytes) 000144[0] 1721217877.826107 write 0x60afdd thru 0x60b55a (0x57e bytes) 000148[0] 1721217877.852332 trunc from 0x60b55b to 0x45fbbe (0x1ab99c bytes) 000149[1] 1721217877.866257 read 0x18c45f thru 0x18d65e (0x1200 bytes) 000150[1] 1721217877.879959 write 0x4c6c73 thru 0x4c82a7 (0x1635 bytes) 000152[1] 1721217877.884462 read 0x1d891e thru 0x1d8c5f (0x342 bytes) 000154[0] 1721217877.889666 read 0x2e3eea thru 0x2e3f3a (0x51 bytes) 000155[1] 1721217877.910035 mapwrite 0x6b4d17 thru 0x6b612d (0x1417 bytes) 000157[0] 1721217877.951041 mapread 0x5d0681 thru 0x5d09c8 (0x348 bytes) 000158[1] 1721217877.960153 mapwrite 0x635bd6 thru 0x636401 (0x82c bytes) 000160[0] 1721217877.985728 mapwrite 0x3549eb thru 0x355aa6 (0x10bc bytes) truncating to largest ever: 0x9a4d4e 000162[0] 1721217878.021050 trunc from 0x6b612e to 0x9a4d4d (0x2eec20 bytes) 000163[0] 1721217878.031523 read 0x30aa3d thru 0x30befe (0x14c2 bytes) 000167[1] 1721217878.083475 mapwrite 0x454ef2 thru 0x454f38 (0x47 bytes) 000169[1] 1721217878.136520 mapread 0x63f3a6 thru 0x640d63 (0x19be bytes) 000171[0] 1721217878.158914 write 0x0f9299 thru 0x0f9350 (0xb8 bytes) 000180[0] 1721217878.197095 trunc from 0x9a4d4e to 0x89236c (0x1129e1 bytes) 000182[1] 1721217878.213395 mapwrite 0x80ff1d thru 0x8116e1 (0x17c5 bytes) 000183[1] 1721217878.266496 mapread 0x010375 thru 0x0118e6 (0x1572 bytes) 000184[1] 1721217878.281187 mapread 0x45e3b7 thru 0x45eead (0xaf7 bytes) 000186[1] 1721217878.300215 mapwrite 0x78fa5e thru 0x790b5d (0x1100 bytes) 000189[0] 1721217878.357332 read 0x85f83d thru 0x860118 (0x8dc bytes) 000191[1] 1721217878.376799 trunc from 0x89236d to 0x49dcdc (0x3f4690 bytes) 000192[1] 1721217878.396484 trunc from 0x49dcdd to 0x3bfe66 (0xdde76 bytes) 000194[1] 1721217878.412448 mapread 0x16d091 thru 0x16dd74 (0xce4 bytes) 000196[0] 1721217878.421567 mapread 0x19daad thru 0x19ee96 (0x13ea bytes) 000199[0] 1721217878.441645 write 0x4f1e12 thru 0x4f1fb7 (0x1a6 bytes) 000207[1] 1721217878.489991 trunc from 0x4f1fb8 to 0x563ecb (0x71f14 bytes) 000208[0] 1721217878.508875 write 0x451069 thru 0x4510d2 (0x6a bytes) 000209[0] 1721217878.522751 mapread 0x1c8b53 thru 0x1c907e (0x52c bytes) 000212[0] 1721217878.553659 read 0x54508b thru 0x5466c7 (0x163d bytes) 000213[0] 1721217878.559560 read 0x372f15 thru 0x3736b7 (0x7a3 bytes) 000217[0] 1721217878.575289 write 0x6475a6 thru 0x6478cf (0x32a bytes) 000218[1] 1721217878.581341 write 0x781625 thru 0x781ec1 (0x89d bytes) 000219[0] 1721217878.588641 read 0x637ff8 thru 0x639833 (0x183c bytes) 000220[1] 1721217878.600319 write 0x98c340 thru 0x98c809 (0x4ca bytes) 000221[1] 1721217878.609243 mapread 0x33df4c thru 0x33f9c8 (0x1a7d bytes) 000222[1] 1721217878.632037 mapread 0x12bb7d thru 0x12c8c3 (0xd47 bytes) 000223[0] 1721217878.648117 mapread 0x5f5110 thru 0x5f6029 (0xf1a bytes) 000224[0] 1721217878.692129 mapwrite 0x063ddb thru 0x064514 (0x73a bytes) 000226[0] 1721217878.728310 mapread 0x701f46 thru 0x702232 (0x2ed bytes) 000228[0] 1721217878.741446 write 0x7756cc thru 0x776fc6 (0x18fb bytes) 000231[0] 1721217878.758907 mapread 0x1cf178 thru 0x1d0c4b (0x1ad4 bytes) 000232[0] 1721217878.769835 mapread 0x9144b8 thru 0x915d70 (0x18b9 bytes) 000235[0] 1721217878.775248 trunc from 0x98c80a to 0x1c5dac (0x7c6a5d bytes) 000237[1] 1721217878.802494 mapwrite 0x1ca3bb thru 0x1cb963 (0x15a9 bytes) 000239[0] 1721217878.856684 mapwrite 0x421bac thru 0x423984 (0x1dd9 bytes) 000242[0] 1721217878.920060 mapread 0x080b1d thru 0x080d12 (0x1f6 bytes) 000243[0] 1721217878.923313 mapwrite 0x4d876d thru 0x4d8ae7 (0x37b bytes) 000244[0] 1721217878.960331 trunc from 0x4d8ae8 to 0x65bc25 (0x18313e bytes) 000245[0] 1721217878.968749 read 0x1d6731 thru 0x1d67e8 (0xb8 bytes) 000246[0] 1721217878.992896 trunc from 0x65bc26 to 0x6b3a7f (0x57e5a bytes) 000247[1] 1721217878.999475 trunc from 0x6b3a80 to 0x737458 (0x839d9 bytes) 000249[0] 1721217879.021310 mapread 0x5f2789 thru 0x5f322b (0xaa3 bytes) 000250[1] 1721217879.035982 mapread 0x0c2574 thru 0x0c3ca5 (0x1732 bytes) 000254[1] 1721217879.041732 mapread 0x60e92d thru 0x60ef21 (0x5f5 bytes) 000256[0] 1721217879.050561 write 0x2845ef thru 0x284917 (0x329 bytes) 000257[0] 1721217879.072385 trunc from 0x737459 to 0x588f18 (0x1ae540 bytes) 000258[0] 1721217879.086640 trunc from 0x588f19 to 0x231ebb (0x35705d bytes) 000259[0] 1721217879.092870 read 0x1e5d9f thru 0x1e648e (0x6f0 bytes) 000260[0] 1721217879.099293 mapwrite 0x39b74f thru 0x39bebf (0x771 bytes) 000263[1] 1721217879.165785 trunc from 0x39bec0 to 0x0f22b4 (0x2a9c0b bytes) 000264[1] 1721217879.192465 mapwrite 0x601249 thru 0x6029d9 (0x1791 bytes) 000269[0] 1721217879.242069 write 0x6c2231 thru 0x6c2bc9 (0x999 bytes) 000270[0] 1721217879.253868 mapread 0x5a9828 thru 0x5a99ee (0x1c7 bytes) 000271[0] 1721217879.288051 trunc from 0x6c2bca to 0x74b3fd (0x88834 bytes) 000272[1] 1721217879.302923 trunc from 0x74b3fe to 0x55f63a (0x1ebdc3 bytes) 000278[0] 1721217879.349702 write 0x032de5 thru 0x034852 (0x1a6e bytes) 000279[0] 1721217879.364901 trunc from 0x55f63b to 0x4ad19f (0xb249b bytes) 000281[1] 1721217879.386143 write 0x381b37 thru 0x381ee0 (0x3aa bytes) 000282[0] 1721217879.399787 mapread 0x137db2 thru 0x139141 (0x1390 bytes) 000283[1] 1721217879.416861 mapwrite 0x7629dc thru 0x7646e2 (0x1d07 bytes) 000286[1] 1721217879.467841 write 0x1d8c6a thru 0x1d997e (0xd15 bytes) 000289[0] 1721217879.489353 mapwrite 0x08ffad thru 0x090f16 (0xf6a bytes) 000291[1] 1721217879.545654 mapread 0x29d2c0 thru 0x29d3cc (0x10d bytes) 000295[0] 1721217879.578029 read 0x38eb59 thru 0x390988 (0x1e30 bytes) 000297[1] 1721217879.612047 mapread 0x316dfa thru 0x317070 (0x277 bytes) 000298[0] 1721217879.617458 write 0x05ff07 thru 0x061521 (0x161b bytes) 000299[1] 1721217879.638351 trunc from 0x7646e3 to 0x77c6f8 (0x18016 bytes) 000300[0] 1721217879.660631 mapread 0x4837b1 thru 0x484876 (0x10c6 bytes) 000302[1] 1721217879.683735 trunc from 0x77c6f9 to 0x2bf9fa (0x4bccfe bytes) 000305[0] 1721217879.713466 trunc from 0x2bf9fb to 0x04c3e8 (0x273612 bytes) 000306[1] 1721217879.730005 trunc from 0x04c3e9 to 0x0c7a25 (0x7b63d bytes) 000308[0] 1721217879.754346 mapwrite 0x6156dc thru 0x616898 (0x11bd bytes) 000311[1] 1721217879.806002 write 0x76c23b thru 0x76c2c9 (0x8f bytes) 000312[1] 1721217879.816200 read 0x4badfc thru 0x4bcd55 (0x1f5a bytes) 000314[1] 1721217879.854643 write 0x6501b8 thru 0x6509b2 (0x7fb bytes) 000315[1] 1721217879.871608 write 0x9307cb thru 0x9326e8 (0x1f1e bytes) 000316[1] 1721217879.880735 mapread 0x917741 thru 0x9195d0 (0x1e90 bytes) 000317[0] 1721217879.901903 read 0x7a16e7 thru 0x7a298c (0x12a6 bytes) 000318[0] 1721217879.929924 write 0x603edb thru 0x604e00 (0xf26 bytes) 000321[1] 1721217879.961015 mapwrite 0x870543 thru 0x870b47 (0x605 bytes) 000322[0] 1721217880.010117 write 0x96504c thru 0x966472 (0x1427 bytes) 000325[0] 1721217880.038919 mapwrite 0x251708 thru 0x251a62 (0x35b bytes) 000326[0] 1721217880.079014 read 0x0b793a thru 0x0b88a6 (0xf6d bytes) 000328[0] 1721217880.123822 mapread 0x439dfb thru 0x439f11 (0x117 bytes) 000329[0] 1721217880.131726 mapwrite 0x5a2ed2 thru 0x5a3590 (0x6bf bytes) 000331[1] 1721217880.169100 trunc from 0x966473 to 0x578724 (0x3edd4e bytes) 000332[0] 1721217880.198298 trunc from 0x578725 to 0x1894dd (0x3ef247 bytes) 000335[1] 1721217880.232022 mapread 0x09d382 thru 0x09e350 (0xfcf bytes) 000336[0] 1721217880.246147 trunc from 0x1894de to 0x1957c8 (0xc2eb bytes) 000340[0] 1721217880.268286 mapread 0x10cf97 thru 0x10e58f (0x15f9 bytes) 000341[1] 1721217880.275854 mapwrite 0x7f459d thru 0x7f58b7 (0x131b bytes) 000344[0] 1721217880.325697 write 0x4b52ef thru 0x4b683b (0x154d bytes) 000345[1] 1721217880.341375 trunc from 0x7f58b8 to 0x77d145 (0x78772 bytes) 000347[1] 1721217880.368987 mapread 0x692c65 thru 0x693e58 (0x11f4 bytes) 000349[1] 1721217880.388256 mapwrite 0x35934d thru 0x35a062 (0xd16 bytes) 000351[1] 1721217880.434752 mapread 0x285e76 thru 0x285ec8 (0x53 bytes) 000359[1] 1721217880.492880 write 0x808d7f thru 0x8093f2 (0x674 bytes) 000360[0] 1721217880.500753 mapread 0x1a997a thru 0x1a9e30 (0x4b7 bytes) 000361[1] 1721217880.513915 trunc from 0x8093f3 to 0x144f6e (0x6c4484 bytes) 000362[1] 1721217880.534910 mapwrite 0x4d2781 thru 0x4d311f (0x99f bytes) 000364[0] 1721217880.586324 write 0x71af67 thru 0x71c355 (0x13ef bytes) 000365[1] 1721217880.599309 mapwrite 0x8e9fd1 thru 0x8eba83 (0x1ab3 bytes) 000369[0] 1721217880.665958 write 0x1ef633 thru 0x1f0f8b (0x1959 bytes) 000371[1] 1721217880.689631 trunc from 0x8eba84 to 0x1acdca (0x73ecb9 bytes) 000372[0] 1721217880.712598 write 0x019fce thru 0x01a41e (0x451 bytes) 000373[0] 1721217880.730524 mapread 0x19f11b thru 0x1a0665 (0x154b bytes) 000374[1] 1721217880.743079 trunc from 0x1acdcb to 0x513c25 (0x366e5b bytes) 000376[1] 1721217880.777320 write 0x826ba6 thru 0x827b65 (0xfc0 bytes) 000377[0] 1721217880.786498 mapwrite 0x27588b thru 0x276bd0 (0x1346 bytes) 000379[1] 1721217880.846199 mapread 0x31ddc7 thru 0x31de94 (0xce bytes) 000380[1] 1721217880.860560 trunc from 0x827b66 to 0x621ae0 (0x206085 bytes) 000381[0] 1721217880.873798 mapwrite 0x1976eb thru 0x197feb (0x901 bytes) 000382[1] 1721217880.911777 write 0x436514 thru 0x43715f (0xc4c bytes) 000383[1] 1721217880.919243 mapread 0x1d883d thru 0x1d8fff (0x7c3 bytes) 000384[0] 1721217880.926461 mapwrite 0x558911 thru 0x558ca4 (0x394 bytes) 000385[1] 1721217880.962307 write 0x72dc71 thru 0x72f269 (0x15f9 bytes) 000386[1] 1721217880.967589 write 0x7d9b69 thru 0x7daf7c (0x1414 bytes) 000387[0] 1721217880.968563 trunc from 0x7daf7d to 0x53f8a9 (0x29b6d3 bytes) 000389[0] 1721217880.991362 write 0x6c53d1 thru 0x6c6899 (0x14c9 bytes) 000391[0] 1721217881.000463 mapread 0x029fd6 thru 0x02bbf1 (0x1c1c bytes) 000392[0] 1721217881.006533 write 0x0b3857 thru 0x0b42b3 (0xa5d bytes) 000393[0] 1721217881.014884 trunc from 0x6c689a to 0x1ee69a (0x4d81ff bytes) 000397[1] 1721217881.030280 read 0x075aab thru 0x075b02 (0x58 bytes) 000398[0] 1721217881.040356 read 0x05cb09 thru 0x05d85e (0xd56 bytes) 000400[0] 1721217881.050776 mapread 0x09dc0a thru 0x09de93 (0x28a bytes) 000401[0] 1721217881.051128 mapread 0x1dcfbe thru 0x1deafa (0x1b3d bytes) 000404[1] 1721217881.059242 read 0x17cc8f thru 0x17ea09 (0x1d7b bytes) 000407[1] 1721217881.069391 mapread 0x0a9288 thru 0x0aaad5 (0x184e bytes) 000408[1] 1721217881.069706 write 0x3789dd thru 0x37a348 (0x196c bytes) 000409[1] 1721217881.075609 write 0x217d94 thru 0x218cb4 (0xf21 bytes) 000411[0] 1721217881.083631 read 0x287d53 thru 0x28847e (0x72c bytes) 000412[0] 1721217881.091580 mapread 0x2e2369 thru 0x2e3978 (0x1610 bytes) 000413[1] 1721217881.099832 mapread 0x19086f thru 0x191cb9 (0x144b bytes) 000414[1] 1721217881.105748 mapwrite 0x2be3bc thru 0x2bebf4 (0x839 bytes) 000416[0] 1721217881.125275 read 0x015b9a thru 0x016fa0 (0x1407 bytes) 000417[1] 1721217881.134829 mapwrite 0x13bc61 thru 0x13c804 (0xba4 bytes) 000418[0] 1721217881.150035 mapread 0x22ad43 thru 0x22ae3a (0xf8 bytes) 000419[1] 1721217881.153278 mapwrite 0x659af9 thru 0x659da5 (0x2ad bytes) 000421[1] 1721217881.173523 mapread 0x3ff1e9 thru 0x400967 (0x177f bytes) 000422[1] 1721217881.175726 read 0x0f226c thru 0x0f22d4 (0x69 bytes) 000423[1] 1721217881.196747 trunc from 0x659da6 to 0x39480b (0x2c559a bytes) 000425[0] 1721217881.209692 trunc from 0x39480c to 0x029735 (0x36b0d6 bytes) 000426[1] 1721217881.221354 mapread 0x01a2a8 thru 0x01a44e (0x1a7 bytes) 000428[1] 1721217881.233087 mapread 0x00465e thru 0x004b20 (0x4c3 bytes) 000429[1] 1721217881.236592 trunc from 0x029736 to 0x214a91 (0x1eb35c bytes) 000433[1] 1721217881.253016 read 0x183fad thru 0x184d1e (0xd72 bytes) 000435[0] 1721217881.261119 mapwrite 0x092bcd thru 0x094928 (0x1d5c bytes) 000436[1] 1721217881.282782 trunc from 0x214a92 to 0x8bea4e (0x6a9fbd bytes) 000437[1] 1721217881.293470 read 0x109478 thru 0x109781 (0x30a bytes) 000438[1] 1721217881.314829 trunc from 0x8bea4f to 0x1c1876 (0x6fd1d8 bytes) 000439[0] 1721217881.325841 mapread 0x1aedec thru 0x1af13d (0x352 bytes) 000440[0] 1721217881.343052 mapwrite 0x586d9b thru 0x588436 (0x169c bytes) 000441[0] 1721217881.392372 mapwrite 0x68f8a3 thru 0x690a0c (0x116a bytes) 000444[1] 1721217881.416183 trunc from 0x690a0d to 0x42fa0d (0x260fff bytes) 000446[0] 1721217881.431491 write 0x534e66 thru 0x535e15 (0xfb0 bytes) 000449[0] 1721217881.444957 mapread 0x4e0a5a thru 0x4e1848 (0xdef bytes) 000450[0] 1721217881.454611 write 0x63d7e1 thru 0x63edb1 (0x15d1 bytes) 000452[1] 1721217881.460550 mapread 0x193eb3 thru 0x194dbf (0xf0d bytes) 000454[0] 1721217881.468312 read 0x01e74f thru 0x01fd9e (0x1650 bytes) 000455[0] 1721217881.483360 trunc from 0x63edb2 to 0x2c891e (0x376493 bytes) 000457[0] 1721217881.501069 write 0x9f7cc9 thru 0x9f9012 (0x134a bytes) 000458[0] 1721217881.506045 mapread 0x68db03 thru 0x68ea6a (0xf68 bytes) 000459[1] 1721217881.517035 mapwrite 0x683e8b thru 0x685459 (0x15cf bytes) 000460[1] 1721217881.554460 read 0x28214b thru 0x282d26 (0xbdc bytes) 000461[0] 1721217881.563875 mapread 0x107041 thru 0x107229 (0x1e9 bytes) 000464[1] 1721217881.579117 mapread 0x9cc912 thru 0x9cdea5 (0x1594 bytes) 000465[0] 1721217881.608041 write 0x75fc9a thru 0x760d33 (0x109a bytes) 000466[0] 1721217881.618486 trunc from 0x9f9013 to 0x1c66e3 (0x83292f bytes) 000470[0] 1721217881.635863 read 0x15bb5d thru 0x15c72e (0xbd2 bytes) 000471[0] 1721217881.640180 trunc from 0x1c66e4 to 0x84c935 (0x686252 bytes) 000472[1] 1721217881.645177 trunc from 0x84c936 to 0x52b731 (0x321204 bytes) 000473[1] 1721217881.652159 read 0x2d0031 thru 0x2d0c02 (0xbd2 bytes) 000475[1] 1721217881.668765 write 0x7fe9b5 thru 0x8008a6 (0x1ef2 bytes) 000477[1] 1721217881.681520 mapwrite 0x547cb3 thru 0x549272 (0x15c0 bytes) 000478[0] 1721217881.706814 mapwrite 0x047464 thru 0x048332 (0xecf bytes) 000479[0] 1721217881.738881 write 0x1ad4ee thru 0x1ad5f0 (0x103 bytes) 000480[1] 1721217881.773826 read 0x7d0d80 thru 0x7d2701 (0x1982 bytes) 000482[1] 1721217881.791486 write 0x072d75 thru 0x074514 (0x17a0 bytes) 000483[1] 1721217881.806775 trunc from 0x8008a7 to 0x249337 (0x5b756f bytes) 000484[1] 1721217881.820328 mapwrite 0x1511ef thru 0x1524b9 (0x12cb bytes) 000485[1] 1721217881.853228 mapwrite 0x58f3a6 thru 0x590d37 (0x1992 bytes) 000486[0] 1721217881.890471 read 0x4beec6 thru 0x4c00e7 (0x1222 bytes) 000487[0] 1721217881.905959 read 0x17bec7 thru 0x17d393 (0x14cd bytes) 000488[0] 1721217881.934668 mapread 0x391781 thru 0x39208b (0x90b bytes) 000491[0] 1721217881.937823 mapwrite 0x5e8bf0 thru 0x5e98d0 (0xce1 bytes) 000492[1] 1721217881.990499 read 0x213cf8 thru 0x21422f (0x538 bytes) 000493[1] 1721217882.016996 mapread 0x51e631 thru 0x51f908 (0x12d8 bytes) 000494[0] 1721217882.053072 mapwrite 0x9b7436 thru 0x9b8cd6 (0x18a1 bytes) 000496[1] 1721217882.111350 mapwrite 0x096def thru 0x0988f5 (0x1b07 bytes) 000499[1] 1721217882.175791 mapwrite 0x537c63 thru 0x538734 (0xad2 bytes) All operations completed A-OK! PASS 16b (8s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 08:04:43 (1721217883) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (2s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 08:04:45 (1721217885) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.158488 s, 13.2 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.10162 s, 20.6 MB/s ldlm.namespaces.MGC192.168.204.157@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800a9d29800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800af5d3800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800a9d29800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800a9d29800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800af5d3800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.12256 s, 25.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.967677 s, 108 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.511147 s, 205 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.38659 s, 75.6 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.0242 s, 51.8 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.996141 s, 105 MB/s PASS 16d (13s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 08:04:58 (1721217898) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0984155 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0450385 s, 23.3 MB/s PASS 16e (2s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 08:05:00 (1721217900) TIMEOUT PASS 16f (22s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 08:05:22 (1721217922) TIMEOUT PASS 16g (21s) == sanityn test 16i: read after truncate file ============ 08:05:44 (1721217944) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (2s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 08:05:45 (1721217945) fail_loc=0x8000030a PASS 17 (4s) == sanityn test 18: mmap sanity check =========================================================================================== 08:05:49 (1721217949) mmap test1: basic mmap operation (PASS, 0.015284s) mmap test2: MAP_PRIVATE not write back (PASS, 0.010037s) mmap test3: concurrent mmap ops on two nodes (PASS, 2.3191s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.0665s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.046004s) mmap test6: check mmap write/read content on two nodes (PASS, 0.050766s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0226s) PASS 18 (8s) == sanityn test 19: test concurrent uncached read races ========================================================================= 08:05:57 (1721217957) oleg457-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg457-client: oleg457-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 08:05:58 (1721217958) PASS 20 (2s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 08:06:00 (1721217960) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) == sanityn test 23: others should see updated atime while another read============================================================== 08:06:02 (1721217962) now is 1721217963 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6821 new atime is 1721218024 PASS 23 (64s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 08:07:06 (1721218026) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 10240 3759104 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 16384 7522304 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 10240 3759104 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 16384 7522304 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.1K 376 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.0K 368 229.7K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.1K 376 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.0K 368 229.7K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 10.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 16.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117912 376 117536 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235568 368 235200 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 10240 3759104 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 16384 7522304 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.1K 376 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.0K 368 229.7K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 117912 376 117536 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre[OST:1] filesystem_summary: 235568 368 235200 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117912 376 117536 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235568 368 235200 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 10240 3759104 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 16384 7522304 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 10240 3759104 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 16384 7522304 1% /mnt/lustre2 PASS 24a (2s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 08:07:08 (1721218028) PASS 24b (2s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 08:07:10 (1721218030) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (2s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 08:07:12 (1721218032) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 08:07:13 (1721218033) utime: good mknod times 1721218033 <= 1721218034 <= 1721218034 for /mnt/lustre/f26a utime: good mknod times 1721218033 <= 1721218034 <= 1721218034 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (3s) == sanityn test 26b: sync mtime between ost and mds ====== 08:07:17 (1721218037) PASS 26b (4s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 08:07:22 (1721218042) 0 PASS 30 (4s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 08:07:26 (1721218046) fail_loc=0x314 PASS 31a (3s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 08:07:29 (1721218049) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg457-server mds-ost sync done. sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.157@tcp:/lustre 7542784 17408 7521280 1% /mnt/lustre2 PASS 31b (13s) == sanityn test 31r: open-rename(replace) race =========== 08:07:42 (1721218062) -rw-r--r-- 1 root root 0 Jul 17 08:07 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (5s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 08:07:48 (1721218068) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 08:07:49 (1721218069) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 08:07:50 (1721218070) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 08:07:51 (1721218071) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 08:07:52 (1721218072) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg457-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a9d29800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800af5d3800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800a9d29800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8800af5d3800.ost_server_uuid in FULL state after 0 sec oleg457-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800a9d29800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800af5d3800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800a9d29800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800af5d3800.ost_server_uuid in FULL state after 0 sec oleg457-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a9d29800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800af5d3800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800a9d29800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8800af5d3800.ost_server_uuid in IDLE state after 0 sec oleg457-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800a9d29800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800af5d3800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800a9d29800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800af5d3800.ost_server_uuid in FULL state after 0 sec oleg457-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a9d29800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800af5d3800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800a9d29800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8800af5d3800.ost_server_uuid in IDLE state after 0 sec oleg457-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800a9d29800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800af5d3800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800a9d29800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800af5d3800.ost_server_uuid in IDLE state after 0 sec PASS 34 (73s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 08:09:05 (1721218145) fail_loc=0x80000317 Race attempt 0 Wait for 15416 15555 for 60 sec... - open/close 2506 (time 1721218157.82 total 10.00 last 250.58) total: 4000 open/close in 15.69 seconds: 254.88 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.204.157@tcp:/lustre 7.2G 18M 7.2G 1% /mnt/lustre 192.168.204.157@tcp:/lustre 7.2G 18M 7.2G 1% /mnt/lustre2 PASS 35 (65s) == sanityn test 36: handle ESTALE/open-unlink correctly == 08:10:10 (1721218210) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg457-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.486884 s, 108 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6821 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7520256) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.41862 s, 125 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6821 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7520256) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.456354 s, 115 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6821 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7520256) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.367847 s, 143 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6821 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7520256) after_dd(7469056) after(7519232) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.471975 s, 111 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6821 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7519232) after_dd(7468032) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.312985 s, 168 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6821 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7520256) after_dd(7468032) after(7519232) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.420462 s, 125 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6821 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7519232) after_dd(7468032) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.453951 s, 115 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6821 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7520256) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.43549 s, 120 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6821 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7520256) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.420287 s, 125 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6821 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7520256) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.27442 s, 191 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6821 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7520256) after_dd(7468032) after(7519232) PASS 36 (228s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 08:13:58 (1721218438) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6821 - create 5596 (time 1721218450.02 total 10.00 last 559.57) total: 10000 create in 18.05 seconds: 554.01 ops/second PASS 37 (22s) == sanityn test 39a: test from 11063 ============================================================================================ 08:14:20 (1721218460) repeat after cancel_lru_locks PASS 39a (2s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 08:14:22 (1721218462) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 08:14:25 (1721218465) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 08:14:28 (1721218468) fail_loc=0x411 fail_loc=0 PASS 39d (2s) == sanityn test 40a: pdirops: create vs others ======================================================================== 08:14:30 (1721218470) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (9s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 08:14:39 (1721218479) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (8s) == sanityn test 40c: pdirops: link and others ======================================================================== 08:14:47 (1721218487) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (9s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 08:14:56 (1721218496) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (8s) == sanityn test 40e: pdirops: rename and others ======================================================================== 08:15:04 (1721218504) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (8s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 08:15:12 (1721218512) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (5s) == sanityn test 41b: pdirops: create vs create ======================================================================== 08:15:17 (1721218517) open: File exists Conflict PASS 41b (5s) == sanityn test 41c: pdirops: create vs link ======================================================================== 08:15:22 (1721218522) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (6s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 08:15:28 (1721218528) Conflict PASS 41d (5s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 08:15:33 (1721218533) Conflict PASS 41e (5s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 08:15:38 (1721218538) Conflict PASS 41f (6s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 08:15:44 (1721218544) Conflict PASS 41g (5s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 08:15:49 (1721218549) Conflict PASS 41h (5s) == sanityn test 41i: reint_open: create vs create ======== 08:15:55 (1721218555) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (698s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 08:27:33 (1721219253) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (6s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 08:27:39 (1721219259) open: File exists Conflict PASS 42b (5s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 08:27:44 (1721219264) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (5s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 08:27:49 (1721219269) Conflict PASS 42d (6s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 08:27:55 (1721219275) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (5s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 08:28:00 (1721219280) Conflict PASS 42f (6s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 08:28:06 (1721219286) Conflict PASS 42g (5s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 08:28:11 (1721219291) Conflict PASS 42h (5s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 08:28:16 (1721219296) PASS 43a (17s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 08:28:33 (1721219313) Conflict PASS 43b (6s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 08:28:39 (1721219319) Conflict PASS 43c (6s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 08:28:45 (1721219325) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (5s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 08:28:50 (1721219330) Conflict PASS 43e (6s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 08:28:56 (1721219336) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (5s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 08:29:01 (1721219341) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (6s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 08:29:07 (1721219347) Conflict PASS 43h (5s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 08:29:13 (1721219353) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (2s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 08:29:14 (1721219354) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (48s) == sanityn test 43k: unlink vs create ==================== 08:30:02 (1721219402) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (464s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 08:37:46 (1721219866) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Wed Jul 17 08:37:49 EDT 2024 PASS 44a (4s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 08:37:50 (1721219870) open: File exists Conflict PASS 44b (6s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 08:37:56 (1721219876) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (5s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 08:38:01 (1721219881) Conflict PASS 44d (6s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 08:38:07 (1721219887) Conflict PASS 44e (5s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 08:38:12 (1721219892) Conflict PASS 44f (5s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 08:38:17 (1721219897) Conflict PASS 44g (6s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 08:38:23 (1721219903) Conflict PASS 44h (5s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 08:38:28 (1721219908) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 08:38:29 (1721219909) PASS 45a (35s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 08:39:04 (1721219944) Conflict PASS 45b (5s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 08:39:09 (1721219949) Conflict PASS 45c (5s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 08:39:14 (1721219954) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (6s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 08:39:20 (1721219960) Conflict PASS 45e (5s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 08:39:25 (1721219965) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (6s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 08:39:31 (1721219971) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (5s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 08:39:36 (1721219976) Conflict PASS 45h (5s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 08:39:41 (1721219981) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 08:39:42 (1721219982) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (470s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 08:47:33 (1721220453) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (5s) == sanityn test 46b: pdirops: link vs create ======================================================================== 08:47:38 (1721220458) open: File exists Conflict PASS 46b (5s) == sanityn test 46c: pdirops: link vs link ======================================================================== 08:47:43 (1721220463) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (6s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 08:47:49 (1721220469) Conflict PASS 46d (6s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 08:47:55 (1721220475) Conflict PASS 46e (5s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 08:48:00 (1721220480) Conflict PASS 46f (6s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 08:48:06 (1721220486) Conflict PASS 46g (5s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 08:48:11 (1721220491) Conflict PASS 46h (5s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 08:48:16 (1721220496) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (2s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 08:48:18 (1721220498) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 08:48:19 (1721220499) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 08:48:20 (1721220500) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 08:48:21 (1721220501) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 08:48:22 (1721220502) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 08:48:23 (1721220503) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 08:48:24 (1721220504) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 08:48:25 (1721220505) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.00792052 s, 1.3 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (8s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 08:48:33 (1721220513) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00719209 s, 22.1 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3175: kill: (24231) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 08:48:37 (1721220517) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0050222 s, 204 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222075 s, 47.2 MB/s 3145728 PASS 51b (14s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 08:48:51 (1721220531) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00693264 s, 148 kB/s PASS 51c (7s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 08:48:58 (1721220538) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143465 s, 73.1 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (3s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 08:49:01 (1721220541) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2437 0x985 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2437 0x985 0 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 08:49:05 (1721220545) llite.lustre-ffff8800a9d29800.xattr_cache=1 llite.lustre-ffff8800af5d3800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504963 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-17 08:49:06.000000000 -0400 Modify: 2024-07-17 08:49:06.000000000 -0400 Change: 2024-07-17 08:49:06.000000000 -0400 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504966 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-17 08:49:11.000000000 -0400 Modify: 2024-07-17 08:49:12.000000000 -0400 Change: 2024-07-17 08:49:12.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504969 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-17 08:49:17.000000000 -0400 Modify: 2024-07-17 08:49:17.000000000 -0400 Change: 2024-07-17 08:49:17.000000000 -0400 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-17 08:49:17.000000000 -0400 Modify: 2024-07-17 08:49:17.000000000 -0400 Change: 2024-07-17 08:49:17.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-17 08:49:23.000000000 -0400 Modify: 2024-07-17 08:49:23.000000000 -0400 Change: 2024-07-17 08:49:23.000000000 -0400 Birth: - File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504972 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-17 08:49:23.000000000 -0400 Modify: 2024-07-17 08:49:23.000000000 -0400 Change: 2024-07-17 08:49:23.000000000 -0400 Birth: - PASS 54 (25s) == sanityn test 55a: rename vs unlink target dir ========= 08:49:30 (1721220570) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 08:49:37 (1721220577) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (8s) == sanityn test 55c: rename vs unlink orphan target dir == 08:49:45 (1721220585) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (13s) == sanityn test 55d: rename file vs link ================= 08:49:58 (1721220598) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) == sanityn test 60: Verify data_version behaviour ======== 08:50:08 (1721220608) PASS 60 (2s) == sanityn test 70a: cd directory && rm directory ======== 08:50:10 (1721220610) PASS 70a (2s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 08:50:12 (1721220612) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 08:50:14 (1721220614) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00428337 s, 15.3 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (2s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 08:50:16 (1721220616) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.0060964 s, 10.7 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 08:50:17 (1721220617) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 08:50:18 (1721220618) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00639983 s, 10.2 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 08:50:19 (1721220619) llite.lustre-ffff8800a9d29800.xattr_cache=1 llite.lustre-ffff8800af5d3800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 08:50:21 (1721220621) llite.lustre-ffff8800a9d29800.xattr_cache=1 llite.lustre-ffff8800af5d3800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (3s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 08:50:24 (1721220624) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00434316 s, 236 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00369672 s, 277 kB/s 2896: get lock2 2897: get lock1 2896: done 2896: get lock1 2897: done 2897: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 2897: cannot get lock2: Resource deadlock avoided 2897: exit rc=0 fcntl cmd 7 failed: Resource deadlock avoided 2896: cannot get lock1: Resource deadlock avoided 2896: done 2896: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 08:50:30 (1721220630) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0215007 s, 97.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128683 s, 81.5 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3743: 3531 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 08:50:37 (1721220637) opening files: ........ closing files: ........ PASS 76 (21s) == sanityn test 77a: check FIFO NRS policy =============== 08:50:58 (1721220658) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.130648 s, 128 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203353 s, 51.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0232676 s, 45.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0204019 s, 51.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209176 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008162 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0230332 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177253 s, 59.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184576 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011682 s, 89.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495779 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0494344 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.053352 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0439276 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0474081 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0406846 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0485644 s, 21.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800af5d3800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00236261 s, 444 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187606 s, 55.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153365 s, 68.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00163198 s, 643 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0248043 s, 42.3 MB/s , 0.018281 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168004 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170599 s, 61.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0251611 s, 41.7 MB/s 1048576 bytes (1.0 MB) copied, 0.00154389 s, 679 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261821 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00537795 s, 195 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126434 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015785 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277051 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158229 s, 66.3 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg457-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (3s) == sanityn test 77b: check CRR-N NRS policy ============== 08:51:01 (1721220661) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.134142 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00760161 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00941502 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120809 s, 86.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00721982 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00613925 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189295 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187035 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157179 s, 66.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109724 s, 95.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328343 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0455181 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0455729 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343265 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400319 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288531 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332232 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00823542 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00659086 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00391463 s, 268 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0012469 s, 841 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00131907 s, 795 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00108339 s, 968 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00108803 s, 964 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0011612 s, 903 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0140697 s, 74.5 MB/s , 0.006103 s, 172 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0134651 s, 77.9 MB/s , 0.0222982 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130679 s, 80.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00231569 s, 453 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00226278 s, 463 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00259701 s, 404 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=1 removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg457-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.140998 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121201 s, 86.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108126 s, 97.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170092 s, 61.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021801 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231118 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157783 s, 66.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152155 s, 68.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139242 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180023 s, 58.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119669 s, 87.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0383165 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.045697 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0434936 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273553 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0567469 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0376781 s, 27.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800af5d3800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out , 0.00835616 s, 125 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00844953 s, 124 MB/s , 0.00959093 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.007996 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109995 s, 95.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100766 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00945032 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00946019 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100822 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00928029 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00986433 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104339 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153748 s, 68.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109738 s, 95.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106325 s, 98.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109119 s, 96.1 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg457-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (4s) == sanityn test 77c: check ORR NRS policy ================ 08:51:05 (1721220665) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.172366 s, 97.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158889 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189432 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0399836 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0566714 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0598409 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0553894 s, 18.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0594513 s, 17.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0483727 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0653756 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.068226 s, 15.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0714557 s, 14.7 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0575341 s, 18.2 MB/s , 0.0700378 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0638796 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0764869 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0706184 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179591 s, 58.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125821 s, 83.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127599 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167887 s, 62.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168494 s, 62.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242808 s, 43.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0244994 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0167267 s, 62.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182484 s, 57.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182194 s, 57.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0193476 s, 54.2 MB/s , 0.0194901 s, 53.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0204612 s, 51.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207993 s, 50.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0211224 s, 49.6 MB/s , 0.0212586 s, 49.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=2 removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg457-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.118209 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148472 s, 70.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211745 s, 49.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341109 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0384786 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324357 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0369335 s, 28.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0350818 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225428 s, 46.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335168 s, 31.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0293948 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0337452 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260619 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030093 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211825 s, 49.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184154 s, 56.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800af5d3800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203787 s, 51.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113829 s, 92.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00868105 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146471 s, 71.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134661 s, 77.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012837 s, 81.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131057 s, 80.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118593 s, 88.4 MB/s , 0.0119526 s, 87.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133466 s, 78.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0137492 s, 76.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0138186 s, 75.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144716 s, 72.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148371 s, 70.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158743 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161319 s, 65.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162858 s, 64.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=1 removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg457-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.13258 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188103 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191486 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250542 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215537 s, 48.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290033 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345119 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0350529 s, 29.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0184692 s, 56.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0316332 s, 33.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0264315 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217999 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213853 s, 49.0 MB/s 1+0 records in 1+0 records out , 0.0205944 s, 50.9 MB/s 1048576 bytes (1.0 MB) copied, 0.014301 s, 73.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278694 s, 37.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800af5d3800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261854 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131728 s, 79.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0109835 s, 95.5 MB/s , 0.0138005 s, 76.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111919 s, 93.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.012118 s, 86.5 MB/s 1048576 bytes (1.0 MB) copied, 0.016611 s, 63.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142945 s, 73.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125739 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127759 s, 82.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108418 s, 96.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120983 s, 86.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167049 s, 62.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104813 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116061 s, 90.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120813 s, 86.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124196 s, 84.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=1 removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg457-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (6s) == sanityn test 77d: check TRR nrs policy ================ 08:51:11 (1721220671) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.15383 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158647 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198535 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225092 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275104 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0414174 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0385622 s, 27.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361233 s, 29.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02546 s, 41.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0315967 s, 33.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0364104 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0466725 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400726 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0452241 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0368271 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326403 s, 32.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800af5d3800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0320511 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101711 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00847916 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0079579 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010488 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00994137 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00821536 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00935723 s, 112 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00154098 s, 680 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00132146 s, 793 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00542617 s, 193 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00931411 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00183816 s, 570 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00149106 s, 703 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00590833 s, 177 MB/s , 0.00769998 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00520382 s, 202 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=1 removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg457-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.158178 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136897 s, 76.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180979 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400869 s, 26.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0497378 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0459952 s, 22.8 MB/s , 0.0436687 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0493282 s, 21.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.057416 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0591813 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0572569 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0455063 s, 23.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff8800af5d3800.lock_unused_count=6 1048576 bytes (1.0 MB) copied, 0.0539395 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0540259 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.049145 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.05302 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0538489 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00564903 s, 186 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00818707 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00813701 s, 129 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120876 s, 86.7 MB/s , 0.0106502 s, 98.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00833728 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121527 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00146809 s, 714 MB/s 1048576 bytes (1.0 MB) copied, 0.00490639 s, 214 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103247 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0013676 s, 767 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104748 s, 100 MB/s 1048576 bytes (1.0 MB) copied, 0.0123675 s, 84.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0143222 s, 73.2 MB/s 1048576 bytes (1.0 MB) copied, 0.00535801 s, 196 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00356946 s, 294 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=1 removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg457-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.162014 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190736 s, 55.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022717 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317761 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327605 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0410254 s, 25.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331318 s, 31.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0393371 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025467 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307979 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0428139 s, 24.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249883 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281673 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0316145 s, 33.2 MB/s , 0.0330255 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.033297 s, 31.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800af5d3800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243657 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00700391 s, 150 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103677 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00943524 s, 111 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100172 s, 105 MB/s , 0.00801268 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00753244 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00127129 s, 825 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00832965 s, 126 MB/s , 0.0128174 s, 81.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00651851 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00699833 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00552052 s, 190 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00977353 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00803758 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00228884 s, 458 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00143097 s, 733 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg457-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (5s) == sanityn test 77e: check TBF NID nrs policy ============ 08:51:16 (1721220676) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.204.57@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.15622 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111204 s, 94.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137754 s, 76.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236942 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0419821 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0482022 s, 21.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0542834 s, 19.3 MB/s , 0.0403632 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0467928 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451271 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0571145 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0629212 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0528249 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0494651 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.053682 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0479005 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0642788 s, 16.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123531 s, 84.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011752 s, 89.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130861 s, 80.1 MB/s , 0.0182062 s, 57.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0178646 s, 58.7 MB/s , 0.0142735 s, 73.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0152748 s, 68.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154192 s, 68.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164159 s, 63.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162926 s, 64.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0141342 s, 74.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143447 s, 73.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187984 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.017302 s, 60.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0168551 s, 62.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196028 s, 53.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg457-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.1512 s, 111 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146177 s, 71.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0147617 s, 71.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178465 s, 58.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00719465 s, 146 MB/s , 0.0233824 s, 44.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0250073 s, 41.9 MB/s , 0.0297464 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00748525 s, 140 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0398399 s, 26.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0316042 s, 33.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0428948 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226694 s, 46.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.042553 s, 24.6 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0198615 s, 52.8 MB/s , 0.0323508 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239226 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143583 s, 73.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.01318 s, 79.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0117971 s, 88.9 MB/s , 0.0131431 s, 79.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198742 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133367 s, 78.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167482 s, 62.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136318 s, 76.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146547 s, 71.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139738 s, 75.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136829 s, 76.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157699 s, 66.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143932 s, 72.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151583 s, 69.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163111 s, 64.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188594 s, 55.6 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg457-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.188138 s, 89.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255691 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220733 s, 47.5 MB/s 1+0 records in 1+0 records out , 0.0231504 s, 45.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0312284 s, 33.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0159271 s, 65.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027957 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0298091 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131581 s, 79.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02165 s, 48.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285448 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0515896 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022583 s, 46.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0251836 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220297 s, 47.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0526165 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0524137 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101868 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00950224 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103261 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00935249 s, 112 MB/s , 0.00151788 s, 691 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00367008 s, 286 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100495 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101816 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107272 s, 97.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00898561 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00914019 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00369862 s, 284 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134727 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107829 s, 97.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126714 s, 82.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00135195 s, 776 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg457-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.163021 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179705 s, 58.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246399 s, 42.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0409732 s, 25.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0299679 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0459518 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0460074 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0605906 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0639547 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0574434 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0672185 s, 15.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0672661 s, 15.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff8800af5d3800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0579913 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0672091 s, 15.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0614266 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0532867 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0525448 s, 20.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0107579 s, 97.5 MB/s , 0.0115286 s, 91.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153851 s, 68.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116756 s, 89.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.019371 s, 54.1 MB/s , 0.0123547 s, 84.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141325 s, 74.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135367 s, 77.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151536 s, 69.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155391 s, 67.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155717 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171155 s, 61.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172444 s, 60.8 MB/s , 0.0177003 s, 59.2 MB/s , 0.0167702 s, 62.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209984 s, 49.9 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg457-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (9s) == sanityn test 77f: check TBF JobID nrs policy ========== 08:51:25 (1721220685) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg457-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.1855 s, 90.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183231 s, 57.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165602 s, 63.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317594 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0474799 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0532929 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.046461 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0536728 s, 19.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0668951 s, 15.7 MB/s , 0.0522567 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0607618 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0668116 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0517155 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0721614 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0507197 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0538895 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0621 s, 16.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118073 s, 88.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01357 s, 77.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141629 s, 74.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172006 s, 61.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01551 s, 67.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0160786 s, 65.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0162166 s, 64.7 MB/s , 0.0204943 s, 51.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171442 s, 61.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180494 s, 58.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164988 s, 63.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176641 s, 59.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179811 s, 58.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178728 s, 58.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017573 s, 59.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0204127 s, 51.4 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg457-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg457-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.160581 s, 104 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200833 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226545 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228542 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238324 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295124 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183284 s, 57.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135957 s, 77.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0379869 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0481006 s, 21.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0443847 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389831 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258592 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0471405 s, 22.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0402343 s, 26.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0315786 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.031921 s, 32.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105541 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100394 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111883 s, 93.7 MB/s , 0.0110657 s, 94.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136387 s, 76.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.015356 s, 68.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0194239 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00656193 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161135 s, 65.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153155 s, 68.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152264 s, 68.9 MB/s , 0.0143877 s, 72.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016642 s, 63.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159006 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158651 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209972 s, 49.9 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg457-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg457-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.179987 s, 93.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175124 s, 59.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270638 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277539 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280552 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103359 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286957 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0231618 s, 45.3 MB/s 1048576 bytes (1.0 MB) copied, 0.019746 s, 53.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148219 s, 70.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0452655 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.05003 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253179 s, 41.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0369214 s, 28.4 MB/s , 0.0546257 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0424161 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353992 s, 29.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0084725 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010087 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00759932 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00338541 s, 310 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00953911 s, 110 MB/s , 0.00952002 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00367567 s, 285 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00142272 s, 737 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00712019 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00640639 s, 164 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00869557 s, 121 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00325795 s, 322 MB/s , 0.00320939 s, 327 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00369574 s, 284 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00160455 s, 654 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00147345 s, 712 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg457-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg457-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.191589 s, 87.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136541 s, 76.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0131424 s, 79.8 MB/s , 0.0298824 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288274 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0404729 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0359141 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215335 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0312792 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197475 s, 53.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342482 s, 30.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800af5d3800.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0428229 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0406906 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021182 s, 49.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0400413 s, 26.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0447415 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0347366 s, 30.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00879107 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0072498 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115391 s, 90.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121779 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00695195 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00895779 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00929102 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010919 s, 96.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00878199 s, 119 MB/s 1048576 bytes (1.0 MB) copied, 0.00873885 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0012852 s, 816 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0107548 s, 97.5 MB/s , 0.0105802 s, 99.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100094 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115292 s, 90.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129451 s, 81.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800af5d3800.lock_unused_count=2 removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg457-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (8s) == sanityn test 77g: Change TBF type directly ============ 08:51:33 (1721220693) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (4s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 08:51:37 (1721220697) oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg457-client: oleg457-server: ssh exited with exit code 2 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg457-client: oleg457-server: ssh exited with exit code 12 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg457-client: oleg457-server: ssh exited with exit code 12 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg457-client: oleg457-server: ssh exited with exit code 12 PASS 77h (4s) == sanityn test 77i: Change rank of TBF rule ============= 08:51:41 (1721220701) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (8s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 08:51:49 (1721220709) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.150598 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016769 s, 62.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176852 s, 59.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0461585 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0440475 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0488721 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395676 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0475338 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0502916 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0533676 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0637565 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0705978 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0532974 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0523429 s, 20.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0562739 s, 18.6 MB/s , 0.0639521 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0567665 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171384 s, 61.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107277 s, 97.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146663 s, 71.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00919889 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105944 s, 99.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151462 s, 69.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187373 s, 56.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00908093 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133724 s, 78.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142171 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139165 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135741 s, 77.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140609 s, 74.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148858 s, 70.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160496 s, 65.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173605 s, 60.4 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg457-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97145 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4441 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (52s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 08:52:41 (1721220761) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg457-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.66875 s, 25.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.149701 s, 7.0 MB/s 1048576 bytes (1.0 MB) copied, 0.139425 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.162858 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.164867 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.162824 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166939 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.163239 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.164531 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.349379 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.362878 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.3561 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.387161 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.388821 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.389997 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.383937 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.373869 s, 2.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148655 s, 7.1 MB/s 1+0 records in 1+0 records out , 0.152068 s, 6.9 MB/s 1048576 bytes (1.0 MB) copied, 0.149246 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153511 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.153396 s, 6.8 MB/s 1048576 bytes (1.0 MB) copied, 0.154015 s, 6.8 MB/s 1048576 bytes (1.0 MB) copied, 0.152048 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15112 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151738 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151174 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155141 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152554 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15307 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151827 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152845 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156651 s, 6.7 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg457-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.7193 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9313 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg457-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.674593 s, 24.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147065 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152423 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.16119 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151853 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.175488 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.190209 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.179252 s, 5.8 MB/s , 0.174287 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.346543 s, 3.0 MB/s , 0.343787 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.362663 s, 2.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.359727 s, 2.9 MB/s , 0.362377 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.360846 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.385804 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.367427 s, 2.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.120745 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121942 s, 8.6 MB/s , 0.126968 s, 8.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.130499 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.127873 s, 8.2 MB/s , 0.122904 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.128971 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.125551 s, 8.4 MB/s , 0.126485 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.125632 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.127913 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.127027 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.128288 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.127006 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.130306 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.133868 s, 7.8 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg457-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.7225 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9295 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (136s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 08:54:57 (1721220897) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg457-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.21763 s, 77.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174857 s, 60.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0391451 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0567106 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0532245 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0569951 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0552261 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0466856 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495398 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0557074 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0885821 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0946143 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106963 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0956708 s, 11.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.102283 s, 10.3 MB/s 1048576 bytes (1.0 MB) copied, 0.115876 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.114812 s, 9.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166246 s, 63.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018427 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135154 s, 77.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165345 s, 63.4 MB/s , 0.0150606 s, 69.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180657 s, 58.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0079113 s, 133 MB/s 1048576 bytes (1.0 MB) copied, 0.0166573 s, 63.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103749 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162489 s, 64.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166046 s, 63.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0170218 s, 61.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0168357 s, 62.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185748 s, 56.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017562 s, 59.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196508 s, 53.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg457-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97462 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74353 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.57@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.57@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.214725 s, 78.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209571 s, 50.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.050264 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0402165 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.045853 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0521676 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0616771 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0592686 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0613095 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0603145 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0795877 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0703006 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0682721 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146193 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148067 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.234596 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.247439 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113301 s, 92.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113423 s, 92.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121786 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129352 s, 81.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133093 s, 78.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0179871 s, 58.3 MB/s , 0.0202574 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142732 s, 73.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159194 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151036 s, 69.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147905 s, 70.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014708 s, 71.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164477 s, 63.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186964 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166372 s, 63.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190828 s, 54.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg457-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.98519 s, 21.0 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7405 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.57@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg457-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.214158 s, 78.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162233 s, 64.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0422796 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317886 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0301675 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0481535 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0500865 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283778 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0302172 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0554555 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0402289 s, 26.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0741651 s, 14.1 MB/s , 0.0688045 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0860243 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0785519 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0795729 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.084567 s, 12.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175009 s, 59.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145511 s, 72.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014857 s, 70.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148237 s, 70.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178847 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159845 s, 65.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0159401 s, 65.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0170722 s, 61.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171642 s, 61.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171251 s, 61.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168594 s, 62.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173152 s, 60.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189937 s, 55.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182723 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177902 s, 58.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179456 s, 58.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg457-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.99095 s, 21.0 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.89028 s, 21.4 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg457-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.364863 s, 46.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191601 s, 54.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147079 s, 71.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.03843 s, 27.3 MB/s , 0.0498049 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0583744 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0570954 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0543285 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0508822 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151436 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.181152 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.182195 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.168626 s, 6.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.334943 s, 3.1 MB/s , 0.34827 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.346731 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.352342 s, 3.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0149261 s, 70.3 MB/s , 0.015115 s, 69.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198683 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155567 s, 67.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017005 s, 61.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0349347 s, 30.0 MB/s , 0.019413 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200751 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261091 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371068 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0221709 s, 47.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0230714 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219965 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237103 s, 44.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234899 s, 44.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266789 s, 39.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg457-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.82922 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74805 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.98148 s, 21.0 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74047 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg457-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.670041 s, 25.0 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135671 s, 77.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191514 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219783 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171984 s, 61.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291305 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166402 s, 63.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123395 s, 85.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164839 s, 63.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.345323 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.373347 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.358736 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.366104 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.385415 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.388341 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.366965 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.377939 s, 2.8 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0125951 s, 83.3 MB/s 1+0 records in 1+0 records out , 0.0116045 s, 90.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0166475 s, 63.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157088 s, 66.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155607 s, 67.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014773 s, 71.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00219542 s, 478 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158632 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177893 s, 58.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174007 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167681 s, 62.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184377 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205759 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182423 s, 57.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193629 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208953 s, 50.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg457-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.7179 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4425 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg457-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.219821 s, 76.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182645 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286277 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0478899 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0492838 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0513443 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.050898 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0426 s, 24.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.057368 s, 18.3 MB/s , 0.0603075 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.082019 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0717293 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0734827 s, 14.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0972543 s, 10.8 MB/s , 0.0794257 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0932386 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.092912 s, 11.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011621 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185169 s, 56.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138572 s, 75.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141733 s, 74.0 MB/s , 0.0128268 s, 81.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149394 s, 70.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200816 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162386 s, 64.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016794 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179113 s, 58.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186887 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186937 s, 56.1 MB/s , 0.0191846 s, 54.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206859 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203167 s, 51.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200567 s, 52.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg457-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.98618 s, 21.0 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74032 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg457-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.374073 s, 44.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194291 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023523 s, 44.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187445 s, 55.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267496 s, 39.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0309347 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00979464 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159571 s, 65.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00810208 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146453 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.179278 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158099 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157825 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.343491 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.338638 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.336927 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.355783 s, 2.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119306 s, 87.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0267064 s, 39.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153362 s, 68.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0134815 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0314818 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173501 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.032004 s, 32.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0363659 s, 28.8 MB/s , 0.0172951 s, 60.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192885 s, 54.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172057 s, 60.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0181192 s, 57.9 MB/s , 0.0168082 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0244421 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201232 s, 52.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211239 s, 49.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg457-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.82583 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74154 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.81902 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74136 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (361s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 09:00:58 (1721221258) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (4s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 09:01:02 (1721221262) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg457-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.66657 s, 286 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg457-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0202 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (51s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 09:01:53 (1721221313) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.138545 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117287 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182271 s, 57.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209617 s, 50.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008181 s, 128 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00782408 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233354 s, 44.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0243731 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193003 s, 54.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107101 s, 97.9 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800af5d3800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0488061 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446272 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0442124 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0560033 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0351859 s, 29.8 MB/s , 0.0334492 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0330489 s, 31.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013777 s, 76.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106001 s, 98.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126184 s, 83.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00974628 s, 108 MB/s 1048576 bytes (1.0 MB) copied, 0.0112413 s, 93.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0136445 s, 76.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0136108 s, 77.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103022 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104433 s, 100 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0123994 s, 84.6 MB/s , 0.0124486 s, 84.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125829 s, 83.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0231595 s, 45.3 MB/s , 0.0231853 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239722 s, 43.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0241567 s, 43.4 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg457-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.99051 s, 21.0 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93467 s, 21.2 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.216956 s, 77.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136981 s, 76.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.024494 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413993 s, 25.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0451037 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0555151 s, 18.9 MB/s , 0.0532417 s, 19.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0920854 s, 11.4 MB/s , 0.0955001 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.131842 s, 8.0 MB/s , 0.132432 s, 7.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.192161 s, 5.5 MB/s , 0.19663 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.255337 s, 4.1 MB/s , 0.237844 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.334202 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.334164 s, 3.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125425 s, 83.6 MB/s , 0.0173738 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144842 s, 72.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128386 s, 81.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0140954 s, 74.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014986 s, 70.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158633 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015962 s, 65.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0167155 s, 62.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0156086 s, 67.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183898 s, 57.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167329 s, 62.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184049 s, 57.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171542 s, 61.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016238 s, 64.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174857 s, 60.0 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg457-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg457-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.98221 s, 21.0 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.9722 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (69s) == sanityn test 77o: Changing rank should not panic ====== 09:03:02 (1721221382) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (4s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 09:03:06 (1721221386) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (36s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 09:03:42 (1721221422) mds.MDS.mdt.nrs_policies=tbf oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg457-client: oleg457-server: ssh exited with exit code 36 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg457-client: oleg457-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (13s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 09:03:55 (1721221435) oleg457-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device pdsh@oleg457-client: oleg457-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (3s) == sanityn test 79: xattr: intent error ================== 09:03:58 (1721221438) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505431 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-17 09:03:59.000000000 -0400 Modify: 2024-07-17 09:03:59.000000000 -0400 Change: 2024-07-17 09:03:59.000000000 -0400 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (13s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 09:04:11 (1721221451) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (2s) == sanityn test 80b: Accessing directory during migration ========================================================== 09:04:13 (1721221453) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 09:04:14 (1721221454) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 09:04:15 (1721221455) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 09:04:16 (1721221456) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 09:04:17 (1721221457) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6821 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6821 PASS 82 (3s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 09:04:20 (1721221460) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 09:04:21 (1721221461) fail_loc=0x8000060b PASS 84 (8s) == sanityn test 90: open/create and unlink striped directory ========================================================== 09:04:29 (1721221469) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 09:04:30 (1721221470) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 09:04:31 (1721221471) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 09:04:32 (1721221472) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2646 0xa56 0 1 2627 0xa43 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2628 0xa44 0 0 2647 0xa57 0 rc1=0 and rc2=0 PASS 93 (9s) == sanityn test 94: signal vs CP callback race =========== 09:04:41 (1721221481) osc.lustre-OST0000-osc-ffff8800a9d29800.idle_timeout=0 osc.lustre-OST0000-osc-ffff8800af5d3800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800a9d29800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800af5d3800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0597613 s, 137 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 2.99797 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5164: 4218 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0243859 s, 168 kB/s osc.lustre-OST0000-osc-ffff8800a9d29800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800af5d3800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800a9d29800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800af5d3800.idle_timeout=debug PASS 94 (11s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 09:04:52 (1721221492) llite.lustre-ffff8800a9d29800.fast_read=0 llite.lustre-ffff8800af5d3800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00168935 s, 7.3 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800a9d29800.fast_read=1 llite.lustre-ffff8800af5d3800.fast_read=1 PASS 95b (8s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 09:05:00 (1721221500) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 09:05:01 (1721221501) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00269514 s, 1.5 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 09:05:03 (1721221503) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0389806 s, 53.8 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (3s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 09:05:06 (1721221506) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0382383 s, 54.8 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 09:05:08 (1721221508) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00298898 s, 4.0 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 09:05:10 (1721221510) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00082897 s, 4.9 MB/s PASS 101a (2s) == sanityn test 101b: Discard DoM data on rename ========= 09:05:13 (1721221513) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000785237 s, 5.2 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 09:05:15 (1721221515) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000781711 s, 5.2 MB/s PASS 101c (3s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 09:05:19 (1721221519) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1721221520 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1721221522 Finishing test test23 at 1721221522 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 09:05:26 (1721221526) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 09:05:27 (1721221527) fail_loc=0x416 PASS 105 (48s) == sanityn test 106a: Verify the btime via statx() ======= 09:06:15 (1721221575) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 09:06:16 (1721221576) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) == sanityn test 106c: Verify statx attributes mask ======= 09:06:17 (1721221577) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) == sanityn test 107a: Basic grouplock conflict =========== 09:06:19 (1721221579) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0830256 s, 126 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2633 0xa49 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6821 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6821 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 09:06:23 (1721221583) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.090531 s, 116 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2653 0xa5d 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6821 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) == sanityn test 108a: lseek: parallel updates ============ 09:06:31 (1721221591) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00292167 s, 3.4 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.24393 s, 2.0 MB/s PASS 108a (7s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 09:06:38 (1721221598) 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions oleg457-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 5 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions oleg457-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 6 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions oleg457-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 7 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions oleg457-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 14 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions oleg457-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 30 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions oleg457-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 33 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre 192.168.204.157@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.157@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg457-client.virtnet /mnt/lustre (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg457-server' oleg457-server: oleg457-server.virtnet: executing load_modules_local oleg457-server: Loading modules from /home/green/git/lustre-release/lustre oleg457-server: detected 4 online CPUs by sysfs oleg457-server: Force libcfs to create 2 CPU partitions Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre Starting client: oleg457-client.virtnet: -o user_xattr,flock oleg457-server@tcp:/lustre /mnt/lustre2 PASS 109 (461s) == sanityn test 110: do not grant another lock on resend ========================================================== 09:14:19 (1721222059) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-17 09:14:20.000000000 -0400 Modify: 2024-07-17 09:14:20.000000000 -0400 Change: 2024-07-17 09:14:20.000000000 -0400 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg457-client.virtnet oleg457-client.virtnet PASS 110 (37s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 09:14:56 (1721222096) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 09:14:57 (1721222097) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 09:14:58 (1721222098) lustre-OST0000-osc-ffff8800a9de5000 active. lustre-OST0001-osc-ffff8800a9de5000 active. lustre-OST0000-osc-ffff8800a9de5000 active. lustre-OST0001-osc-ffff8800a9de5000 active. MGC192.168.204.157@tcp active. PASS 113 (2s) cleanup: ====================================================== == sanityn test complete, duration 4473 sec ============== 09:15:01 (1721222101) Stopping clients: oleg457-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg457-client.virtnet /mnt/lustre2 opts: