-----============= acceptance-small: sanityn ============----- Fri Jun 12 12:16:02 EDT 2026 oleg231-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Started clients oleg231-client.virtnet: 192.168.202.131@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt) oleg231-client.virtnet: executing check_config_client /mnt/lustre oleg231-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg231-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff97ba450e7800.idle_timeout=debug osc.lustre-OST0000-osc-ffff97ba4541b800.idle_timeout=debug osc.lustre-OST0001-osc-ffff97ba450e7800.idle_timeout=debug osc.lustre-OST0001-osc-ffff97ba4541b800.idle_timeout=debug disable quota as required oleg231-server: oleg231-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160722 s, 65.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f7136] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 12:16:56 (1781281016) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (7s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 12:17:03 (1781281023) -rw-r--r-- 1 root root 0 Jun 12 12:17 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (7s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 12:17:10 (1781281030) -rw-r--r-- 1 root root 0 Jun 12 12:17 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (8s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 12:17:18 (1781281038) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (7s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 12:17:25 (1781281045) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (8s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 12:17:34 (1781281054) total 1 -rwxrwxrwx 1 root root 0 Jun 12 12:17 f2a -rwxrwxrwx 1 root root 0 Jun 12 12:17 f2b total 1 -rwxrwxrwx 1 root root 0 Jun 12 12:17 f2a -rwxrwxrwx 1 root root 0 Jun 12 12:17 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.011154 s, 45.9 kB/s PASS 2e (7s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 12:17:41 (1781281061) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (5s) == sanityn test 2g: check blocks update on sync write ==== 12:17:46 (1781281066) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.299705 s, 7.0 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (9s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 12:17:55 (1781281075) PASS 3 (8s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 12:18:03 (1781281083) PASS 4 (8s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 12:18:11 (1781281091) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (7s) == sanityn test 6: remove of open file on other node ============================================================================ 12:18:18 (1781281098) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (8s) == sanityn test 7: remove of open directory on other node ======================================================================= 12:18:26 (1781281106) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (8s) == sanityn test 8: remove of open special file on other node ==================================================================== 12:18:34 (1781281114) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (11s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 12:18:45 (1781281125) PASS 9a (13s) == sanityn test 9b: append to striped sparse file ======== 12:18:58 (1781281138) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140102 s, 7.5 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.127816 s, 0.0 kB/s Data read (expecting 'foo'): foo PASS 9b (9s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 12:19:07 (1781281147) 1+0 records in 1+0 records out 1 byte copied, 0.00927942 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0149112 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0105862 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00945539 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00478316 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00555362 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00719293 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00463144 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00496731 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00659935 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00449325 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.004158 s, 0.2 kB/s PASS 10a (7s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 12:19:15 (1781281155) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.0141447 s, 217 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0608471 s, 67.3 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.000506622 s, 6.1 MB/s PASS 10b (8s) == sanityn test 11: execution of file opened for write should return error ============================================================== 12:19:23 (1781281163) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7136 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (8s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 12:19:31 (1781281171) warning: '-runtime' deprecated, use '-t runtime' instead - link 360 (time 1781281183.77 total 10.04 last 35.87) - link 663 (time 1781281193.78 total 20.05 last 30.27) - link 982 (time 1781281203.80 total 30.07 last 31.83) - link 1308 (time 1781281213.82 total 40.09 last 32.54) - link 1610 (time 1781281223.84 total 50.11 last 30.14) - link 1899 (time 1781281233.87 total 60.14 last 28.81) - link 2158 (time 1781281243.88 total 70.15 last 25.86) - link 2414 (time 1781281253.93 total 80.20 last 25.49) - link 2653 (time 1781281263.93 total 90.20 last 23.89) total: 2940 link in 100.03 seconds: 29.39 ops/second using seed 3651435150 running for 100 seconds - stat 8926 (time 1781281185 ; total 11 ; last 11) - stat 10000 (time 1781281186 ; total 12 ; last 1) - stat 18526 (time 1781281197 ; total 23 ; last 11) - stat 20000 (time 1781281199 ; total 25 ; last 2) - stat 29936 (time 1781281210 ; total 36 ; last 11) - stat 30000 (time 1781281210 ; total 36 ; last 0) - stat 39668 (time 1781281221 ; total 47 ; last 11) - stat 40000 (time 1781281221 ; total 47 ; last 0) - stat 44886 (time 1781281232 ; total 58 ; last 11) - stat 49333 (time 1781281243 ; total 69 ; last 11) - stat 50000 (time 1781281244 ; total 70 ; last 1) - stat 54124 (time 1781281255 ; total 81 ; last 11) - stat 58242 (time 1781281266 ; total 92 ; last 11) - stat 60000 (time 1781281269 ; total 95 ; last 3) total: 61686 stats in 100 seconds: 616.859985 stats/second - unlinked 0 (time 1781281224 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile2940) error: No such file or directory total: 2939 unlinks in 88 seconds: 33.397728 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (19718) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (19720) - No such process PASS 12 (147s) == sanityn test 13: test directory page revocation ======= 12:21:59 (1781281319) f13 f13 PASS 13 (10s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 12:22:10 (1781281330) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7136 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 20892 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (9s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 12:22:18 (1781281338) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 21466 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (9s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 12:22:27 (1781281347) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 22038 Terminated $DIR1/$tdir/sleep 60 PASS 14b (9s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 12:22:36 (1781281356) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 22615 Terminated $DIR1/$tdir/sleep 60 PASS 14c (8s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 12:22:44 (1781281364) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 23193 Terminated $DIR1/$tdir/sleep 60 PASS 14d (10s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 12:22:54 (1781281374) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7522304 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7522304kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (25s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 12:23:19 (1781281399) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.449863 s, 4.7 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.206842 s, 10.1 MB/s Chance of close/open is 1 in 50 Seed set to 1403 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read 000010[0] 1781281404.753120 mapwrite 0x400432 thru 0x40fc0c (0xf7db bytes) 000011[1] 1781281404.962639 write 0x53e082 thru 0x54a9b2 (0xc931 bytes) 000016[1] 1781281405.077406 write 0x7277b2 thru 0x728c16 (0x1465 bytes) 000018[1] 1781281405.105512 read 0x1b22fd thru 0x1badf3 (0x8af7 bytes) 000023[1] 1781281405.178134 mapwrite 0x6f06e6 thru 0x6f15ca (0xee5 bytes) 000024[0] 1781281405.286414 mapwrite 0x61e2af thru 0x620689 (0x23db bytes) truncating to largest ever: 0x4543df 000025[1] 1781281405.391479 trunc from 0x728c17 to 0x4543de (0x2d4838 bytes) 000026[0] 1781281405.426918 read 0x378cf7 thru 0x385e4d (0xd157 bytes) 000027[1] 1781281405.465783 mapread 0x1b7c90 thru 0x1c37ba (0xbb2b bytes) 000031[1] 1781281405.527123 mapwrite 0x48e9c4 thru 0x491ad7 (0x3114 bytes) 000032[1] 1781281405.616401 trunc from 0x491ad8 to 0x153661 (0x33e476 bytes) 000033[1] 1781281405.633348 mapwrite 0x21acea thru 0x2205a5 (0x58bc bytes) 000035[0] 1781281405.762962 mapwrite 0x6a9dd1 thru 0x6b6465 (0xc695 bytes) 000036[1] 1781281405.970689 write 0x0db71f thru 0x0e50b6 (0x9998 bytes) 000038[1] 1781281406.033216 mapwrite 0x002ca4 thru 0x00a60a (0x7967 bytes) 000039[1] 1781281406.130748 mapread 0x31351d thru 0x320e2e (0xd912 bytes) 000040[1] 1781281406.191676 mapwrite 0x8de0b6 thru 0x8ec312 (0xe25d bytes) 000042[0] 1781281406.347776 read 0x341e4c thru 0x34ab47 (0x8cfc bytes) 000046[1] 1781281406.444665 write 0x185ffc thru 0x189734 (0x3739 bytes) 000047[0] 1781281406.528105 mapread 0x86c424 thru 0x879584 (0xd161 bytes) 000050[0] 1781281406.720708 mapwrite 0x982cb3 thru 0x992085 (0xf3d3 bytes) 000052[1] 1781281406.933850 write 0x77299c thru 0x77c867 (0x9ecc bytes) 000055[1] 1781281407.007811 mapread 0x6bed06 thru 0x6c7ed6 (0x91d1 bytes) truncating to largest ever: 0x5150f3 000059[1] 1781281407.098988 trunc from 0x992086 to 0x5150f2 (0x47cf93 bytes) 000061[0] 1781281407.156385 read 0x40b14a thru 0x415c84 (0xab3b bytes) 000062[1] 1781281407.215889 read 0x4ea175 thru 0x4f37fd (0x9689 bytes) truncating to largest ever: 0x860cd7 000063[0] 1781281407.269664 trunc from 0x5150f3 to 0x860cd6 (0x34bbe4 bytes) 000066[1] 1781281407.458376 write 0x87bd23 thru 0x881adb (0x5db9 bytes) 000068[1] 1781281407.518083 trunc from 0x881adc to 0x69b4ca (0x1e6611 bytes) 000070[0] 1781281407.592836 mapread 0x33c81e thru 0x34a826 (0xe009 bytes) 000072[1] 1781281407.890904 read 0x107874 thru 0x107f6a (0x6f7 bytes) 000073[1] 1781281408.014206 mapread 0x4de19f thru 0x4e2270 (0x40d2 bytes) 000074[1] 1781281408.136049 read 0x2308d2 thru 0x23afe2 (0xa711 bytes) 000075[1] 1781281408.147915 read 0x4176ee thru 0x41e95b (0x726e bytes) 000076[1] 1781281408.178978 write 0x47cb36 thru 0x481bd7 (0x50a2 bytes) 000077[0] 1781281408.218706 trunc from 0x69b4cb to 0x2f2788 (0x3a8d42 bytes) 000080[1] 1781281408.320302 mapwrite 0x94a63a thru 0x94b103 (0xaca bytes) 000081[1] 1781281408.480089 mapread 0x688b49 thru 0x694b9d (0xc055 bytes) 000082[1] 1781281408.576462 mapread 0x5a12aa thru 0x5a5372 (0x40c9 bytes) 000083[1] 1781281408.599434 mapwrite 0x99f2fd thru 0x9aa9e8 (0xb6ec bytes) 000086[0] 1781281408.923794 mapread 0x2fc3cd thru 0x304bc2 (0x87f6 bytes) 000088[1] 1781281409.147125 write 0x7bc4ea thru 0x7bd3e2 (0xef9 bytes) 000089[1] 1781281409.197398 mapread 0x1a1489 thru 0x1aab04 (0x967c bytes) 000090[0] 1781281409.310031 trunc from 0x9aa9e9 to 0x58084e (0x42a19a bytes) 000092[0] 1781281409.431181 mapwrite 0x464c10 thru 0x4695de (0x49cf bytes) 000093[0] 1781281409.552165 write 0x710bcf thru 0x715ab3 (0x4ee5 bytes) 000094[1] 1781281409.602087 mapwrite 0x71c1af thru 0x724d27 (0x8b79 bytes) 000096[0] 1781281410.062025 mapread 0x1f6611 thru 0x1fbc35 (0x5625 bytes) 000097[1] 1781281410.179071 mapwrite 0x2603d4 thru 0x26de57 (0xda84 bytes) 000099[1] 1781281410.395226 write 0x5ef2a9 thru 0x5f7f3c (0x8c94 bytes) 000100[1] 1781281410.423378 mapread 0x38232d thru 0x38b25c (0x8f30 bytes) 000101[1] 1781281410.484271 trunc from 0x724d28 to 0x84a1c7 (0x1254a0 bytes) 000102[0] 1781281410.525627 mapwrite 0x444953 thru 0x45398b (0xf039 bytes) 000103[1] 1781281410.693387 mapwrite 0x31fe9e thru 0x32eaa1 (0xec04 bytes) 000105[0] 1781281410.963389 mapread 0x4bf228 thru 0x4c1131 (0x1f0a bytes) 000107[1] 1781281411.048840 read 0x042213 thru 0x050e7a (0xec68 bytes) 000110[0] 1781281411.262299 write 0x524518 thru 0x529bfe (0x56e7 bytes) 000111[0] 1781281411.378314 trunc from 0x84a1c8 to 0x2fb7aa (0x54ea1d bytes) 000114[1] 1781281411.527489 mapread 0x1f78bd thru 0x1fd576 (0x5cba bytes) 000115[0] 1781281411.638131 mapread 0x2cc66f thru 0x2d1af4 (0x5486 bytes) 000117[0] 1781281411.697712 mapread 0x2bd48c thru 0x2c8110 (0xac85 bytes) truncating to largest ever: 0x9312da 000119[1] 1781281411.781373 trunc from 0x2fb7ab to 0x9312d9 (0x635b2f bytes) 000120[1] 1781281411.866101 read 0x8b9a4e thru 0x8c50ed (0xb6a0 bytes) 000123[0] 1781281412.007385 read 0x693321 thru 0x69d57f (0xa25f bytes) 000124[0] 1781281412.067313 read 0x0c7c4c thru 0x0ca45c (0x2811 bytes) 000128[0] 1781281412.093666 read 0x8f61be thru 0x90371a (0xd55d bytes) 000132[1] 1781281412.186423 write 0x1d1837 thru 0x1d1f0c (0x6d6 bytes) 000133[1] 1781281412.267248 mapwrite 0x2c7b13 thru 0x2d30a0 (0xb58e bytes) 000134[0] 1781281412.493890 write 0x2c8849 thru 0x2d7acd (0xf285 bytes) 000136[1] 1781281412.580199 read 0x1d5182 thru 0x1e4df3 (0xfc72 bytes) 000138[0] 1781281412.632647 mapwrite 0x658be4 thru 0x65961a (0xa37 bytes) 000141[1] 1781281412.886510 write 0x94bb9c thru 0x9573a6 (0xb80b bytes) 000142[1] 1781281412.900739 mapwrite 0x174309 thru 0x183df8 (0xfaf0 bytes) 000143[1] 1781281413.217270 read 0x2c70c5 thru 0x2c7e8c (0xdc8 bytes) 000144[1] 1781281413.402243 read 0x8fee41 thru 0x9023c1 (0x3581 bytes) 000145[1] 1781281413.426644 write 0x036d9d thru 0x03a327 (0x358b bytes) 000146[1] 1781281413.498760 trunc from 0x9573a7 to 0x813fff (0x1433a7 bytes) 000147[1] 1781281413.554218 trunc from 0x814000 to 0x2b7af4 (0x55c50b bytes) 000148[0] 1781281413.601438 read 0x17eb55 thru 0x17febe (0x136a bytes) 000149[1] 1781281413.639361 write 0x3f3b53 thru 0x4033d2 (0xf880 bytes) 000151[0] 1781281413.707053 trunc from 0x4033d3 to 0x1c99af (0x239a23 bytes) 000152[0] 1781281413.805255 read 0x111d71 thru 0x11d892 (0xbb22 bytes) 000154[1] 1781281413.858151 trunc from 0x1c99b0 to 0x59f483 (0x3d5ad4 bytes) 000155[1] 1781281413.968607 mapread 0x0650c7 thru 0x06d7d4 (0x870e bytes) 000158[1] 1781281414.101989 trunc from 0x59f484 to 0x336ccd (0x2687b6 bytes) 000162[0] 1781281414.197129 trunc from 0x336cce to 0x52078a (0x1e9abd bytes) 000163[1] 1781281414.286415 write 0x848b3e thru 0x84fa3e (0x6f01 bytes) 000166[1] 1781281414.377226 trunc from 0x84fa3f to 0x252194 (0x5fd8aa bytes) 000167[0] 1781281414.420196 mapread 0x231f57 thru 0x233619 (0x16c3 bytes) 000169[1] 1781281414.525226 write 0x545f1e thru 0x54d682 (0x7765 bytes) 000172[1] 1781281414.584156 read 0x3ce486 thru 0x3da468 (0xbfe3 bytes) 000173[0] 1781281414.657054 read 0x47e49b thru 0x48c4ae (0xe014 bytes) 000175[0] 1781281414.788781 trunc from 0x54d683 to 0x765f54 (0x2188d2 bytes) 000177[0] 1781281414.928494 trunc from 0x765f55 to 0x822765 (0xbc811 bytes) 000178[0] 1781281414.973338 write 0x859ac9 thru 0x860e3e (0x7376 bytes) 000185[0] 1781281415.142817 mapread 0x057f9c thru 0x05e31a (0x637f bytes) 000186[1] 1781281415.352916 trunc from 0x860e3f to 0x82bc8b (0x351b3 bytes) 000189[1] 1781281415.609586 read 0x50a529 thru 0x50bd18 (0x17f0 bytes) truncating to largest ever: 0x9bcf48 000190[1] 1781281415.749545 trunc from 0x82bc8c to 0x9bcf47 (0x1912bc bytes) 000192[1] 1781281415.793778 mapread 0x5b3f4f thru 0x5b6528 (0x25da bytes) 000193[0] 1781281415.851002 mapwrite 0x52a747 thru 0x5309b6 (0x6270 bytes) 000195[0] 1781281416.090451 mapwrite 0x8e161a thru 0x8e57a3 (0x418a bytes) 000196[0] 1781281416.196414 read 0x2e628b thru 0x2f28c4 (0xc63a bytes) 000197[1] 1781281416.299376 trunc from 0x9bcf48 to 0x4a0284 (0x51ccc3 bytes) 000198[0] 1781281416.425684 trunc from 0x4a0285 to 0x8dafbc (0x43ad38 bytes) 000201[0] 1781281416.546443 mapread 0x353f8a thru 0x36069b (0xc712 bytes) 000202[0] 1781281416.653800 read 0x049321 thru 0x0547a2 (0xb482 bytes) 000205[1] 1781281416.865907 mapread 0x65dabe thru 0x667c94 (0xa1d7 bytes) 000206[0] 1781281416.994099 mapread 0x2848e9 thru 0x287ca7 (0x33bf bytes) 000208[0] 1781281417.043324 write 0x91df5c thru 0x92c673 (0xe718 bytes) 000212[1] 1781281417.085300 mapwrite 0x616803 thru 0x61780c (0x100a bytes) 000213[0] 1781281417.208014 read 0x4fa40d thru 0x504fd8 (0xabcc bytes) 000214[1] 1781281417.297195 write 0x197600 thru 0x19ea12 (0x7413 bytes) 000215[0] 1781281417.340822 mapwrite 0x0c4271 thru 0x0d194e (0xd6de bytes) 000217[0] 1781281417.667774 trunc from 0x92c674 to 0x2db219 (0x65145a bytes) 000218[1] 1781281417.756229 read 0x215b5a thru 0x21dc77 (0x811e bytes) 000220[1] 1781281417.823876 trunc from 0x2db21a to 0x2bbcd9 (0x1f540 bytes) 000221[0] 1781281417.887304 trunc from 0x2bbcda to 0x97435c (0x6b8683 bytes) 000224[1] 1781281417.973311 write 0x4b8845 thru 0x4ba32e (0x1aea bytes) 000226[1] 1781281418.033427 read 0x3b816c thru 0x3c21ac (0xa041 bytes) 000227[1] 1781281418.153190 read 0x8bf2f5 thru 0x8cbcc8 (0xc9d4 bytes) 000228[0] 1781281418.173710 mapwrite 0x762d04 thru 0x76e570 (0xb86d bytes) 000229[1] 1781281418.387629 mapread 0x527976 thru 0x52d585 (0x5c10 bytes) 000230[0] 1781281418.453785 trunc from 0x97435d to 0x2bd9c0 (0x6b699c bytes) 000231[0] 1781281418.553064 mapwrite 0x109a41 thru 0x111d80 (0x8340 bytes) 000235[1] 1781281418.693732 write 0x6c5631 thru 0x6c714b (0x1b1b bytes) 000244[0] 1781281418.857201 write 0x141db2 thru 0x15141b (0xf66a bytes) 000247[1] 1781281418.920313 write 0x2cc2a8 thru 0x2d8cd0 (0xca29 bytes) 000249[0] 1781281418.960977 trunc from 0x6c714c to 0x8ffbac (0x238a61 bytes) 000256[1] 1781281419.120577 trunc from 0x8ffbad to 0x1cf95a (0x730252 bytes) 000259[0] 1781281419.240252 read 0x03270a thru 0x0344bb (0x1db2 bytes) 000262[0] 1781281419.348265 read 0x010886 thru 0x010d05 (0x480 bytes) 000266[1] 1781281419.437760 mapread 0x1970ee thru 0x1986ac (0x15bf bytes) 000267[0] 1781281419.487344 mapread 0x0aa6ab thru 0x0b4946 (0xa29c bytes) 000268[0] 1781281419.506878 trunc from 0x1cf95b to 0x46f30e (0x29f9b4 bytes) 000270[0] 1781281419.654729 mapwrite 0x1cf92d thru 0x1d877c (0x8e50 bytes) 000271[0] 1781281419.883756 mapwrite 0x54822b thru 0x54d8d5 (0x56ab bytes) 000272[0] 1781281420.097924 mapwrite 0x3d551a thru 0x3e5286 (0xfd6d bytes) 000273[0] 1781281420.257009 mapwrite 0x160999 thru 0x1637e2 (0x2e4a bytes) 000275[1] 1781281420.429310 mapread 0x20ce81 thru 0x2149b6 (0x7b36 bytes) 000276[0] 1781281420.607633 write 0x1458c4 thru 0x14fcee (0xa42b bytes) 000277[0] 1781281420.671806 mapwrite 0x98ac6a thru 0x98ae47 (0x1de bytes) 000278[1] 1781281420.881390 mapread 0x48e548 thru 0x48fe08 (0x18c1 bytes) 000279[0] 1781281420.956842 mapwrite 0x987715 thru 0x98a74f (0x303b bytes) 000281[1] 1781281421.022951 write 0x25be3c thru 0x261c68 (0x5e2d bytes) 000283[1] 1781281421.081866 write 0x7e6cfb thru 0x7e97c7 (0x2acd bytes) 000286[1] 1781281421.182961 mapread 0x786b3a thru 0x790d21 (0xa1e8 bytes) 000289[1] 1781281421.501795 trunc from 0x98ae48 to 0x5fa48f (0x3909b8 bytes) 000290[0] 1781281421.538796 mapread 0x50bd30 thru 0x50bec5 (0x196 bytes) 000294[1] 1781281421.674751 trunc from 0x5fa490 to 0x47d32e (0x17d161 bytes) 000295[1] 1781281421.733425 read 0x379c69 thru 0x3874e3 (0xd87b bytes) 000298[1] 1781281421.954055 read 0x1a83d6 thru 0x1aee64 (0x6a8f bytes) 000301[0] 1781281422.066338 read 0x1d0b53 thru 0x1d7557 (0x6a05 bytes) 000303[1] 1781281422.160807 trunc from 0x47d32f to 0x1404e0 (0x33ce4e bytes) 000304[0] 1781281422.272109 mapwrite 0x01dfa6 thru 0x0231dd (0x5238 bytes) 000305[1] 1781281422.472654 mapwrite 0x4e49aa thru 0x4eb31b (0x6972 bytes) 000307[1] 1781281422.671436 mapread 0x256f29 thru 0x259120 (0x21f8 bytes) 000308[0] 1781281422.795154 read 0x354996 thru 0x3610e2 (0xc74d bytes) 000309[1] 1781281422.892283 trunc from 0x4eb31c to 0x49c6a3 (0x4ec78 bytes) 000310[0] 1781281422.979713 mapread 0x3ce7f2 thru 0x3dbac5 (0xd2d4 bytes) 000314[0] 1781281423.211853 trunc from 0x49c6a4 to 0x0d01f4 (0x3cc4af bytes) 000315[1] 1781281423.311908 mapwrite 0x4f5a7b thru 0x504fcd (0xf553 bytes) 000316[1] 1781281423.546173 trunc from 0x504fce to 0x213529 (0x2f1aa4 bytes) 000318[0] 1781281423.574692 write 0x48a547 thru 0x4957d7 (0xb291 bytes) 000321[0] 1781281423.667846 trunc from 0x4957d8 to 0x5aa1e5 (0x114a0e bytes) 000322[0] 1781281423.732553 write 0x5667d2 thru 0x568eb9 (0x26e8 bytes) 000325[0] 1781281423.891362 write 0x8ac830 thru 0x8bc360 (0xfb31 bytes) 000326[1] 1781281423.920230 write 0x5ca128 thru 0x5d39d5 (0x98ae bytes) 000327[1] 1781281423.960701 read 0x1685ed thru 0x1707f3 (0x8207 bytes) 000328[1] 1781281424.068831 write 0x50b978 thru 0x518dab (0xd434 bytes) 000329[0] 1781281424.127053 trunc from 0x8bc361 to 0x86bd0c (0x50654 bytes) 000330[0] 1781281424.300517 read 0x6b781b thru 0x6bab86 (0x336c bytes) 000331[1] 1781281424.431664 trunc from 0x86bd0d to 0x0783af (0x7f395d bytes) 000333[1] 1781281424.583513 write 0x6ec277 thru 0x6f29d8 (0x6762 bytes) 000335[0] 1781281424.648315 trunc from 0x6f29d9 to 0x8c23d2 (0x1cf9fa bytes) 000336[0] 1781281424.738350 trunc from 0x8c23d3 to 0x533e15 (0x38e5bd bytes) 000337[0] 1781281424.772754 write 0x43d086 thru 0x43f2ed (0x2268 bytes) 000340[0] 1781281424.815112 read 0x4f7ac9 thru 0x4fa709 (0x2c41 bytes) 000341[0] 1781281424.834177 read 0x42539d thru 0x431641 (0xc2a5 bytes) 000343[1] 1781281424.900865 write 0x550f80 thru 0x560c17 (0xfc98 bytes) 000344[0] 1781281424.933333 trunc from 0x560c18 to 0x211961 (0x34f2b6 bytes) 000348[1] 1781281425.066808 mapread 0x014a26 thru 0x020fb3 (0xc58e bytes) 000352[0] 1781281425.218624 read 0x1611d2 thru 0x16d5f6 (0xc425 bytes) 000354[1] 1781281425.257776 mapwrite 0x5a1655 thru 0x5a2d70 (0x171c bytes) 000355[0] 1781281425.455458 read 0x04b09d thru 0x0513d4 (0x6338 bytes) 000356[0] 1781281425.496788 trunc from 0x5a2d71 to 0x05d0d5 (0x545c9b bytes) 000357[1] 1781281425.536041 write 0x187960 thru 0x192d99 (0xb43a bytes) 000358[1] 1781281425.566663 read 0x08f48f thru 0x091527 (0x2099 bytes) 000359[0] 1781281425.596256 read 0x00e651 thru 0x019fcb (0xb97b bytes) 000361[1] 1781281425.652174 read 0x175e03 thru 0x1828e0 (0xcade bytes) 000369[0] 1781281425.683398 read 0x17e2fe thru 0x186b68 (0x886b bytes) 000370[0] 1781281425.753299 read 0x0d773a thru 0x0d89c2 (0x1289 bytes) 000371[0] 1781281425.773121 mapwrite 0x5088c3 thru 0x510afd (0x823b bytes) 000374[0] 1781281426.012097 read 0x46691b thru 0x46cdf2 (0x64d8 bytes) 000377[0] 1781281426.051320 read 0x3108f7 thru 0x315d97 (0x54a1 bytes) 000379[0] 1781281426.072378 read 0x0f405d thru 0x103eb4 (0xfe58 bytes) 000381[1] 1781281426.142245 mapread 0x2e3082 thru 0x2e6a8e (0x3a0d bytes) 000382[1] 1781281426.204513 write 0x054a91 thru 0x05e047 (0x95b7 bytes) 000385[1] 1781281426.289230 trunc from 0x510afe to 0x332b86 (0x1ddf77 bytes) 000386[0] 1781281426.322222 mapwrite 0x6a0762 thru 0x6ace29 (0xc6c8 bytes) 000387[0] 1781281426.477337 mapwrite 0x55950b thru 0x562f52 (0x9a48 bytes) 000389[1] 1781281426.662406 trunc from 0x6ace2a to 0x6d3e51 (0x27028 bytes) 000392[1] 1781281426.802713 read 0x47ac02 thru 0x483b4c (0x8f4b bytes) 000393[1] 1781281426.889024 mapread 0x472533 thru 0x4731c9 (0xc97 bytes) 000394[1] 1781281426.903192 trunc from 0x6d3e52 to 0x35c2f9 (0x377b58 bytes) 000396[0] 1781281426.988093 mapwrite 0x6383c5 thru 0x63adc2 (0x29fe bytes) 000397[0] 1781281427.081087 mapread 0x054cea thru 0x061433 (0xc74a bytes) 000398[1] 1781281427.142397 mapwrite 0x9a2b24 thru 0x9a3f42 (0x141f bytes) 000399[0] 1781281427.324271 mapwrite 0x2768c3 thru 0x285ae7 (0xf225 bytes) 000401[0] 1781281427.514258 mapread 0x344c72 thru 0x348253 (0x35e2 bytes) 000402[0] 1781281427.588615 mapread 0x069939 thru 0x070cb6 (0x737e bytes) 000403[1] 1781281427.637648 write 0x93638d thru 0x944aec (0xe760 bytes) 000404[0] 1781281427.671910 trunc from 0x9a3f43 to 0x308e20 (0x69b122 bytes) 000405[1] 1781281427.730077 write 0x4dd289 thru 0x4e0e9e (0x3c16 bytes) 000406[1] 1781281427.767056 read 0x1292aa thru 0x1294b2 (0x209 bytes) 000409[1] 1781281427.952390 trunc from 0x4e0e9f to 0x9ae9c2 (0x4cdb24 bytes) 000410[1] 1781281427.993029 mapread 0x98541a thru 0x994c2a (0xf811 bytes) 000411[0] 1781281428.103673 mapread 0x93bd36 thru 0x942010 (0x62db bytes) 000412[0] 1781281428.244006 mapwrite 0x22c4cd thru 0x22cb4b (0x67f bytes) 000413[0] 1781281428.484182 mapwrite 0x3fa39a thru 0x3fbe97 (0x1afe bytes) 000414[0] 1781281428.687129 mapwrite 0x6e843b thru 0x6ee570 (0x6136 bytes) 000415[0] 1781281428.796841 trunc from 0x9ae9c3 to 0x698f0c (0x315ab6 bytes) 000416[1] 1781281428.827457 mapread 0x181830 thru 0x1900f3 (0xe8c4 bytes) 000418[1] 1781281428.958501 trunc from 0x698f0d to 0x1431ba (0x555d52 bytes) 000419[0] 1781281429.035060 read 0x0eda16 thru 0x0fd92b (0xff16 bytes) 000421[0] 1781281429.096178 mapread 0x09f06d thru 0x0a079f (0x1733 bytes) 000422[1] 1781281429.131770 mapread 0x08bf67 thru 0x097328 (0xb3c2 bytes) 000423[0] 1781281429.262832 read 0x09a54c thru 0x0a28ce (0x8383 bytes) 000424[1] 1781281429.283934 read 0x0f72b1 thru 0x0fb0e6 (0x3e36 bytes) 000425[0] 1781281429.334349 mapread 0x114a96 thru 0x11820f (0x377a bytes) 000426[0] 1781281429.373167 write 0x997d62 thru 0x998f26 (0x11c5 bytes) 000428[0] 1781281429.398311 trunc from 0x998f27 to 0x983344 (0x15be2 bytes) 000430[0] 1781281429.477179 mapwrite 0x5592a2 thru 0x55ca54 (0x37b3 bytes) 000435[1] 1781281429.631131 mapwrite 0x3d1c0e thru 0x3e09d1 (0xedc4 bytes) 000439[1] 1781281429.864753 mapwrite 0x959237 thru 0x9684ef (0xf2b9 bytes) 000441[1] 1781281430.043003 mapread 0x8ad335 thru 0x8b256a (0x5236 bytes) 000443[0] 1781281430.194248 read 0x40891d thru 0x41333e (0xaa22 bytes) 000445[1] 1781281430.236582 mapwrite 0x8a6409 thru 0x8aa592 (0x418a bytes) 000448[0] 1781281430.395093 mapread 0x1bcdc3 thru 0x1c7b64 (0xada2 bytes) 000449[1] 1781281430.526717 write 0x68b7da thru 0x69505c (0x9883 bytes) 000451[0] 1781281430.559118 mapwrite 0x8ad3d6 thru 0x8b9ab1 (0xc6dc bytes) 000452[1] 1781281430.885803 mapwrite 0x9a8380 thru 0x9a85fb (0x27c bytes) 000453[0] 1781281431.162204 mapwrite 0x28d5ec thru 0x295e81 (0x8896 bytes) 000457[1] 1781281431.466844 read 0x43a0af thru 0x445d5f (0xbcb1 bytes) 000459[0] 1781281431.549305 mapread 0x6eb91e thru 0x6ec9ff (0x10e2 bytes) 000461[0] 1781281431.629014 trunc from 0x9a85fc to 0x3c79dd (0x5e0c1e bytes) 000462[0] 1781281431.694283 trunc from 0x3c79de to 0x0d4f31 (0x2f2aac bytes) 000465[0] 1781281431.741516 trunc from 0x0d4f32 to 0x4c29b6 (0x3eda85 bytes) 000472[1] 1781281431.818638 read 0x13f9f2 thru 0x1462ce (0x68dd bytes) 000473[0] 1781281431.849789 mapwrite 0x50bf6e thru 0x50f030 (0x30c3 bytes) 000474[1] 1781281431.975303 mapread 0x09b300 thru 0x0aa5fa (0xf2fb bytes) 000480[1] 1781281432.138708 read 0x3c1916 thru 0x3c9aac (0x8197 bytes) 000481[0] 1781281432.191768 mapwrite 0x3f34b7 thru 0x3f3c04 (0x74e bytes) 000484[1] 1781281432.275752 read 0x402606 thru 0x404e7e (0x2879 bytes) 000486[0] 1781281432.276415 mapwrite 0x8910e8 thru 0x8a0172 (0xf08b bytes) 000488[0] 1781281432.567385 mapwrite 0x3ebaf7 thru 0x3f5bde (0xa0e8 bytes) 000491[1] 1781281432.901313 trunc from 0x8a0173 to 0x8ce003 (0x2de91 bytes) 000494[1] 1781281433.039618 read 0x846440 thru 0x853e1a (0xd9db bytes) 000495[1] 1781281433.104057 mapread 0x58298d thru 0x5914b7 (0xeb2b bytes) 000496[1] 1781281433.236150 read 0x2991d4 thru 0x2a7077 (0xdea4 bytes) 000499[0] 1781281433.383086 read 0x3052af thru 0x310b44 (0xb896 bytes) 000500[1] 1781281433.446189 write 0x047ff0 thru 0x04efa2 (0x6fb3 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 1433 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000004[0] 1781281434.402803 mapwrite 0x62b000 thru 0x634148 (0x9149 bytes) 000006[0] 1781281434.649670 write 0x0e3000 thru 0x0e7e23 (0x4e24 bytes) 000012[0] 1781281434.718633 mapread 0x628000 thru 0x62fe36 (0x7e37 bytes) 000014[0] 1781281434.746596 read 0x101000 thru 0x105b65 (0x4b66 bytes) 000015[0] 1781281434.775744 mapwrite 0x90f000 thru 0x912782 (0x3783 bytes) 000016[1] 1781281434.915180 mapwrite 0x613000 thru 0x61461b (0x161c bytes) 000017[1] 1781281435.163052 mapwrite 0x1cf000 thru 0x1d05e7 (0x15e8 bytes) truncating to largest ever: 0x331cd3 000021[1] 1781281435.350488 trunc from 0x912783 to 0x331cd2 (0x5e0ab0 bytes) 000023[0] 1781281435.405329 mapread 0x175000 thru 0x1838eb (0xe8ec bytes) 000026[0] 1781281435.539993 mapwrite 0x007000 thru 0x00730d (0x30e bytes) 000029[0] 1781281435.686328 read 0x183000 thru 0x185ad0 (0x2ad1 bytes) truncating to largest ever: 0x8c7953 000030[0] 1781281435.723096 trunc from 0x331cd3 to 0x8c7952 (0x595c80 bytes) 000032[0] 1781281435.777636 read 0x296000 thru 0x2a3921 (0xd922 bytes) 000034[1] 1781281435.855055 mapwrite 0x3d3000 thru 0x3d5a97 (0x2a98 bytes) 000036[0] 1781281436.020195 trunc from 0x8c7953 to 0x733fb4 (0x19399e bytes) 000038[0] 1781281436.068840 write 0x546000 thru 0x551c9a (0xbc9b bytes) 000041[0] 1781281436.107327 mapread 0x642000 thru 0x643f67 (0x1f68 bytes) 000043[0] 1781281436.118902 mapread 0x643000 thru 0x6448c1 (0x18c2 bytes) 000044[0] 1781281436.137428 read 0x045000 thru 0x0485b8 (0x35b9 bytes) 000045[0] 1781281436.247227 read 0x403000 thru 0x405cae (0x2caf bytes) 000046[1] 1781281436.273608 write_OD 0x252000 thru 0x252fff (0x1000 bytes) 000047[0] 1781281436.354479 write 0x188000 thru 0x18889d (0x89e bytes) 000051[0] 1781281436.419950 write 0x42c000 thru 0x43863a (0xc63b bytes) 000052[1] 1781281436.426184 mapread 0x1d7000 thru 0x1d7b2a (0xb2b bytes) 000053[1] 1781281436.464510 read_OD 0x47e000 thru 0x484fff (0x7000 bytes) 000054[0] 1781281436.472679 mapread 0x168000 thru 0x174525 (0xc526 bytes) 000055[1] 1781281436.537686 read_OD 0x477000 thru 0x478fff (0x2000 bytes) 000056[1] 1781281436.543592 trunc from 0x733fb5 to 0x521efa (0x2120ba bytes) 000057[0] 1781281436.589551 trunc from 0x521efb to 0x5e9359 (0xc745f bytes) 000059[1] 1781281436.638921 write_OD 0x560000 thru 0x563fff (0x4000 bytes) 000064[0] 1781281436.696840 mapread 0x071000 thru 0x07aea3 (0x9ea4 bytes) 000065[1] 1781281436.741625 mapread 0x3a4000 thru 0x3b3c80 (0xfc81 bytes) 000066[1] 1781281436.842181 mapwrite 0x5ac000 thru 0x5bb62a (0xf62b bytes) 000067[1] 1781281437.056153 mapread 0x4a7000 thru 0x4a9154 (0x2155 bytes) 000068[1] 1781281437.084418 mapwrite 0x599000 thru 0x59c016 (0x3017 bytes) 000069[1] 1781281437.198224 write_OD 0x37f000 thru 0x386fff (0x8000 bytes) 000072[1] 1781281437.338305 mapread 0x2c0000 thru 0x2cba85 (0xba86 bytes) 000077[1] 1781281437.434647 trunc from 0x5e935a to 0x79a7f8 (0x1b149f bytes) 000079[1] 1781281437.525137 write_OD 0x305000 thru 0x30efff (0xa000 bytes) 000081[1] 1781281437.609844 trunc from 0x79a7f9 to 0x15afc3 (0x63f835 bytes) 000082[0] 1781281437.676055 read 0x01a000 thru 0x01f157 (0x5158 bytes) 000083[1] 1781281437.744219 read_OD 0x072000 thru 0x074fff (0x3000 bytes) 000084[1] 1781281437.750961 mapwrite 0x121000 thru 0x128749 (0x774a bytes) 000087[1] 1781281437.928017 read_OD 0x0d6000 thru 0x0dbfff (0x6000 bytes) 000088[1] 1781281437.969612 mapread 0x0f0000 thru 0x0fcc2c (0xcc2d bytes) 000090[0] 1781281438.022424 mapwrite 0x4ec000 thru 0x4f9c23 (0xdc24 bytes) 000091[1] 1781281438.218298 mapread 0x29c000 thru 0x2a1b1d (0x5b1e bytes) 000096[1] 1781281438.364741 write_OD 0x5dc000 thru 0x5ddfff (0x2000 bytes) 000097[1] 1781281438.430665 read_OD 0x1f7000 thru 0x1fdfff (0x7000 bytes) 000098[1] 1781281438.442976 mapread 0x09c000 thru 0x0a0a9b (0x4a9c bytes) 000101[0] 1781281438.544805 write 0x85f000 thru 0x866718 (0x7719 bytes) 000102[0] 1781281438.581738 read 0x2bc000 thru 0x2c95d1 (0xd5d2 bytes) 000104[1] 1781281438.655984 mapread 0x514000 thru 0x521e15 (0xde16 bytes) 000105[1] 1781281438.843626 mapwrite 0x3ff000 thru 0x402506 (0x3507 bytes) 000107[0] 1781281439.177684 mapread 0x744000 thru 0x74a0da (0x60db bytes) 000109[0] 1781281439.247184 trunc from 0x866719 to 0x6372c2 (0x22f456 bytes) 000110[1] 1781281439.285571 read_OD 0x592000 thru 0x594fff (0x3000 bytes) 000112[1] 1781281439.316103 mapread 0x22c000 thru 0x22f366 (0x3367 bytes) 000113[0] 1781281439.346312 trunc from 0x6372c3 to 0x78bacb (0x154809 bytes) 000115[1] 1781281439.402859 mapwrite 0x17e000 thru 0x18a41a (0xc41b bytes) 000118[0] 1781281439.611657 write 0x3b7000 thru 0x3c226f (0xb270 bytes) 000119[1] 1781281439.643565 write_OD 0x891000 thru 0x89cfff (0xc000 bytes) 000122[1] 1781281439.710768 mapwrite 0x4e5000 thru 0x4e66fd (0x16fe bytes) 000126[0] 1781281439.865417 mapread 0x016000 thru 0x01ac52 (0x4c53 bytes) 000127[0] 1781281439.923183 read 0x33c000 thru 0x3452c9 (0x92ca bytes) 000128[0] 1781281439.987349 write 0x90f000 thru 0x912d93 (0x3d94 bytes) 000129[0] 1781281440.086081 mapread 0x529000 thru 0x538712 (0xf713 bytes) 000131[0] 1781281440.184461 read 0x7ac000 thru 0x7bae6d (0xee6e bytes) 000135[1] 1781281440.256311 write_OD 0x326000 thru 0x333fff (0xe000 bytes) 000136[1] 1781281440.341427 trunc from 0x912d94 to 0x1afe31 (0x762f62 bytes) 000137[0] 1781281440.446168 mapread 0x0ed000 thru 0x0ef7d1 (0x27d2 bytes) 000138[1] 1781281440.560984 write_OD 0x98b000 thru 0x994fff (0xa000 bytes) 000139[0] 1781281440.753665 read 0x78f000 thru 0x79151e (0x251f bytes) 000142[0] 1781281440.844230 mapread 0x113000 thru 0x11adcb (0x7dcc bytes) 000148[0] 1781281440.926219 trunc from 0x995000 to 0x6378f5 (0x35d70a bytes) 000149[1] 1781281441.048120 mapwrite 0x3f4000 thru 0x403f8a (0xff8b bytes) 000150[1] 1781281441.414338 mapwrite 0x1c3000 thru 0x1cc5ab (0x95ac bytes) 000152[1] 1781281441.550138 trunc from 0x6378f6 to 0x511413 (0x1264e2 bytes) 000157[1] 1781281441.609504 write_OD 0x39c000 thru 0x39cfff (0x1000 bytes) 000158[1] 1781281441.672788 read_OD 0x452000 thru 0x45dfff (0xc000 bytes) 000159[1] 1781281441.686126 trunc from 0x511414 to 0x3d80c2 (0x139351 bytes) 000162[0] 1781281441.783435 mapwrite 0x031000 thru 0x03b1b0 (0xa1b1 bytes) 000164[0] 1781281441.952192 mapwrite 0x6d0000 thru 0x6d7a11 (0x7a12 bytes) 000167[1] 1781281442.089497 mapwrite 0x493000 thru 0x4a0b1b (0xdb1c bytes) 000168[1] 1781281442.250073 mapwrite 0x1f0000 thru 0x1f820a (0x820b bytes) 000170[0] 1781281442.414555 mapwrite 0x7f9000 thru 0x805f57 (0xcf58 bytes) 000171[1] 1781281442.646983 mapread 0x7c2000 thru 0x7cfa67 (0xda68 bytes) 000172[0] 1781281442.782075 read 0x35e000 thru 0x365427 (0x7428 bytes) 000178[1] 1781281442.838353 write_OD 0x6bb000 thru 0x6c4fff (0xa000 bytes) 000180[1] 1781281443.068275 trunc from 0x805f58 to 0x3d5521 (0x430a36 bytes) 000182[0] 1781281443.136275 mapread 0x15b000 thru 0x167b48 (0xcb49 bytes) 000183[0] 1781281443.371922 mapwrite 0x732000 thru 0x73ae77 (0x8e78 bytes) 000185[1] 1781281443.654357 write_OD 0x7dd000 thru 0x7e4fff (0x8000 bytes) 000186[1] 1781281443.757474 read_OD 0x59a000 thru 0x5a8fff (0xf000 bytes) 000188[0] 1781281443.795403 trunc from 0x7e5000 to 0x56a23e (0x27adc1 bytes) 000190[0] 1781281443.883591 mapread 0x522000 thru 0x523cd9 (0x1cda bytes) 000193[0] 1781281443.917961 trunc from 0x56a23f to 0x7bec3e (0x254a00 bytes) 000194[1] 1781281443.966380 mapread 0x025000 thru 0x028b5f (0x3b60 bytes) 000195[1] 1781281444.066925 read_OD 0x36a000 thru 0x375fff (0xc000 bytes) 000196[0] 1781281444.091570 read 0x5e9000 thru 0x5f896d (0xf96e bytes) 000198[0] 1781281444.145881 mapread 0x36a000 thru 0x36bf2c (0x1f2d bytes) truncating to largest ever: 0x96f09c 000200[1] 1781281444.166028 trunc from 0x7bec3f to 0x96f09b (0x1b045d bytes) 000201[1] 1781281444.205863 read_OD 0x4e1000 thru 0x4eafff (0xa000 bytes) 000203[1] 1781281444.270084 read_OD 0x208000 thru 0x212fff (0xb000 bytes) 000206[1] 1781281444.282959 mapwrite 0x22d000 thru 0x23a81d (0xd81e bytes) 000207[0] 1781281444.514149 mapread 0x64d000 thru 0x6532bc (0x62bd bytes) 000208[1] 1781281444.629175 write_OD 0x27f000 thru 0x282fff (0x4000 bytes) 000209[1] 1781281444.721247 write_OD 0x544000 thru 0x54bfff (0x8000 bytes) 000210[0] 1781281444.842845 mapread 0x1c8000 thru 0x1c9324 (0x1325 bytes) 000211[0] 1781281444.872400 write 0x192000 thru 0x198a8d (0x6a8e bytes) 000212[0] 1781281444.924237 write 0x5fa000 thru 0x5fc7ab (0x27ac bytes) 000213[0] 1781281444.971038 mapwrite 0x18c000 thru 0x19b744 (0xf745 bytes) 000214[0] 1781281445.177608 trunc from 0x96f09c to 0x032f8c (0x93c10f bytes) 000217[0] 1781281445.261872 trunc from 0x032f8d to 0x6c6811 (0x693885 bytes) 000219[1] 1781281445.304864 mapread 0x3c7000 thru 0x3ccc77 (0x5c78 bytes) 000223[0] 1781281445.408164 mapread 0x2a8000 thru 0x2ac52b (0x452c bytes) 000224[1] 1781281445.459719 mapwrite 0x87b000 thru 0x883214 (0x8215 bytes) 000225[1] 1781281445.767128 read_OD 0x694000 thru 0x69dfff (0xa000 bytes) 000227[1] 1781281445.841600 trunc from 0x883215 to 0x169374 (0x719ea0 bytes) 000229[1] 1781281445.961860 read_OD 0x044000 thru 0x04dfff (0xa000 bytes) 000230[0] 1781281446.010441 write 0x5fd000 thru 0x60ccd8 (0xfcd9 bytes) 000231[1] 1781281446.077440 mapwrite 0x722000 thru 0x726f19 (0x4f1a bytes) 000232[0] 1781281446.470320 trunc from 0x726f1a to 0x6a5dd6 (0x81143 bytes) 000233[0] 1781281446.577245 trunc from 0x6a5dd7 to 0x04e08e (0x657d48 bytes) 000236[1] 1781281446.695156 write_OD 0x171000 thru 0x17cfff (0xc000 bytes) 000238[1] 1781281446.860411 mapread 0x176000 thru 0x17b4f7 (0x54f8 bytes) 000239[1] 1781281446.939107 mapread 0x0b1000 thru 0x0befb5 (0xdfb6 bytes) 000242[0] 1781281447.133621 trunc from 0x17d000 to 0x737a2e (0x5baa2f bytes) 000244[0] 1781281447.302091 trunc from 0x737a2f to 0x6544f8 (0xe3536 bytes) 000245[1] 1781281447.366122 trunc from 0x6544f9 to 0x5f395a (0x60b9e bytes) 000246[0] 1781281447.472258 mapwrite 0x9dc000 thru 0x9e9274 (0xd275 bytes) 000250[1] 1781281447.962364 trunc from 0x9e9275 to 0x06ae2b (0x97e449 bytes) 000251[1] 1781281448.039720 mapread 0x03c000 thru 0x042d11 (0x6d12 bytes) 000253[0] 1781281448.100740 read 0x010000 thru 0x01c6ad (0xc6ae bytes) 000254[1] 1781281448.130480 trunc from 0x06ae2c to 0x4c98c9 (0x45ea9e bytes) 000259[0] 1781281448.190284 mapread 0x122000 thru 0x12820b (0x620c bytes) 000260[1] 1781281448.290839 trunc from 0x4c98ca to 0x148d58 (0x380b71 bytes) 000262[1] 1781281448.345693 trunc from 0x148d59 to 0x8cd13f (0x7843e7 bytes) 000264[0] 1781281448.380256 write 0x54a000 thru 0x559cfe (0xfcff bytes) 000265[0] 1781281448.401052 read 0x32c000 thru 0x32c309 (0x30a bytes) 000266[1] 1781281448.484361 trunc from 0x8cd140 to 0x6a2a7e (0x22a6c1 bytes) 000267[0] 1781281448.529199 mapread 0x140000 thru 0x1401e7 (0x1e8 bytes) 000268[1] 1781281448.564206 read_OD 0x64c000 thru 0x657fff (0xc000 bytes) 000269[0] 1781281448.574455 trunc from 0x6a2a7f to 0x807dc2 (0x165344 bytes) 000271[0] 1781281448.651320 write 0x6e6000 thru 0x6e9850 (0x3851 bytes) 000272[0] 1781281448.683757 mapread 0x1f8000 thru 0x2030cd (0xb0ce bytes) 000273[0] 1781281448.782929 trunc from 0x807dc3 to 0x719e66 (0xedf5c bytes) 000279[1] 1781281448.913507 write_OD 0x15d000 thru 0x15ffff (0x3000 bytes) 000280[1] 1781281448.986655 mapwrite 0x467000 thru 0x475bd1 (0xebd2 bytes) 000281[1] 1781281449.287977 mapread 0x1f5000 thru 0x1f696c (0x196d bytes) 000283[0] 1781281449.363424 mapwrite 0x198000 thru 0x198177 (0x178 bytes) 000284[0] 1781281449.527397 read 0x152000 thru 0x1537ff (0x1800 bytes) 000285[1] 1781281449.776174 mapwrite 0x02f000 thru 0x03e8f7 (0xf8f8 bytes) 000294[1] 1781281450.095714 mapread 0x470000 thru 0x4718bd (0x18be bytes) 000297[0] 1781281450.133582 write 0x002000 thru 0x00b93b (0x993c bytes) 000298[0] 1781281450.156735 read 0x4f8000 thru 0x4fb36e (0x336f bytes) 000299[0] 1781281450.208439 read 0x3a0000 thru 0x3ac263 (0xc264 bytes) 000300[1] 1781281450.257860 read_OD 0x150000 thru 0x152fff (0x3000 bytes) 000301[1] 1781281450.314802 trunc from 0x719e67 to 0x1228ec (0x5f757a bytes) 000302[1] 1781281450.373871 trunc from 0x1228ed to 0x124833 (0x1f47 bytes) 000303[1] 1781281450.417767 write_OD 0x700000 thru 0x70dfff (0xe000 bytes) 000304[1] 1781281450.488865 read_OD 0x250000 thru 0x251fff (0x2000 bytes) 000305[0] 1781281450.502287 trunc from 0x70e000 to 0x86aee9 (0x15ceea bytes) 000309[1] 1781281450.589640 read_OD 0x122000 thru 0x12bfff (0xa000 bytes) 000312[1] 1781281450.621752 write_OD 0x330000 thru 0x336fff (0x7000 bytes) 000314[0] 1781281450.681158 mapread 0x5f6000 thru 0x6051e6 (0xf1e7 bytes) 000317[1] 1781281450.803133 mapread 0x063000 thru 0x070c6a (0xdc6b bytes) 000318[0] 1781281451.053468 read 0x062000 thru 0x06e176 (0xc177 bytes) 000319[0] 1781281451.164732 read 0x593000 thru 0x5991b4 (0x61b5 bytes) 000321[0] 1781281451.183441 mapread 0x29f000 thru 0x2a8dc5 (0x9dc6 bytes) 000324[1] 1781281451.184487 mapread 0x5c0000 thru 0x5c4561 (0x4562 bytes) 000326[1] 1781281451.185092 mapread 0x206000 thru 0x20f5eb (0x95ec bytes) 000327[0] 1781281451.185507 trunc from 0x86aeea to 0x6babd3 (0x1b0316 bytes) 000328[1] 1781281451.327833 mapwrite 0x541000 thru 0x54a155 (0x9156 bytes) 000330[1] 1781281451.721451 write_OD 0x511000 thru 0x511fff (0x1000 bytes) 000332[0] 1781281451.864314 write 0x3e0000 thru 0x3edd16 (0xdd17 bytes) 000333[0] 1781281451.899409 trunc from 0x6babd4 to 0x15e55a (0x55c679 bytes) 000334[1] 1781281451.934900 mapwrite 0x072000 thru 0x07c8fd (0xa8fe bytes) 000340[1] 1781281452.258804 mapread 0x0e0000 thru 0x0e1af9 (0x1afa bytes) 000344[1] 1781281452.287365 trunc from 0x15e55b to 0x5cf79b (0x471241 bytes) 000346[1] 1781281452.321198 mapread 0x500000 thru 0x50fbfc (0xfbfd bytes) 000347[0] 1781281452.423431 read 0x349000 thru 0x35722c (0xe22d bytes) 000348[0] 1781281452.481848 read 0x238000 thru 0x23dfb4 (0x5fb5 bytes) 000351[0] 1781281452.511117 read 0x1c2000 thru 0x1c5b39 (0x3b3a bytes) 000354[1] 1781281452.528958 mapread 0x51e000 thru 0x51f172 (0x1173 bytes) truncating to largest ever: 0x98fad0 000355[0] 1781281452.540514 trunc from 0x5cf79c to 0x98facf (0x3c0334 bytes) 000356[1] 1781281452.604793 trunc from 0x98fad0 to 0x776ea4 (0x218c2b bytes) 000362[1] 1781281452.688412 read_OD 0x075000 thru 0x07dfff (0x9000 bytes) 000363[1] 1781281452.705690 read_OD 0x544000 thru 0x54cfff (0x9000 bytes) 000364[1] 1781281452.716653 read_OD 0x14c000 thru 0x155fff (0xa000 bytes) 000365[0] 1781281452.723911 mapread 0x355000 thru 0x362279 (0xd27a bytes) 000367[0] 1781281452.818987 mapwrite 0x2d5000 thru 0x2dd6d2 (0x86d3 bytes) 000370[1] 1781281452.959077 read_OD 0x201000 thru 0x202fff (0x2000 bytes) 000371[0] 1781281452.993730 mapread 0x076000 thru 0x082209 (0xc20a bytes) 000372[0] 1781281453.076117 trunc from 0x776ea5 to 0x6132e3 (0x163bc1 bytes) 000375[0] 1781281453.146366 trunc from 0x6132e4 to 0x4d4740 (0x13eba3 bytes) 000376[1] 1781281453.198361 read_OD 0x097000 thru 0x09ffff (0x9000 bytes) 000377[1] 1781281453.239472 mapwrite 0x4ca000 thru 0x4ccfcf (0x2fd0 bytes) 000379[0] 1781281453.414733 write 0x023000 thru 0x02aeed (0x7eee bytes) 000383[1] 1781281453.506470 read_OD 0x1d5000 thru 0x1dcfff (0x8000 bytes) 000385[1] 1781281453.551317 read_OD 0x31b000 thru 0x322fff (0x8000 bytes) 000386[1] 1781281453.571485 read_OD 0x378000 thru 0x383fff (0xc000 bytes) 000388[0] 1781281453.585208 mapwrite 0x16a000 thru 0x175542 (0xb543 bytes) 000389[0] 1781281453.768791 read 0x1f4000 thru 0x203b49 (0xfb4a bytes) 000390[1] 1781281453.839025 write_OD 0x48e000 thru 0x491fff (0x4000 bytes) 000393[0] 1781281453.921429 mapread 0x299000 thru 0x2a6d4d (0xdd4e bytes) 000395[0] 1781281453.994355 trunc from 0x4d4741 to 0x6564da (0x181d9a bytes) 000396[0] 1781281454.087832 mapwrite 0x6c5000 thru 0x6d1ff2 (0xcff3 bytes) 000397[0] 1781281454.405162 mapread 0x630000 thru 0x63122c (0x122d bytes) 000398[1] 1781281454.427404 read_OD 0x481000 thru 0x484fff (0x4000 bytes) 000400[1] 1781281454.458761 mapwrite 0x112000 thru 0x115c55 (0x3c56 bytes) 000401[0] 1781281454.608690 write 0x773000 thru 0x780459 (0xd45a bytes) 000403[0] 1781281454.637124 read 0x0f8000 thru 0x0fdfc4 (0x5fc5 bytes) 000405[1] 1781281454.788006 mapwrite 0x1ac000 thru 0x1b91fa (0xd1fb bytes) 000409[0] 1781281455.125487 mapread 0x635000 thru 0x63a0af (0x50b0 bytes) 000410[1] 1781281455.199207 trunc from 0x78045a to 0x5e5ab3 (0x19a9a6 bytes) 000411[0] 1781281455.251455 mapwrite 0x6bd000 thru 0x6c19ab (0x49ac bytes) 000412[0] 1781281455.444839 mapwrite 0x740000 thru 0x74af67 (0xaf68 bytes) 000419[0] 1781281455.709162 mapwrite 0x2b7000 thru 0x2c6469 (0xf46a bytes) 000422[0] 1781281455.889078 write 0x34e000 thru 0x355929 (0x792a bytes) 000423[1] 1781281455.909132 trunc from 0x74af68 to 0x5d0673 (0x17a8f4 bytes) 000424[1] 1781281455.978903 mapread 0x0ce000 thru 0x0dc03c (0xe03d bytes) 000434[0] 1781281456.246021 read 0x116000 thru 0x124be7 (0xebe8 bytes) 000435[1] 1781281456.408486 write_OD 0x79b000 thru 0x7a4fff (0xa000 bytes) 000436[0] 1781281456.524268 trunc from 0x7a5000 to 0x35f91d (0x4456e2 bytes) 000438[0] 1781281456.623237 trunc from 0x35f91e to 0x94bf47 (0x5ec62a bytes) 000439[1] 1781281456.719054 trunc from 0x94bf48 to 0x946c67 (0x52e0 bytes) 000441[0] 1781281456.768406 write 0x231000 thru 0x233afc (0x2afd bytes) 000442[1] 1781281456.870667 mapread 0x92f000 thru 0x936ae4 (0x7ae5 bytes) 000445[1] 1781281457.008956 write_OD 0x93d000 thru 0x941fff (0x5000 bytes) 000447[1] 1781281457.082281 write_OD 0x209000 thru 0x214fff (0xc000 bytes) 000449[0] 1781281457.147136 mapread 0x93e000 thru 0x946503 (0x8504 bytes) 000450[0] 1781281457.206810 trunc from 0x946c68 to 0x47ca49 (0x4ca21e bytes) 000457[0] 1781281457.334098 mapwrite 0x1f0000 thru 0x1f0492 (0x493 bytes) 000458[1] 1781281457.512838 mapwrite 0x862000 thru 0x86f61a (0xd61b bytes) 000459[1] 1781281457.861827 mapwrite 0x183000 thru 0x18547f (0x2480 bytes) 000460[1] 1781281458.012161 mapread 0x708000 thru 0x71089c (0x889d bytes) 000462[0] 1781281458.157959 mapread 0x173000 thru 0x175946 (0x2947 bytes) 000463[0] 1781281458.199738 write 0x3b9000 thru 0x3c4dc9 (0xbdca bytes) 000464[0] 1781281458.220635 read 0x368000 thru 0x368e8f (0xe90 bytes) 000465[1] 1781281458.275901 mapwrite 0x076000 thru 0x077f16 (0x1f17 bytes) 000468[1] 1781281458.483390 read_OD 0x859000 thru 0x85dfff (0x5000 bytes) 000469[1] 1781281458.513035 mapread 0x296000 thru 0x2994c5 (0x34c6 bytes) 000470[0] 1781281458.563739 write 0x1d7000 thru 0x1dbe43 (0x4e44 bytes) 000474[1] 1781281458.695377 trunc from 0x86f61b to 0x06ea0a (0x800c10 bytes) 000475[0] 1781281458.785613 read 0x03b000 thru 0x03c7fa (0x17fb bytes) 000476[0] 1781281458.838746 trunc from 0x06ea0b to 0x33fc0f (0x2d1205 bytes) 000477[0] 1781281458.922155 write 0x617000 thru 0x622e70 (0xbe71 bytes) 000478[0] 1781281458.955212 mapwrite 0x029000 thru 0x034ac4 (0xbac5 bytes) 000479[1] 1781281459.299666 trunc from 0x622e71 to 0x88a088 (0x267218 bytes) 000480[1] 1781281459.384940 read_OD 0x268000 thru 0x26cfff (0x5000 bytes) 000483[0] 1781281459.411938 write 0x8a8000 thru 0x8b33d4 (0xb3d5 bytes) 000484[0] 1781281459.444240 write 0x422000 thru 0x4302a3 (0xe2a4 bytes) 000491[1] 1781281459.591168 mapread 0x008000 thru 0x016031 (0xe032 bytes) 000492[0] 1781281459.868252 mapread 0x1b3000 thru 0x1bc26d (0x926e bytes) 000493[1] 1781281459.926619 write_OD 0x17b000 thru 0x189fff (0xf000 bytes) 000494[1] 1781281460.086395 read_OD 0x783000 thru 0x78dfff (0xb000 bytes) 000498[1] 1781281460.122938 write_OD 0x4d3000 thru 0x4d6fff (0x4000 bytes) All operations completed A-OK! PASS 16a (67s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 12:24:26 (1781281466) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.694597 s, 3.0 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.362822 s, 5.8 MB/s Chance of close/open is 1 in 50 Seed set to 1470 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x3fcf04 000002[0] 1781281471.757677 trunc from 00000000 to 0x3fcf03 (0x3fcf04 bytes) 000004[0] 1781281471.868350 mapread 0x1a3508 thru 0x1a472f (0x1228 bytes) 000006[0] 1781281472.011911 mapread 0x17faec thru 0x1805cd (0xae2 bytes) 000009[1] 1781281472.134356 mapwrite 0x5e1322 thru 0x5e26eb (0x13ca bytes) 000010[0] 1781281472.524484 write 0x4a669b thru 0x4a8645 (0x1fab bytes) 000013[1] 1781281472.695922 mapwrite 0x868ddf thru 0x86969d (0x8bf bytes) 000015[0] 1781281472.963139 mapread 0x22c17a thru 0x22dc50 (0x1ad7 bytes) 000018[1] 1781281473.099135 write 0x55632f thru 0x556cc6 (0x998 bytes) 000020[1] 1781281473.177055 mapwrite 0x8e1cd3 thru 0x8e2014 (0x342 bytes) 000021[0] 1781281473.327722 mapwrite 0x16268d thru 0x1642ff (0x1c73 bytes) 000022[1] 1781281473.511334 mapread 0x35d909 thru 0x35db74 (0x26c bytes) 000026[1] 1781281473.543640 read 0x4fa87c thru 0x4faa71 (0x1f6 bytes) 000027[1] 1781281473.566246 trunc from 0x8e2015 to 0x227aba (0x6ba55a bytes) 000028[0] 1781281473.619688 write 0x196c00 thru 0x1979c3 (0xdc4 bytes) 000032[1] 1781281473.755973 read 0x15628f thru 0x1574ca (0x123c bytes) 000034[1] 1781281473.851511 mapwrite 0x715e2e thru 0x7175c3 (0x1796 bytes) truncating to largest ever: 0x6ea135 000035[0] 1781281474.134261 trunc from 0x7175c4 to 0x6ea134 (0x2d48f bytes) 000036[0] 1781281474.226195 mapwrite 0x8301f0 thru 0x830894 (0x6a5 bytes) 000042[1] 1781281474.394426 mapwrite 0x584270 thru 0x585266 (0xff7 bytes) 000044[0] 1781281474.527694 read 0x4c2f95 thru 0x4c3e42 (0xeae bytes) 000047[0] 1781281474.627135 write 0x35269e thru 0x35392d (0x1290 bytes) 000048[0] 1781281474.669205 mapwrite 0x7b870b thru 0x7ba3e5 (0x1cdb bytes) 000049[0] 1781281474.902773 write 0x8b79e5 thru 0x8b7bdd (0x1f9 bytes) 000050[1] 1781281474.912857 mapread 0x76edd8 thru 0x76f15d (0x386 bytes) 000051[0] 1781281475.002332 read 0x1546a7 thru 0x156412 (0x1d6c bytes) 000052[0] 1781281475.153593 trunc from 0x8b7bde to 0x2e964e (0x5ce58f bytes) 000056[0] 1781281475.451613 write 0x6385be thru 0x63a0b1 (0x1af4 bytes) 000057[1] 1781281475.515701 write 0x98406b thru 0x98544b (0x13e1 bytes) 000058[0] 1781281475.581738 write 0x5f2a3d thru 0x5f3a30 (0xff4 bytes) 000060[0] 1781281475.709479 read 0x77ed02 thru 0x77ee55 (0x154 bytes) 000061[1] 1781281475.814401 mapwrite 0x06179b thru 0x061a3a (0x2a0 bytes) 000062[1] 1781281476.032411 read 0x8abedc thru 0x8ac3c9 (0x4ee bytes) 000063[1] 1781281476.160115 trunc from 0x98544c to 0x4bfdd1 (0x4c567a bytes) 000064[0] 1781281476.292681 trunc from 0x4bfdd2 to 0x6b5309 (0x1f5538 bytes) 000065[1] 1781281476.383963 read 0x1a9e76 thru 0x1aa8b1 (0xa3c bytes) 000066[0] 1781281476.430695 trunc from 0x6b530a to 0x202ba8 (0x4b2761 bytes) 000070[0] 1781281476.585465 mapwrite 0x3b8293 thru 0x3ba1d7 (0x1f45 bytes) 000072[1] 1781281476.763158 trunc from 0x3ba1d8 to 0x17a5cf (0x23fc08 bytes) 000073[0] 1781281476.856003 mapwrite 0x3d5473 thru 0x3d69cc (0x155a bytes) 000074[1] 1781281477.040251 read 0x16c072 thru 0x16d25c (0x11eb bytes) 000075[0] 1781281477.150162 read 0x3a8c7b thru 0x3a9012 (0x398 bytes) 000079[0] 1781281477.233444 write 0x1957df thru 0x19744a (0x1c6c bytes) 000080[1] 1781281477.281703 read 0x0279aa thru 0x02815d (0x7b4 bytes) 000081[0] 1781281477.354678 write 0x7abd49 thru 0x7aceba (0x1172 bytes) 000082[0] 1781281477.359668 mapwrite 0x9c7617 thru 0x9c878f (0x1179 bytes) 000085[1] 1781281477.741827 write 0x3c5240 thru 0x3c7202 (0x1fc3 bytes) 000086[1] 1781281477.826998 mapread 0x01a586 thru 0x01b3e8 (0xe63 bytes) 000087[1] 1781281478.096499 mapwrite 0x23ed41 thru 0x23fa02 (0xcc2 bytes) 000090[0] 1781281478.300072 read 0x7e81ce thru 0x7e8b94 (0x9c7 bytes) 000091[1] 1781281478.400871 read 0x0b9b11 thru 0x0ba5c2 (0xab2 bytes) 000093[1] 1781281478.429480 mapwrite 0x9a7521 thru 0x9a8ef5 (0x19d5 bytes) 000094[0] 1781281478.699853 write 0x157914 thru 0x157957 (0x44 bytes) 000095[1] 1781281478.778485 mapread 0x8c2d68 thru 0x8c3caf (0xf48 bytes) 000096[1] 1781281478.845658 read 0x05ae0b thru 0x05c1e2 (0x13d8 bytes) 000097[1] 1781281478.912207 read 0x3128b9 thru 0x313336 (0xa7e bytes) 000098[0] 1781281479.119275 read 0x8a0ee5 thru 0x8a1a24 (0xb40 bytes) 000102[1] 1781281479.227501 read 0x477c8e thru 0x479a6f (0x1de2 bytes) 000103[1] 1781281479.295296 read 0x8e290c thru 0x8e3236 (0x92b bytes) 000105[1] 1781281479.342131 write 0x4d5fc5 thru 0x4d744c (0x1488 bytes) 000106[1] 1781281479.438030 mapread 0x6d5b73 thru 0x6d6226 (0x6b4 bytes) 000109[0] 1781281479.465856 mapwrite 0x239273 thru 0x23aa16 (0x17a4 bytes) 000112[0] 1781281479.841231 read 0x35aff1 thru 0x35b135 (0x145 bytes) 000118[1] 1781281480.114246 read 0x2bea8b thru 0x2c0071 (0x15e7 bytes) 000121[1] 1781281480.248309 read 0x8faeb0 thru 0x8fc8e4 (0x1a35 bytes) 000123[0] 1781281480.263117 trunc from 0x9c8790 to 0x0958a0 (0x932eef bytes) 000124[1] 1781281480.438610 write 0x428d35 thru 0x42a53b (0x1807 bytes) 000125[1] 1781281480.539982 write 0x22c4ed thru 0x22de75 (0x1989 bytes) 000126[0] 1781281480.609911 mapread 0x1ec802 thru 0x1ec8e7 (0xe6 bytes) 000127[0] 1781281480.703249 read 0x39a7a5 thru 0x39a9d1 (0x22d bytes) 000128[1] 1781281480.782494 mapread 0x0f2080 thru 0x0f35a5 (0x1526 bytes) 000132[0] 1781281480.926685 mapread 0x16cda5 thru 0x16d0a6 (0x302 bytes) 000133[1] 1781281480.998562 write 0x5892f8 thru 0x58a754 (0x145d bytes) 000134[1] 1781281481.022276 mapwrite 0x64805d thru 0x649ae5 (0x1a89 bytes) 000135[1] 1781281481.180765 mapwrite 0x876dd8 thru 0x877a5a (0xc83 bytes) 000136[0] 1781281481.388346 mapwrite 0x4d824b thru 0x4d8289 (0x3f bytes) 000137[1] 1781281481.501418 read 0x560118 thru 0x5620f1 (0x1fda bytes) 000138[0] 1781281481.602795 read 0x20d06e thru 0x20ea6f (0x1a02 bytes) truncating to largest ever: 0x85033d 000139[1] 1781281481.694656 trunc from 0x877a5b to 0x85033c (0x2771e bytes) 000140[1] 1781281481.871897 mapread 0x752199 thru 0x754157 (0x1fbf bytes) 000141[0] 1781281482.066026 read 0x07af5b thru 0x07bf6d (0x1013 bytes) 000145[1] 1781281482.295818 mapread 0x39e1d4 thru 0x39e370 (0x19d bytes) 000147[1] 1781281482.317445 mapread 0x732123 thru 0x733ab3 (0x1991 bytes) 000153[0] 1781281482.463269 mapread 0x32a52e thru 0x32aa92 (0x565 bytes) 000155[0] 1781281482.760123 read 0x076ca3 thru 0x078393 (0x16f1 bytes) 000156[1] 1781281482.771841 mapread 0x54e72d thru 0x550338 (0x1c0c bytes) 000157[1] 1781281482.831025 write 0x354a7b thru 0x355200 (0x786 bytes) 000162[1] 1781281482.955914 write 0x23f8d3 thru 0x240c4c (0x137a bytes) 000163[0] 1781281483.036500 mapwrite 0x47eea6 thru 0x47f860 (0x9bb bytes) 000164[0] 1781281483.177176 mapread 0x5c3b83 thru 0x5c5701 (0x1b7f bytes) 000165[1] 1781281483.227218 read 0x6d4783 thru 0x6d5769 (0xfe7 bytes) 000166[1] 1781281483.283582 write 0x5cf3dc thru 0x5cf6eb (0x310 bytes) 000167[1] 1781281483.307500 trunc from 0x85033d to 0x57f465 (0x2d0ed7 bytes) 000168[1] 1781281483.373816 mapread 0x2bd430 thru 0x2bde0f (0x9e0 bytes) 000171[0] 1781281483.457396 mapwrite 0x7a497c thru 0x7a4e1f (0x4a4 bytes) 000174[0] 1781281483.670459 write 0x5fe685 thru 0x5ff841 (0x11bd bytes) 000176[0] 1781281483.729584 mapwrite 0x91f8d8 thru 0x9202c5 (0x9ee bytes) 000177[1] 1781281483.955914 read 0x539432 thru 0x53a0ff (0xcce bytes) 000179[0] 1781281484.019870 mapread 0x8650a1 thru 0x865153 (0xb3 bytes) 000181[1] 1781281484.071747 write 0x57e95a thru 0x5807b2 (0x1e59 bytes) 000189[0] 1781281484.393870 trunc from 0x9202c6 to 0x43655c (0x4e9d69 bytes) 000190[1] 1781281484.451076 write 0x9a37a8 thru 0x9a3dda (0x633 bytes) 000191[0] 1781281484.503482 mapread 0x4c73d4 thru 0x4c8501 (0x112e bytes) 000192[0] 1781281484.548514 read 0x38c43a thru 0x38e142 (0x1d09 bytes) 000193[0] 1781281484.684185 mapwrite 0x96b6ad thru 0x96d243 (0x1b97 bytes) 000194[0] 1781281484.911075 read 0x964edc thru 0x966601 (0x1726 bytes) 000196[1] 1781281485.059749 read 0x454f4f thru 0x455cd2 (0xd84 bytes) 000197[1] 1781281485.168187 mapwrite 0x7703cf thru 0x771ea8 (0x1ada bytes) 000198[0] 1781281485.403163 write 0x8f6660 thru 0x8f7707 (0x10a8 bytes) truncating to largest ever: 0x89dff0 000199[1] 1781281485.480787 trunc from 0x9a3ddb to 0x89dfef (0x105deb bytes) 000201[0] 1781281485.720081 mapread 0x25f4f7 thru 0x26014f (0xc59 bytes) truncating to largest ever: 0x9e37e1 000202[1] 1781281485.823390 trunc from 0x89dff0 to 0x9e37e0 (0x1457f1 bytes) 000206[1] 1781281485.915311 read 0x487624 thru 0x489251 (0x1c2e bytes) 000207[1] 1781281486.025856 read 0x402dcb thru 0x404392 (0x15c8 bytes) 000208[0] 1781281486.112293 write 0x69d33d thru 0x69f14d (0x1e11 bytes) 000209[0] 1781281486.222148 mapread 0x21acbd thru 0x21af07 (0x24b bytes) 000211[1] 1781281486.332500 trunc from 0x9e37e1 to 0x3e87b5 (0x5fb02b bytes) 000213[0] 1781281486.485375 mapread 0x3590e9 thru 0x359ac2 (0x9da bytes) 000214[1] 1781281486.562775 mapwrite 0x33783f thru 0x338069 (0x82b bytes) 000215[0] 1781281486.819611 read 0x124ecb thru 0x126a96 (0x1bcc bytes) 000216[0] 1781281486.898590 mapread 0x043bff thru 0x04526f (0x1671 bytes) 000220[1] 1781281487.005672 mapwrite 0x3c6f09 thru 0x3c7199 (0x291 bytes) 000222[1] 1781281487.165406 write 0x50f0a9 thru 0x50fa3a (0x992 bytes) 000224[1] 1781281487.167063 trunc from 0x50fa3b to 0x5c710c (0xb76d2 bytes) 000225[0] 1781281487.293188 mapread 0x23159c thru 0x2326f0 (0x1155 bytes) 000229[0] 1781281487.445897 mapread 0x307630 thru 0x30865c (0x102d bytes) 000231[0] 1781281487.560786 write 0x32cfde thru 0x32da64 (0xa87 bytes) 000233[0] 1781281487.618573 mapwrite 0x89db61 thru 0x89efd3 (0x1473 bytes) 000235[1] 1781281487.845123 read 0x02a75c thru 0x02b053 (0x8f8 bytes) 000236[1] 1781281487.896009 write 0x8e2229 thru 0x8e3929 (0x1701 bytes) 000237[0] 1781281487.931197 mapread 0x07dac9 thru 0x07ee47 (0x137f bytes) 000238[0] 1781281487.966792 trunc from 0x8e392a to 0x4a452c (0x43f3fd bytes) 000239[0] 1781281488.031586 trunc from 0x4a452d to 0x616a14 (0x1724e8 bytes) 000240[0] 1781281488.059155 read 0x0f1d66 thru 0x0f3238 (0x14d3 bytes) 000241[0] 1781281488.207781 write 0x669eac thru 0x66af36 (0x108b bytes) 000243[1] 1781281488.251472 read 0x36676b thru 0x3668ff (0x195 bytes) 000244[0] 1781281488.302189 read 0x4cb877 thru 0x4cc179 (0x903 bytes) 000246[1] 1781281488.308074 mapwrite 0x9465f7 thru 0x94722c (0xc36 bytes) 000248[0] 1781281488.507083 mapread 0x06f47e thru 0x0705c9 (0x114c bytes) 000249[0] 1781281488.552127 mapwrite 0x80ef42 thru 0x80f18a (0x249 bytes) 000250[0] 1781281488.666076 mapwrite 0x50828f thru 0x5088d0 (0x642 bytes) 000251[0] 1781281488.787379 read 0x3f075c thru 0x3f0b7f (0x424 bytes) 000252[0] 1781281488.950221 write 0x12acab thru 0x12c006 (0x135c bytes) 000256[0] 1781281489.029836 trunc from 0x94722d to 0x6c706c (0x2801c0 bytes) 000257[1] 1781281489.115058 mapwrite 0x519129 thru 0x5192d1 (0x1a9 bytes) 000258[0] 1781281489.307667 mapread 0x46cfc5 thru 0x46d34b (0x387 bytes) 000264[1] 1781281489.481306 mapwrite 0x2404b5 thru 0x241072 (0xbbe bytes) 000265[1] 1781281489.582116 read 0x4d8170 thru 0x4d90b7 (0xf48 bytes) 000266[0] 1781281489.629587 read 0x279bde thru 0x279d5c (0x17f bytes) 000267[1] 1781281489.659723 write 0x653723 thru 0x653efa (0x7d8 bytes) 000268[0] 1781281489.703215 trunc from 0x6c706d to 0x68a35d (0x3cd0f bytes) 000271[0] 1781281489.820776 trunc from 0x68a35e to 0x25310c (0x437251 bytes) 000272[1] 1781281489.840830 mapwrite 0x6a16bd thru 0x6a2678 (0xfbc bytes) 000273[0] 1781281489.970500 read 0x39dc9e thru 0x39f37c (0x16df bytes) 000275[0] 1781281490.024182 mapread 0x4d3e47 thru 0x4d4b5a (0xd14 bytes) 000279[1] 1781281490.111486 mapwrite 0x8471f5 thru 0x8484b0 (0x12bc bytes) 000283[1] 1781281490.442145 read 0x003a3e thru 0x0047c6 (0xd89 bytes) 000284[1] 1781281490.675775 read 0x2dc4d2 thru 0x2dce24 (0x953 bytes) 000287[1] 1781281490.770278 read 0x2db8d8 thru 0x2dbe2b (0x554 bytes) 000288[0] 1781281490.776719 mapwrite 0x6f6e10 thru 0x6f898c (0x1b7d bytes) 000289[1] 1781281490.944137 mapread 0x1bd1c0 thru 0x1bdf23 (0xd64 bytes) 000291[0] 1781281491.009606 mapread 0x4b3faf thru 0x4b455c (0x5ae bytes) 000292[1] 1781281491.087403 write 0x66c28f thru 0x66cab9 (0x82b bytes) 000293[1] 1781281491.213432 read 0x18ba06 thru 0x18c8b7 (0xeb2 bytes) 000297[1] 1781281491.309430 write 0x4e7cb9 thru 0x4e7e8f (0x1d7 bytes) 000298[1] 1781281491.311287 mapwrite 0x8670b0 thru 0x86843a (0x138b bytes) 000299[1] 1781281491.521391 trunc from 0x86843b to 0x0177ae (0x850c8c bytes) 000300[1] 1781281491.626010 mapwrite 0x0a0f87 thru 0x0a206b (0x10e5 bytes) 000302[0] 1781281491.783805 trunc from 0x0a206c to 0x7d61e5 (0x73417a bytes) 000304[1] 1781281491.934932 mapwrite 0x618da9 thru 0x61a541 (0x1799 bytes) 000306[0] 1781281492.192187 trunc from 0x7d61e6 to 0x7237ec (0xb29f9 bytes) 000308[1] 1781281492.323516 trunc from 0x7237ed to 0x5076ea (0x21c102 bytes) 000310[0] 1781281492.389366 read 0x234b58 thru 0x2359bf (0xe68 bytes) 000312[0] 1781281492.476394 write 0x4118c2 thru 0x412ec7 (0x1606 bytes) 000313[0] 1781281492.537493 mapwrite 0x1505f7 thru 0x152593 (0x1f9d bytes) 000315[1] 1781281492.749976 read 0x3bc512 thru 0x3bd8ac (0x139b bytes) 000318[1] 1781281492.859394 mapwrite 0x662787 thru 0x663be6 (0x1460 bytes) 000319[1] 1781281493.072466 trunc from 0x663be7 to 0x3a9f7a (0x2b9c6c bytes) 000320[1] 1781281493.139858 trunc from 0x3a9f7b to 0x439e00 (0x8fe86 bytes) 000321[0] 1781281493.186108 mapread 0x3530ac thru 0x353208 (0x15d bytes) 000323[1] 1781281493.292439 mapread 0x2f0245 thru 0x2f1e75 (0x1c31 bytes) 000324[1] 1781281493.319451 trunc from 0x439e01 to 0x0e73cb (0x352a35 bytes) 000325[0] 1781281493.367479 mapread 0x048ad7 thru 0x04a31e (0x1848 bytes) 000326[0] 1781281493.429391 write 0x1e8d8b thru 0x1e978b (0xa01 bytes) 000327[0] 1781281493.471109 write 0x2e8765 thru 0x2e877c (0x18 bytes) 000328[1] 1781281493.520810 write 0x6fdad9 thru 0x6fed4f (0x1277 bytes) 000329[0] 1781281493.592469 read 0x26393e thru 0x2654ec (0x1baf bytes) 000331[1] 1781281493.754073 write 0x1f0e8f thru 0x1f1f3f (0x10b1 bytes) 000334[1] 1781281493.825239 read 0x2c0f37 thru 0x2c2a3d (0x1b07 bytes) 000335[0] 1781281493.924481 write 0x026110 thru 0x0278de (0x17cf bytes) 000337[1] 1781281494.012319 mapwrite 0x320957 thru 0x3210f6 (0x7a0 bytes) 000339[1] 1781281494.142127 write 0x822d29 thru 0x823b0d (0xde5 bytes) 000343[0] 1781281494.202497 read 0x0932b0 thru 0x0942f6 (0x1047 bytes) 000345[0] 1781281494.289602 mapwrite 0x71d61f thru 0x71e253 (0xc35 bytes) 000346[1] 1781281494.557464 trunc from 0x823b0e to 0x9935ae (0x16faa1 bytes) 000348[1] 1781281494.670472 read 0x8f8d60 thru 0x8fa2aa (0x154b bytes) 000351[0] 1781281494.722483 mapwrite 0x4556f0 thru 0x45746c (0x1d7d bytes) 000352[0] 1781281494.826017 trunc from 0x9935af to 0x4e8b32 (0x4aaa7c bytes) 000353[0] 1781281494.864957 mapwrite 0x9c1096 thru 0x9c11db (0x146 bytes) 000354[1] 1781281494.984990 mapwrite 0x87e6f0 thru 0x87f970 (0x1281 bytes) 000355[1] 1781281495.264461 read 0x3ff624 thru 0x3ffc37 (0x614 bytes) 000356[0] 1781281495.537435 mapread 0x712d4a thru 0x713620 (0x8d7 bytes) 000357[1] 1781281495.578339 write 0x0c0aeb thru 0x0c0f0e (0x424 bytes) 000358[1] 1781281495.604565 mapread 0x11172d thru 0x111c00 (0x4d4 bytes) 000359[1] 1781281495.613052 mapwrite 0x671c2f thru 0x672d1f (0x10f1 bytes) 000360[0] 1781281495.692369 write 0x55a493 thru 0x55b04f (0xbbd bytes) 000361[0] 1781281495.797471 read 0x995d8f thru 0x995dbc (0x2e bytes) 000362[0] 1781281495.835227 mapwrite 0x5c6e62 thru 0x5c7139 (0x2d8 bytes) 000365[0] 1781281495.955461 write 0x2b2282 thru 0x2b3f75 (0x1cf4 bytes) 000366[1] 1781281495.991962 mapwrite 0x4a3e85 thru 0x4a3efa (0x76 bytes) 000367[1] 1781281496.112658 read 0x4f845d thru 0x4f90be (0xc62 bytes) 000368[1] 1781281496.140433 mapwrite 0x4fc2ee thru 0x4fca7f (0x792 bytes) 000371[1] 1781281496.324083 mapread 0x4cb722 thru 0x4cbd1b (0x5fa bytes) 000372[1] 1781281496.371104 trunc from 0x9c11dc to 0x2c8eb8 (0x6f8323 bytes) 000374[0] 1781281496.516686 mapread 0x05651f thru 0x058384 (0x1e66 bytes) 000375[1] 1781281496.641427 write 0x862122 thru 0x86244c (0x32b bytes) 000377[0] 1781281496.716622 write 0x8acd7f thru 0x8ad118 (0x39a bytes) 000379[0] 1781281496.793658 trunc from 0x8ad119 to 0x6d0638 (0x1dcae0 bytes) 000380[1] 1781281496.874242 write 0x6f523d thru 0x6f6ae9 (0x18ad bytes) 000383[1] 1781281496.995003 trunc from 0x6f6aea to 0x96dd9c (0x2772b3 bytes) 000384[0] 1781281497.067434 write 0x2fa3be thru 0x2fa50b (0x14e bytes) 000385[1] 1781281497.149151 read 0x776095 thru 0x777829 (0x1795 bytes) 000387[0] 1781281497.294617 mapwrite 0x1c45a5 thru 0x1c5def (0x184b bytes) 000388[0] 1781281497.424232 write 0x482d82 thru 0x483cb7 (0xf36 bytes) 000389[0] 1781281497.483499 read 0x70099f thru 0x702111 (0x1773 bytes) 000390[0] 1781281497.595882 write 0x41fe65 thru 0x420044 (0x1e0 bytes) 000391[1] 1781281497.621159 mapwrite 0x86f3aa thru 0x8708e2 (0x1539 bytes) 000393[0] 1781281497.783593 mapwrite 0x9b97c8 thru 0x9b9a7b (0x2b4 bytes) 000394[1] 1781281497.904109 read 0x7270c7 thru 0x7273a6 (0x2e0 bytes) 000400[1] 1781281497.965741 mapread 0x86868b thru 0x868d7a (0x6f0 bytes) 000401[1] 1781281498.009892 mapread 0x1b3bb2 thru 0x1b4a59 (0xea8 bytes) 000404[1] 1781281498.037359 write 0x7c5f00 thru 0x7c6808 (0x909 bytes) 000406[0] 1781281498.186816 mapwrite 0x8f912f thru 0x8f9658 (0x52a bytes) 000407[0] 1781281498.408752 write 0x72caee thru 0x72da22 (0xf35 bytes) 000409[0] 1781281498.554199 mapwrite 0x22d9b5 thru 0x22f763 (0x1daf bytes) 000415[0] 1781281498.813051 mapwrite 0x3bad85 thru 0x3bcbc0 (0x1e3c bytes) 000416[1] 1781281498.964514 write 0x8ad0ce thru 0x8ad5d5 (0x508 bytes) 000420[0] 1781281499.124057 mapwrite 0x033ab1 thru 0x0358f1 (0x1e41 bytes) 000422[0] 1781281499.230870 trunc from 0x9b9a7c to 0x12a4be (0x88f5bd bytes) 000426[1] 1781281499.358246 mapwrite 0x3b2aa6 thru 0x3b3b40 (0x109b bytes) 000429[1] 1781281499.529562 mapread 0x3a21e2 thru 0x3a2865 (0x684 bytes) 000430[1] 1781281499.552216 write 0x2e054f thru 0x2e1db3 (0x1865 bytes) 000433[1] 1781281499.717494 trunc from 0x3b3b41 to 0x58596e (0x1d1e2e bytes) 000435[1] 1781281499.837635 mapread 0x488511 thru 0x489199 (0xc89 bytes) 000437[1] 1781281499.863196 write 0x9d1b76 thru 0x9d243d (0x8c8 bytes) 000440[0] 1781281499.951472 read 0x8aa831 thru 0x8abdd0 (0x15a0 bytes) 000441[0] 1781281500.066159 write 0x461254 thru 0x461c0c (0x9b9 bytes) 000443[0] 1781281500.195350 mapwrite 0x54e8e4 thru 0x5508d2 (0x1fef bytes) 000445[1] 1781281500.404235 read 0x134763 thru 0x135bd2 (0x1470 bytes) 000450[0] 1781281500.555466 mapwrite 0x1644ec thru 0x1646f3 (0x208 bytes) 000451[1] 1781281500.617501 read 0x5cf413 thru 0x5cfddb (0x9c9 bytes) 000452[0] 1781281500.664364 mapwrite 0x2271bf thru 0x228fb6 (0x1df8 bytes) 000453[0] 1781281500.848067 write 0x7038a0 thru 0x703de4 (0x545 bytes) 000455[1] 1781281500.997608 read 0x3605f9 thru 0x360e04 (0x80c bytes) 000456[0] 1781281501.060064 mapread 0x45bbad thru 0x45da5c (0x1eb0 bytes) 000458[1] 1781281501.122073 mapwrite 0x5a9b0a thru 0x5aa805 (0xcfc bytes) 000460[0] 1781281501.267392 mapwrite 0x4e6186 thru 0x4e6c69 (0xae4 bytes) 000461[1] 1781281501.412343 mapwrite 0x77ada5 thru 0x77b7b6 (0xa12 bytes) 000464[1] 1781281501.555277 trunc from 0x9d243e to 0x6fe7c9 (0x2d3c74 bytes) 000465[0] 1781281501.624984 mapwrite 0x6f5b29 thru 0x6f6378 (0x850 bytes) 000467[1] 1781281501.765693 write 0x2ea7f3 thru 0x2eab39 (0x347 bytes) 000468[0] 1781281501.869608 read 0x25b60e thru 0x25be1c (0x80f bytes) 000470[0] 1781281502.005084 mapwrite 0x7227ba thru 0x723cc0 (0x1507 bytes) 000471[0] 1781281502.316883 mapwrite 0x3d09e5 thru 0x3d14d8 (0xaf4 bytes) 000473[0] 1781281502.378420 trunc from 0x723cc1 to 0x226380 (0x4fd940 bytes) 000475[1] 1781281502.401982 read 0x21e1c6 thru 0x21f9e0 (0x181b bytes) 000477[1] 1781281502.449675 write 0x2df2fc thru 0x2e0052 (0xd57 bytes) 000478[0] 1781281502.489417 read 0x198ce6 thru 0x199ef1 (0x120c bytes) 000479[1] 1781281502.519946 mapread 0x24dc47 thru 0x24f4ad (0x1867 bytes) 000480[1] 1781281502.561681 trunc from 0x2e0053 to 0x6bcde1 (0x3dcd8f bytes) 000481[1] 1781281502.661388 trunc from 0x6bcde2 to 0x2ca474 (0x3f296d bytes) 000484[0] 1781281502.727714 trunc from 0x2ca475 to 0x115144 (0x1b5330 bytes) 000486[1] 1781281502.818579 write 0x932141 thru 0x932771 (0x631 bytes) 000487[0] 1781281502.879479 mapread 0x793e44 thru 0x79490a (0xac7 bytes) 000489[1] 1781281503.155147 write 0x137d47 thru 0x1381a5 (0x45f bytes) 000490[1] 1781281503.246370 mapread 0x6f13db thru 0x6f2841 (0x1467 bytes) 000494[0] 1781281503.352860 read 0x8f7d84 thru 0x8f9afb (0x1d78 bytes) 000495[0] 1781281503.480817 read 0x255bbc thru 0x256abc (0xf01 bytes) 000497[1] 1781281503.540857 trunc from 0x932772 to 0x57d196 (0x3b55db bytes) 000498[0] 1781281503.661679 read 0x4f068c thru 0x4f2013 (0x1988 bytes) 000499[0] 1781281503.716067 read 0x311bbc thru 0x312187 (0x5cc bytes) 000500[1] 1781281503.859743 mapwrite 0x933421 thru 0x934122 (0xd02 bytes) All operations completed A-OK! PASS 16b (44s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 12:25:11 (1781281511) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (5s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 12:25:16 (1781281516) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.318648 s, 6.6 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.227032 s, 9.2 MB/s ldlm.namespaces.MGC192.168.202.131@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff97ba450e7800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff97ba4541b800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff97ba4541b800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff97ba450e7800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff97ba4541b800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 13.713 s, 7.6 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.33557 s, 24.2 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 3.40162 s, 30.8 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 7.27715 s, 14.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 11.061 s, 9.5 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.35157 s, 24.1 MB/s PASS 16d (60s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 12:26:16 (1781281576) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.431875 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133097 s, 7.9 MB/s PASS 16e (9s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 12:26:25 (1781281585) TIMEOUT PASS 16f (29s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 12:26:55 (1781281615) TIMEOUT PASS 16g (29s) == sanityn test 16i: read after truncate file ============ 12:27:23 (1781281643) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (7s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 12:27:30 (1781281650) fail_loc=0x8000030a PASS 17 (12s) == sanityn test 18: mmap sanity check =========================================================================================== 12:27:42 (1781281662) mmap test1: basic mmap operation (PASS, 0.17118s) mmap test2: MAP_PRIVATE not write back (PASS, 0.088693s) mmap test3: concurrent mmap ops on two nodes (PASS, 5.9472s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.6662s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.22099s) mmap test6: check mmap write/read content on two nodes (PASS, 0.3569s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.131s) PASS 18 (20s) == sanityn test 19: test concurrent uncached read races ========================================================================= 12:28:02 (1781281682) oleg231-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg231-client: oleg231-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (5s) == sanityn test 20: test extra readahead page left in cache ============================================================== 12:28:07 (1781281687) PASS 20 (8s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 12:28:15 (1781281695) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (8s) == sanityn test 23: others should see updated atime while another read============================================================== 12:28:24 (1781281704) now is 1781281707 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7136 new atime is 1781281768 PASS 23 (71s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 12:29:35 (1781281775) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.3K 375 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.2K 368 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 375 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 368 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 5.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 11.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118071 375 117696 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235728 368 235360 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 375 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 368 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118071 375 117696 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre[OST:1] filesystem_summary: 235728 368 235360 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118071 375 117696 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235728 368 235360 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 PASS 24a (7s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 12:29:42 (1781281782) PASS 24b (8s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 12:29:50 (1781281790) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (9s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 12:29:59 (1781281799) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (4s) == sanityn test 26a: allow mtime to get older ============ 12:30:03 (1781281803) utime: good mknod times 1781281803 <= 1781281804 <= 1781281804 for /mnt/lustre/f26a utime: good mknod times 1781281803 <= 1781281804 <= 1781281804 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (8s) == sanityn test 26b: sync mtime between ost and mds ====== 12:30:12 (1781281812) PASS 26b (11s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 12:30:26 (1781281826) 0 PASS 30 (9s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 12:30:36 (1781281836) fail_loc=0x314 PASS 31a (10s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 12:30:45 (1781281845) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg231-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.202.131@tcp:/lustre 7542784 12288 7524352 1% /mnt/lustre2 PASS 31b (26s) == sanityn test 31r: open-rename(replace) race =========== 12:31:12 (1781281872) -rw-r--r-- 1 root root 0 Jun 12 12:31 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (11s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 12:31:25 (1781281885) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (4s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 12:31:29 (1781281889) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (4s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 12:31:33 (1781281893) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (3s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 12:31:37 (1781281897) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (4s) == sanityn test 34: no lock timeout under IO ============= 12:31:41 (1781281901) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg231-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff97ba450e7800.ost_server_uuid,osc.lustre-OST0000-osc-ffff97ba4541b800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff97ba450e7800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff97ba4541b800.ost_server_uuid in IDLE state after 0 sec oleg231-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff97ba450e7800.ost_server_uuid,osc.lustre-OST0001-osc-ffff97ba4541b800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff97ba450e7800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff97ba4541b800.ost_server_uuid in FULL state after 0 sec oleg231-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff97ba450e7800.ost_server_uuid,osc.lustre-OST0000-osc-ffff97ba4541b800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff97ba450e7800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff97ba4541b800.ost_server_uuid in IDLE state after 0 sec oleg231-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff97ba450e7800.ost_server_uuid,osc.lustre-OST0001-osc-ffff97ba4541b800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff97ba450e7800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff97ba4541b800.ost_server_uuid in FULL state after 0 sec oleg231-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff97ba450e7800.ost_server_uuid,osc.lustre-OST0000-osc-ffff97ba4541b800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff97ba450e7800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff97ba4541b800.ost_server_uuid in IDLE state after 0 sec oleg231-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff97ba450e7800.ost_server_uuid,osc.lustre-OST0001-osc-ffff97ba4541b800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff97ba450e7800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff97ba4541b800.ost_server_uuid in FULL state after 0 sec PASS 34 (150s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 12:34:11 (1781282051) fail_loc=0x80000317 Race attempt 0 Wait for 44722 44736 for 60 sec... - open/close 661 (time 1781282068.68 total 10.02 last 65.96) - open/close 1351 (time 1781282078.68 total 20.03 last 68.95) - open/close 2062 (time 1781282088.69 total 30.03 last 71.08) - open/close 2861 (time 1781282098.69 total 40.04 last 79.85) - open/close 3559 (time 1781282108.71 total 50.05 last 69.70) total: 4000 open/close in 56.44 seconds: 70.87 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.202.131@tcp:/lustre 7.2G 14M 7.2G 1% /mnt/lustre 192.168.202.131@tcp:/lustre 7.2G 14M 7.2G 1% /mnt/lustre2 PASS 35 (74s) == sanityn test 36: handle ESTALE/open-unlink correctly == 12:35:25 (1781282125) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg231-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.7046 s, 30.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7136 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.76866 s, 29.6 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7136 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.34977 s, 38.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7136 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7524352) after_dd(7467008) after(7518208) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.45794 s, 36.0 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7136 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7518208) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.43576 s, 36.5 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7136 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.38609 s, 37.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7136 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7524352) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.19331 s, 43.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7136 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7520256) after_dd(7471104) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.68509 s, 31.1 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7136 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.19364 s, 43.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7136 sleep 5 for ZFS MDS Waiting for MDT destroys to complete *** cycle(8) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.78838 s, 29.3 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7136 sleep 5 for ZFS MDS Waiting for MDT destroys to complete *** cycle(9) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.45905 s, 35.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7136 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7524352) after_dd(7471104) after(7522304) PASS 36 (325s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 12:40:50 (1781282450) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7136 - create 1283 (time 1781282465.25 total 10.01 last 128.21) - create 2529 (time 1781282475.26 total 20.01 last 124.56) - create 3880 (time 1781282485.27 total 30.02 last 134.95) - create 5112 (time 1781282495.27 total 40.03 last 123.15) - create 6547 (time 1781282505.28 total 50.03 last 143.42) - create 7933 (time 1781282515.28 total 60.04 last 138.53) - create 9219 (time 1781282525.29 total 70.04 last 128.54) total: 10000 create in 75.64 seconds: 132.20 ops/second PASS 37 (88s) == sanityn test 39a: test from 11063 ============================================================================================ 12:42:18 (1781282538) repeat after cancel_lru_locks PASS 39a (8s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 12:42:26 (1781282546) repeat after cancel_lru_locks PASS 39b (9s) == sanityn test 39c: check truncate mtime update ================================================================================ 12:42:35 (1781282555) repeat after cancel_lru_locks PASS 39c (8s) == sanityn test 39d: sync write should update mtime ====== 12:42:44 (1781282564) fail_loc=0x411 fail_loc=0 PASS 39d (6s) == sanityn test 40a: pdirops: create vs others ======================================================================== 12:42:51 (1781282571) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (19s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 12:43:10 (1781282590) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (18s) == sanityn test 40c: pdirops: link and others ======================================================================== 12:43:28 (1781282608) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (19s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 12:43:47 (1781282627) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (17s) == sanityn test 40e: pdirops: rename and others ======================================================================== 12:44:04 (1781282644) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (18s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 12:44:22 (1781282662) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (15s) == sanityn test 41b: pdirops: create vs create ======================================================================== 12:44:38 (1781282678) open: File exists Conflict PASS 41b (15s) == sanityn test 41c: pdirops: create vs link ======================================================================== 12:44:53 (1781282693) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (14s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 12:45:07 (1781282707) Conflict PASS 41d (13s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 12:45:21 (1781282721) Conflict PASS 41e (14s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 12:45:34 (1781282734) Conflict PASS 41f (15s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 12:45:49 (1781282749) Conflict PASS 41g (14s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 12:46:03 (1781282763) Conflict PASS 41h (15s) == sanityn test 41i: reint_open: create vs create ======== 12:46:19 (1781282779) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (1162s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 13:05:41 (1781283941) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (16s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 13:05:57 (1781283957) open: File exists Conflict PASS 42b (17s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 13:06:15 (1781283975) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (16s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 13:06:31 (1781283991) Conflict PASS 42d (14s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 13:06:45 (1781284005) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (15s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 13:07:00 (1781284020) Conflict PASS 42f (16s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 13:07:16 (1781284036) Conflict PASS 42g (15s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 13:07:32 (1781284052) Conflict PASS 42h (15s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 13:07:47 (1781284067) PASS 43a (60s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 13:08:47 (1781284127) Conflict PASS 43b (14s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 13:09:02 (1781284142) Conflict PASS 43c (18s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 13:09:20 (1781284160) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (15s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 13:09:35 (1781284175) Conflict PASS 43e (17s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 13:09:52 (1781284192) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (21s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 13:10:14 (1781284214) stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (14s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 13:10:28 (1781284228) Conflict PASS 43h (13s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 13:10:41 (1781284241) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (4s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 13:10:45 (1781284245) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (150s) == sanityn test 43k: unlink vs create ==================== 13:13:15 (1781284395) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (1353s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 13:35:48 (1781285748) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Fri Jun 12 13:35:57 EDT 2026 PASS 44a (14s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 13:36:02 (1781285762) open: File exists Conflict PASS 44b (16s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 13:36:18 (1781285778) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (15s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 13:36:33 (1781285793) Conflict PASS 44d (15s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 13:36:48 (1781285808) Conflict PASS 44e (18s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 13:37:06 (1781285826) Conflict PASS 44f (17s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 13:37:23 (1781285843) Conflict PASS 44g (14s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 13:37:37 (1781285857) Conflict PASS 44h (14s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 13:37:51 (1781285871) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (4s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 13:37:55 (1781285875) PASS 45a (108s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 13:39:43 (1781285983) Conflict PASS 45b (15s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 13:39:58 (1781285998) Conflict PASS 45c (13s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 13:40:11 (1781286011) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (13s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 13:40:24 (1781286024) Conflict PASS 45e (14s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 13:40:38 (1781286038) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (13s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 13:40:51 (1781286051) stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (13s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 13:41:04 (1781286064) Conflict PASS 45h (13s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 13:41:17 (1781286077) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (4s) == sanityn test 45j: read vs rename ====================== 13:41:21 (1781286081) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (1388s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 14:04:29 (1781287469) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (15s) == sanityn test 46b: pdirops: link vs create ======================================================================== 14:04:44 (1781287484) open: File exists Conflict PASS 46b (17s) == sanityn test 46c: pdirops: link vs link ======================================================================== 14:05:01 (1781287501) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (16s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 14:05:17 (1781287517) Conflict PASS 46d (16s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 14:05:33 (1781287533) Conflict PASS 46e (15s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 14:05:48 (1781287548) Conflict PASS 46f (14s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 14:06:03 (1781287563) Conflict PASS 46g (15s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 14:06:18 (1781287578) Conflict PASS 46h (16s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 14:06:34 (1781287594) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (4s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 14:06:39 (1781287599) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (3s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 14:06:42 (1781287602) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (3s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 14:06:45 (1781287605) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (4s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 14:06:49 (1781287609) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (3s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 14:06:52 (1781287612) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (4s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 14:06:56 (1781287616) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (3s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 14:06:59 (1781287619) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (3s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 14:07:03 (1781287623) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.217743 s, 47.0 kB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (13s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 14:07:15 (1781287635) 0+1 records in 0+1 records out 158 bytes copied, 0.0545805 s, 2.9 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (242482) - No such process PASS 51a (9s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 14:07:24 (1781287644) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0192698 s, 53.1 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.062959 s, 16.7 MB/s 3145728 PASS 51b (20s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 14:07:45 (1781287665) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0207667 s, 49.3 kB/s PASS 51c (12s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 14:07:57 (1781287677) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0544648 s, 19.3 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (8s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 14:08:05 (1781287685) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2436 0x984 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2436 0x984 0 read lease(1) has applied. PASS 51e (9s) == sanityn test 54: rename locking ======================= 14:08:14 (1781287694) llite.lustre-ffff97ba450e7800.xattr_cache=1 llite.lustre-ffff97ba4541b800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504962 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:08:16.000000000 -0400 Modify: 2026-06-12 14:08:16.000000000 -0400 Change: 2026-06-12 14:08:16.000000000 -0400 Birth: 2026-06-12 14:08:17.000000000 -0400 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504965 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:08:22.000000000 -0400 Modify: 2026-06-12 14:08:23.000000000 -0400 Change: 2026-06-12 14:08:23.000000000 -0400 Birth: 2026-06-12 14:08:23.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:08:29.000000000 -0400 Modify: 2026-06-12 14:08:29.000000000 -0400 Change: 2026-06-12 14:08:29.000000000 -0400 Birth: 2026-06-12 14:08:29.000000000 -0400 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504967 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:08:29.000000000 -0400 Modify: 2026-06-12 14:08:30.000000000 -0400 Change: 2026-06-12 14:08:30.000000000 -0400 Birth: 2026-06-12 14:08:29.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:08:35.000000000 -0400 Modify: 2026-06-12 14:08:36.000000000 -0400 Change: 2026-06-12 14:08:36.000000000 -0400 Birth: 2026-06-12 14:08:35.000000000 -0400 File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504970 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:08:35.000000000 -0400 Modify: 2026-06-12 14:08:36.000000000 -0400 Change: 2026-06-12 14:08:36.000000000 -0400 Birth: 2026-06-12 14:08:35.000000000 -0400 PASS 54 (32s) == sanityn test 55a: rename vs unlink target dir ========= 14:08:46 (1781287726) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (14s) == sanityn test 55b: rename vs unlink source dir ========= 14:09:00 (1781287740) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (13s) == sanityn test 55c: rename vs unlink orphan target dir == 14:09:13 (1781287753) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (20s) == sanityn test 55d: rename file vs link ================= 14:09:33 (1781287773) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (16s) == sanityn test 60: Verify data_version behaviour ======== 14:09:49 (1781287789) PASS 60 (7s) == sanityn test 70a: cd directory && rm directory ======== 14:09:57 (1781287797) PASS 70a (7s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 14:10:03 (1781287803) PASS 70b (7s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 14:10:10 (1781287810) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00744363 s, 8.8 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (3s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 14:10:13 (1781287813) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0215554 s, 3.0 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (4s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 14:10:17 (1781287817) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (3s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 14:10:21 (1781287821) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0117534 s, 5.6 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (4s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 14:10:24 (1781287824) llite.lustre-ffff97ba450e7800.xattr_cache=1 llite.lustre-ffff97ba4541b800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (8s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 14:10:33 (1781287833) llite.lustre-ffff97ba450e7800.xattr_cache=1 llite.lustre-ffff97ba4541b800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (10s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 14:10:44 (1781287844) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0140742 s, 72.8 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0452693 s, 22.6 kB/s 252529: get lock2 252530: get lock1 252529: done 252529: get lock1 252530: done 252530: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 252530: cannot get lock2: Resource deadlock avoided 252530: exit rc=0 252529: done 252529: exit rc=0 PASS 74 (13s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 14:10:56 (1781287856) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0654791 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292482 s, 35.9 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 253103 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (12s) == sanityn test 76: Verify MDT open_files listing ======== 14:11:08 (1781287868) opening files: ........ closing files: ........ PASS 76 (80s) == sanityn test 77a: check FIFO NRS policy =============== 14:12:28 (1781287948) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.645833 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0629259 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0727028 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128011 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124972 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170442 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172251 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20706 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206915 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203758 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.243094 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.257178 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241244 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.265894 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.293157 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.308438 s, 3.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff97ba450e7800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.343079 s, 3.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0605517 s, 17.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0627447 s, 16.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0769548 s, 13.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0628741 s, 16.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104683 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0890604 s, 11.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0708729 s, 14.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0683806 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114856 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0869927 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105227 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0831642 s, 12.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139975 s, 7.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109335 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104507 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135519 s, 7.7 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg231-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg231-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (11s) == sanityn test 77b: check CRR-N NRS policy ============== 14:12:39 (1781287959) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.368914 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170189 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160145 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.275531 s, 3.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147799 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.336388 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0734812 s, 14.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171274 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.309739 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.089044 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17035 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.258348 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191253 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.358148 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189014 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147173 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.392416 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00246942 s, 425 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00263596 s, 398 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967853 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00458683 s, 229 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143511 s, 73.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10028 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122348 s, 8.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116796 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0825839 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150508 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183188 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0949416 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189058 s, 55.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0858671 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265768 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20633 s, 5.1 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg231-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg231-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.226191 s, 74.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0848334 s, 12.4 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0832462 s, 12.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0717024 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0562694 s, 18.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0998207 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107119 s, 9.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104043 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121777 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0844684 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125869 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134562 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0858329 s, 12.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143829 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0991356 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0618328 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146993 s, 7.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0347396 s, 30.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307172 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320425 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290148 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303015 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306739 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.036268 s, 28.9 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0383066 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0401542 s, 26.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0455654 s, 23.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0421477 s, 24.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0507838 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.05027 s, 20.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0523838 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0523465 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100409 s, 10.4 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg231-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg231-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (15s) == sanityn test 77c: check ORR NRS policy ================ 14:12:54 (1781287974) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.448434 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0456582 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0826167 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131281 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159837 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132938 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184788 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199713 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206519 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.254423 s, 4.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff97ba450e7800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.2768 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.267496 s, 3.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230814 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228654 s, 4.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245583 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.288452 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800029 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0515224 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294781 s, 35.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0493167 s, 21.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0743633 s, 14.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0734332 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.048691 s, 21.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0548198 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210409 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.391415 s, 2.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855249 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182245 s, 57.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299021 s, 35.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190176 s, 55.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694186 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167465 s, 62.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124302 s, 84.4 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg231-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg231-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.295096 s, 56.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111647 s, 9.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0515417 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136669 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0600542 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141981 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138915 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0624901 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151917 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.2481 s, 4.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.346804 s, 3.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.366019 s, 2.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.352845 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.412348 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.422257 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.428433 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.324792 s, 3.2 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff97ba450e7800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0656249 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265234 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296601 s, 35.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100204 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0704459 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0675242 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0799675 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0894569 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0888118 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117025 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0847296 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104309 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0910124 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101566 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0942935 s, 11.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0949215 s, 11.0 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg231-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg231-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.374113 s, 44.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0489187 s, 21.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0619697 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11828 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0410953 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147002 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162874 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172538 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118392 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172583 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138085 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120566 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109016 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128711 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196318 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181126 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0917794 s, 11.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550507 s, 190 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0656269 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.356458 s, 2.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00934954 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0620284 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589077 s, 178 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0614071 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0615816 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0725547 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0802567 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0761323 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0716162 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0856342 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119346 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0751554 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0844027 s, 12.4 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg231-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg231-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (15s) == sanityn test 77d: check TRR nrs policy ================ 14:13:09 (1781287989) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.515373 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0936137 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0650089 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117967 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139087 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0722248 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142242 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230819 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139255 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106737 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187257 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.262537 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.296719 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193368 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.315536 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0852908 s, 12.3 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff97ba450e7800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103319 s, 10.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0376021 s, 27.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303733 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0874508 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381012 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0670023 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.048121 s, 21.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0571439 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130007 s, 8.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0675113 s, 15.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0666808 s, 15.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0763583 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0709067 s, 14.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0775567 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0817705 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0875857 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0992891 s, 10.6 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg231-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg231-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.428368 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0918169 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0991771 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0803792 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237711 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.261662 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227347 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.250125 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223453 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223827 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.341723 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.295179 s, 3.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.323385 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0722488 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117221 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.419557 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102018 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.063539 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116586 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121484 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00878921 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.605554 s, 1.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.52794 s, 2.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131942 s, 7.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170172 s, 61.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013368 s, 78.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00936965 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166046 s, 63.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105082 s, 99.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20691 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180981 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.506082 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174313 s, 60.2 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg231-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg231-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.565761 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108826 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0465327 s, 22.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11873 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160279 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0987185 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16056 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109337 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216158 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184697 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20742 s, 5.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137815 s, 7.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.299861 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0995163 s, 10.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230766 s, 4.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.305705 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.318758 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147367 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768143 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0513582 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117816 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0834466 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788717 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0735671 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0880363 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0768153 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0789354 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142944 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0561041 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257645 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236653 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0846119 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0846838 s, 12.4 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg231-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg231-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (15s) == sanityn test 77e: check TBF NID nrs policy ============ 14:13:25 (1781288005) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.202.31@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.491694 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122896 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0825878 s, 12.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.266154 s, 3.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.28712 s, 3.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218655 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.290396 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248 s, 4.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218089 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.270172 s, 3.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.31937 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244826 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.299504 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.287702 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.374028 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.298863 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.433582 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149176 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188461 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168602 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0947619 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215714 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0988805 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18427 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197806 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125913 s, 8.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191723 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203491 s, 5.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213056 s, 4.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229736 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149807 s, 7.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155457 s, 6.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173597 s, 6.0 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg231-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg231-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.514984 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0772975 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0502918 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0929494 s, 11.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209912 s, 5.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165121 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191746 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.219775 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.295964 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.27102 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.272026 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.371126 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.3546 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.32313 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.384521 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.362705 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.382328 s, 2.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156587 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106946 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162679 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171055 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208229 s, 5.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18354 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198188 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112723 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122574 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255374 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161312 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.27511 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137794 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155404 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15191 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153062 s, 6.9 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg231-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg231-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.609977 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0758326 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.092244 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141013 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166644 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175819 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.238763 s, 4.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196232 s, 5.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227189 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220975 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.301013 s, 3.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff97ba450e7800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.31631 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.31477 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.281581 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.355893 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.329212 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.449006 s, 2.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014963 s, 70.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143466 s, 73.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0645747 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329815 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254493 s, 41.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0927159 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.054644 s, 19.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0458869 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0593587 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101468 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0680147 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.042537 s, 24.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0574372 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0482492 s, 21.7 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=2 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0943932 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0685886 s, 15.3 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg231-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg231-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.440145 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0884055 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0902351 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0504197 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0775242 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.279788 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.296596 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.325728 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.344928 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.337784 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.356948 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.332598 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.376094 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.359851 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.357489 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.406125 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.458586 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0787172 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0580136 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0845549 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988384 s, 106 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293239 s, 35.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0521188 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00527521 s, 199 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0780672 s, 13.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235565 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00532094 s, 197 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169447 s, 61.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00536297 s, 196 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231134 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15384 s, 6.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0429015 s, 24.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0396859 s, 26.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=3 removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg231-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg231-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (24s) == sanityn test 77f: check TBF JobID nrs policy ========== 14:13:50 (1781288030) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg231-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.543891 s, 30.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0657937 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0660159 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141313 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129127 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23135 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.257012 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183238 s, 5.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff97ba450e7800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203796 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213487 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244313 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.25005 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.329404 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.29333 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.322618 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.331964 s, 3.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.404664 s, 2.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103965 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00520057 s, 202 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0345581 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315401 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676613 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00451171 s, 232 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552342 s, 190 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00529037 s, 198 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239038 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00496326 s, 211 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798076 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00397975 s, 263 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00309605 s, 339 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102989 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0699442 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633385 s, 166 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg231-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg231-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg231-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.549433 s, 30.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100961 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11149 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0870992 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192074 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.259858 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145896 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.276872 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148005 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211209 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.308687 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.335701 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196842 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.333428 s, 3.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff97ba450e7800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227649 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.288 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.299577 s, 3.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0445081 s, 23.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0674541 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284727 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00388908 s, 270 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0508309 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258664 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0791919 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0347472 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0612032 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0545978 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826825 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00232072 s, 452 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00404892 s, 259 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00252188 s, 416 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103559 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.048304 s, 21.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=3 removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg231-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg231-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg231-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.534006 s, 31.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100082 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107997 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197032 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206094 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190417 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.226295 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.267304 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.292627 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.320492 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.32231 s, 3.3 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245841 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.331847 s, 3.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.39137 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.291185 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.380502 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.40839 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328326 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0493708 s, 21.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00418957 s, 250 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00488195 s, 215 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.041207 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00278926 s, 376 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00409969 s, 256 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0567907 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0960796 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0789398 s, 13.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0525283 s, 20.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0596852 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262561 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0545194 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0563411 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208588 s, 50.3 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg231-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg231-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg231-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.51549 s, 32.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0901488 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0973543 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14496 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267506 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139545 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165152 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193363 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163344 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141303 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115002 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.278575 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166356 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.268832 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232602 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14767 s, 7.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.429197 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0789753 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262877 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189019 s, 55.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024157 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029559 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335708 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0354716 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123546 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0418534 s, 25.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0552907 s, 19.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0501425 s, 20.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0523775 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0562857 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0867968 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0664726 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0640768 s, 16.4 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg231-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg231-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (26s) == sanityn test 77g: Change TBF type directly ============ 14:14:15 (1781288055) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (13s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 14:14:28 (1781288068) oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg231-client: oleg231-server: ssh exited with exit code 2 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg231-client: oleg231-server: ssh exited with exit code 12 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg231-client: oleg231-server: ssh exited with exit code 12 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg231-client: oleg231-server: ssh exited with exit code 12 PASS 77h (13s) == sanityn test 77i: Change rank of TBF rule ============= 14:14:41 (1781288081) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (23s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 14:15:04 (1781288104) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.444175 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0812638 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0421947 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107322 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.077962 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0740304 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0831253 s, 12.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138157 s, 7.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11989 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120382 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169866 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.08704 s, 12.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198738 s, 5.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162114 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167557 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.204635 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199379 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0756179 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0660871 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0686819 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.073991 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.079989 s, 13.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0867562 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0857608 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0933581 s, 11.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0980283 s, 10.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0865782 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123836 s, 8.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10831 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106602 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10993 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111163 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114376 s, 9.2 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg231-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg231-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.81599 s, 11.9 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6442 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (70s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 14:16:14 (1781288174) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg231-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.84969 s, 19.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0796604 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107929 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119937 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170544 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215205 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118494 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164392 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232613 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206327 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.364159 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.346095 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.2535 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.389781 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.32027 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.289027 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.423929 s, 2.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0499979 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0443353 s, 23.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0423519 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0664015 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0556451 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0514688 s, 20.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0735228 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0980314 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.080849 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15968 s, 6.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.070848 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0798847 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.09482 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101518 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108798 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143733 s, 7.3 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg231-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg231-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.0879 s, 5.2 MB/s Write runtime is 22 s, speed is 4.545454 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8327 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg231-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.843435 s, 19.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0802128 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128468 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0635 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102355 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150895 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171525 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213455 s, 4.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.278743 s, 3.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.279826 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.276477 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.284743 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264029 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260828 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.24215 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.350575 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.312471 s, 3.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263371 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298071 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299072 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977044 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0678709 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0403445 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0340022 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00234308 s, 448 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603406 s, 174 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635084 s, 165 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0709569 s, 14.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0584695 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0718241 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0561052 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00459671 s, 228 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0630244 s, 16.6 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg231-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg231-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.0218 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.0242 s, 5.2 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (150s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 14:18:44 (1781288324) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg231-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.76762 s, 21.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15752 s, 6.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161083 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124905 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.270249 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.266666 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.321553 s, 3.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.310519 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.415929 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.328913 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.384283 s, 2.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.390761 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.3564 s, 2.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff97ba450e7800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.46473 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.485994 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.486665 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.46841 s, 2.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0482096 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0645438 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.065946 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0777201 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0626672 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0951813 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104652 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120731 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152496 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119876 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0936218 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136226 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0974036 s, 10.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133782 s, 7.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112617 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137149 s, 7.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg231-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg231-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.1482 s, 12.9 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.91843 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.31@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.31@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.643332 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109929 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103433 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120596 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168582 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.25355 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192721 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218176 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.374539 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.343272 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.263769 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.34827 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.386173 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.371038 s, 2.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.538126 s, 1.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.501327 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.576759 s, 1.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff97ba450e7800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129249 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300835 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0806861 s, 13.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109661 s, 9.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14918 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0564866 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138819 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156423 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127449 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108424 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159346 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1041 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162641 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133423 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124742 s, 84.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230028 s, 4.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg231-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg231-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.0326 s, 10.5 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.98213 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.31@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg231-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.598894 s, 28.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126559 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135443 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223226 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.3068 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239779 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.310942 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.278881 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.32135 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.381572 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.258305 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.363331 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.373284 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.491704 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.405501 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.405316 s, 2.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff97ba450e7800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.453834 s, 2.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0933913 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138831 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0678319 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0540989 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145626 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111005 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153819 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106059 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135903 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12618 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134301 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0905424 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110143 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129713 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111391 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115223 s, 9.1 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg231-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg231-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.96503 s, 10.5 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.32874 s, 19.7 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg231-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.553356 s, 30.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0601286 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101897 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175101 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188804 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248255 s, 4.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.221821 s, 4.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.263159 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.277764 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.261949 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.278451 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.309111 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.295993 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.271051 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.296242 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.270826 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.292604 s, 3.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0401301 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608827 s, 172 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0090489 s, 116 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128018 s, 81.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080222 s, 131 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00942941 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859478 s, 122 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106575 s, 9.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134672 s, 77.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10886 s, 9.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0881018 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0986073 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14567 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0824047 s, 12.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0540094 s, 19.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0618033 s, 17.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg231-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg231-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 11.0799 s, 9.5 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.87725 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.5202 s, 12.3 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.0587 s, 10.4 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg231-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.925601 s, 18.1 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.089464 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130588 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115128 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0814425 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0622377 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.054608 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113545 s, 9.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.055711 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0923171 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0415981 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.247306 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.324377 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.414914 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.474915 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.390935 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.356481 s, 2.9 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0527478 s, 19.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0486537 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0449084 s, 23.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.070728 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0606014 s, 17.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0737035 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224004 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0765854 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0831023 s, 12.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0918313 s, 11.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0817292 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0866894 s, 12.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0870273 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216795 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220493 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111044 s, 9.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg231-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg231-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.161 s, 5.2 MB/s Write runtime is 22 s, speed is 4.545454 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.607 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg231-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.457199 s, 36.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0631012 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.071289 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117501 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148383 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165526 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201806 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198399 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203592 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206501 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.225552 s, 4.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245608 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248884 s, 4.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.250226 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248631 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273245 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.300761 s, 3.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0540507 s, 19.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113304 s, 9.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116676 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0698474 s, 15.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130358 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0669081 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0661058 s, 15.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0724959 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0766822 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0838896 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0868277 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.092465 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0975128 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.09788 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102247 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117946 s, 8.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg231-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg231-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.67402 s, 12.1 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.0973 s, 10.4 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg231-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.604742 s, 27.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0961085 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151718 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13233 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178661 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.259859 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.280545 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23683 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.38045 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.346216 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.435997 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.433658 s, 2.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.380936 s, 2.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.332538 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.410746 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.422651 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.397863 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0477705 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109787 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0601252 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0567913 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143503 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0883182 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0832865 s, 12.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0958401 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105355 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104336 s, 10.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11268 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111804 s, 9.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112411 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119261 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134629 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151248 s, 6.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg231-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg231-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 12.279 s, 8.5 MB/s Write runtime is 13 s, speed is 7.692307 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.97158 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 11.423 s, 9.2 MB/s Write runtime is 13 s, speed is 7.692307 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.94992 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (446s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 14:26:10 (1781288770) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (12s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 14:26:22 (1781288782) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg231-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 4.05412 s, 259 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg231-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.0176 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (60s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 14:27:22 (1781288842) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.512027 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0649924 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0993672 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113309 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168138 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132444 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133771 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15832 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18291 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317525 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.29179 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197196 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222825 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.293668 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.233405 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220439 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.256581 s, 4.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0406862 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0405274 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101598 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0865412 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0602461 s, 17.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0659856 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122033 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0984143 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.075208 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.073087 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0864113 s, 12.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0929655 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.087022 s, 12.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123726 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0939661 s, 11.2 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff97ba450e7800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110786 s, 9.5 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg231-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg231-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.69543 s, 10.8 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.34839 s, 19.6 MB/s Read runtime is 7 s, speed is 14.285714 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.539599 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0848159 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0792699 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120045 s, 8.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133897 s, 7.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145808 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.235445 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200334 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.283365 s, 3.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227699 s, 4.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196262 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.225175 s, 4.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.296691 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.29459 s, 3.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.282111 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241099 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.334012 s, 3.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff97ba450e7800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00234909 s, 446 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0650867 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297431 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102536 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00240782 s, 435 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00229295 s, 457 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696934 s, 150 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0738152 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124769 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617183 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128177 s, 8.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130774 s, 80.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0522564 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126253 s, 83.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105285 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0027768 s, 378 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg231-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg231-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 7.84556 s, 13.4 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.21879 s, 20.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (92s) == sanityn test 77o: Changing rank should not panic ====== 14:28:54 (1781288934) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (15s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 14:29:09 (1781288949) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (143s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 14:31:32 (1781289092) mds.MDS.mdt.nrs_policies=tbf oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg231-client: oleg231-server: ssh exited with exit code 36 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg231-client: oleg231-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (40s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 14:32:12 (1781289132) oleg231-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device pdsh@oleg231-client: oleg231-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (10s) == sanityn test 79: xattr: intent error ================== 14:32:22 (1781289142) File: /mnt/lustre/d79.sanityn Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505430 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:32:24.000000000 -0400 Modify: 2026-06-12 14:32:24.000000000 -0400 Change: 2026-06-12 14:32:24.000000000 -0400 Birth: 2026-06-12 14:32:24.000000000 -0400 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (19s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 14:32:41 (1781289161) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (3s) == sanityn test 80b: Accessing directory during migration ========================================================== 14:32:45 (1781289165) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (5s) == sanityn test 81a: rename and stat under striped directory ========================================================== 14:32:49 (1781289169) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (3s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 14:32:53 (1781289173) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (4s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 14:32:57 (1781289177) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (4s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 14:33:01 (1781289181) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7136 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7136 PASS 82 (9s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 14:33:10 (1781289190) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (4s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 14:33:15 (1781289195) fail_loc=0x8000060b PASS 84 (14s) == sanityn test 90: open/create and unlink striped directory ========================================================== 14:33:29 (1781289209) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (3s) == sanityn test 91: chmod and unlink striped directory === 14:33:32 (1781289212) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (4s) == sanityn test 92: create remote directory under orphan directory ========================================================== 14:33:36 (1781289216) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (3s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 14:33:40 (1781289220) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2625 0xa41 0 0 2646 0xa56 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2626 0xa42 0 0 2647 0xa57 0 rc1=0 and rc2=0 PASS 93 (19s) == sanityn test 94: signal vs CP callback race =========== 14:33:58 (1781289238) osc.lustre-OST0000-osc-ffff97ba450e7800.idle_timeout=0 osc.lustre-OST0000-osc-ffff97ba4541b800.idle_timeout=0 osc.lustre-OST0001-osc-ffff97ba450e7800.idle_timeout=0 osc.lustre-OST0001-osc-ffff97ba4541b800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.279903 s, 29.3 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 2.89744 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 282295 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.10593 s, 38.7 kB/s osc.lustre-OST0000-osc-ffff97ba450e7800.idle_timeout=debug osc.lustre-OST0000-osc-ffff97ba4541b800.idle_timeout=debug osc.lustre-OST0001-osc-ffff97ba450e7800.idle_timeout=debug osc.lustre-OST0001-osc-ffff97ba4541b800.idle_timeout=debug PASS 94 (18s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 14:34:16 (1781289256) llite.lustre-ffff97ba450e7800.fast_read=0 llite.lustre-ffff97ba4541b800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.0118156 s, 1.0 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff97ba450e7800.fast_read=1 llite.lustre-ffff97ba4541b800.fast_read=1 PASS 95b (15s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 14:34:31 (1781289271) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (3s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 14:34:34 (1781289274) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0086933 s, 471 kB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (9s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 14:34:43 (1781289283) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.130664 s, 16.0 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (9s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 14:34:52 (1781289292) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.152097 s, 13.8 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (10s) == sanityn test 100e: DoM: read on open and file size ==== 14:35:02 (1781289302) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.00792043 s, 1.5 MB/s PASS 100e (7s) == sanityn test 101a: Discard DoM data on unlink ========= 14:35:10 (1781289310) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00739132 s, 554 kB/s PASS 101a (7s) == sanityn test 101b: Discard DoM data on rename ========= 14:35:16 (1781289316) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00550229 s, 744 kB/s PASS 101b (8s) == sanityn test 101c: Discard DoM data on close-unlink === 14:35:24 (1781289324) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0085492 s, 479 kB/s PASS 101c (11s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 14:35:37 (1781289337) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1781289340 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1781289344 Finishing test test23 at 1781289344 PASS 103 (16s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 14:35:53 (1781289353) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (4s) == sanityn test 105: Glimpse and lock cancel race ======== 14:35:57 (1781289357) fail_loc=0x416 PASS 105 (54s) == sanityn test 106a: Verify the btime via statx() ======= 14:36:52 (1781289412) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (4s) == sanityn test 106b: Glimpse RPCs test for statx ======== 14:36:56 (1781289416) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0216038 s, 47.4 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505458 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:36:59.000000000 -0400 Modify: 2026-06-12 14:36:59.000000000 -0400 Change: 2026-06-12 14:36:59.000000000 -0400 Birth: 2026-06-12 14:36:59.000000000 -0400 /mnt/lustre/f106b.sanityn 144115205272505458 -rw-r--r-- 1 0 0 1781289419 1781289419 1781289419 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505458 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 14:36:59.000000000 -0400 Modify: 2026-06-12 14:36:59.000000000 -0400 Change: 2026-06-12 14:36:59.000000000 -0400 Birth: 2026-06-12 14:36:59.000000000 -0400 1781289419 1024 1 PASS 106b (10s) == sanityn test 106c: Verify statx attributes mask ======= 14:37:06 (1781289426) PASS 106c (9s) == sanityn test 107a: Basic grouplock conflict =========== 14:37:15 (1781289435) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.455693 s, 23.0 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2632 0xa48 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7136 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7136 First grouplock blocks second one PASS 107a (11s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 14:37:27 (1781289447) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.551311 s, 19.0 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2654 0xa5e 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7136 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (14s) == sanityn test 108a: lseek: parallel updates ============ 14:37:41 (1781289461) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.0129458 s, 772 kB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 5.18281 s, 1.6 MB/s PASS 108a (14s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 14:37:55 (1781289475) 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre 192.168.202.131@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre2 PASS 109 (1457s) == sanityn test 110: do not grant another lock on resend ========================================================== 15:02:12 (1781290932) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-12 15:02:14.000000000 -0400 Modify: 2026-06-12 15:02:14.000000000 -0400 Change: 2026-06-12 15:02:14.000000000 -0400 Birth: 2026-06-12 15:02:14.000000000 -0400 fail_loc=0 fail_val=0 Connected clients: oleg231-client.virtnet oleg231-client.virtnet PASS 110 (44s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 15:02:56 (1781290976) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (4s) == sanityn test 112: update max-inherit in default LMV === 15:03:00 (1781290980) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (5s) == sanityn test 113: check servers of specified fs ======= 15:03:05 (1781290985) lustre-OST0000-osc-ffff97ba608e8800 active. lustre-OST0001-osc-ffff97ba608e8800 active. lustre-OST0000-osc-ffff97ba608e8800 active. lustre-OST0001-osc-ffff97ba608e8800 active. MGC192.168.202.131@tcp active. PASS 113 (9s) cleanup: ====================================================== == sanityn test complete, duration 10032 sec ============= 15:03:17 (1781290997) Stopping clients: oleg231-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg231-client.virtnet /mnt/lustre2 opts: