-----============= acceptance-small: sanityn ============----- Wed Oct 22 04:22:14 EDT 2025 oleg415-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Started clients oleg415-client.virtnet: 192.168.204.115@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt) oleg415-client.virtnet: executing check_config_client /mnt/lustre oleg415-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg415-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8fb4c84a4000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8fb4c8790000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8fb4c84a4000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8fb4c8790000.idle_timeout=debug disable quota as required oleg415-server: oleg415-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128449 s, 81.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f7132] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 04:22:28 (1761121348) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (2s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 04:22:30 (1761121350) -rw-r--r-- 1 root root 0 Oct 22 04:22 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (3s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 04:22:33 (1761121353) -rw-r--r-- 1 root root 0 Oct 22 04:22 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (2s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 04:22:35 (1761121355) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (3s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 04:22:38 (1761121358) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (2s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 04:22:40 (1761121360) total 1 -rwxrwxrwx 1 root root 0 Oct 22 04:22 f2a -rwxrwxrwx 1 root root 0 Oct 22 04:22 f2b total 1 -rwxrwxrwx 1 root root 0 Oct 22 04:22 f2a -rwxrwxrwx 1 root root 0 Oct 22 04:22 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.0036349 s, 141 kB/s PASS 2e (2s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 04:22:42 (1761121362) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (2s) == sanityn test 2g: check blocks update on sync write ==== 04:22:44 (1761121364) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.102997 s, 20.4 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (2s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 04:22:46 (1761121366) PASS 3 (3s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 04:22:49 (1761121369) PASS 4 (4s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 04:22:53 (1761121373) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (2s) == sanityn test 6: remove of open file on other node ============================================================================ 04:22:55 (1761121375) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (3s) == sanityn test 7: remove of open directory on other node ======================================================================= 04:22:58 (1761121378) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (2s) == sanityn test 8: remove of open special file on other node ==================================================================== 04:23:00 (1761121380) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (3s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 04:23:03 (1761121383) PASS 9a (2s) == sanityn test 9b: append to striped sparse file ======== 04:23:05 (1761121385) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175382 s, 59.8 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.0194473 s, 0.2 kB/s Data read (expecting 'foo'): foo PASS 9b (3s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 04:23:08 (1761121388) 1+0 records in 1+0 records out 1 byte copied, 0.00196663 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00283951 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0030828 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00264883 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00293588 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00230216 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00209265 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00217851 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00214749 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00205813 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00384066 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00233155 s, 0.4 kB/s PASS 10a (3s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 04:23:11 (1761121391) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.0036441 s, 843 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0104645 s, 391 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.000440137 s, 7.0 MB/s PASS 10b (2s) == sanityn test 11: execution of file opened for write should return error ============================================================== 04:23:13 (1761121393) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (3s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 04:23:16 (1761121396) warning: '-runtime' deprecated, use '-t runtime' instead - link 1458 (time 1761121406.93 total 10.01 last 145.68) - link 2731 (time 1761121416.93 total 20.01 last 127.28) - link 3900 (time 1761121426.94 total 30.02 last 116.81) - link 5477 (time 1761121436.94 total 40.02 last 157.66) - link 7261 (time 1761121446.95 total 50.02 last 178.33) - link 8758 (time 1761121456.95 total 60.03 last 149.62) - link 10000 (time 1761121464.09 total 67.17 last 173.96) - link 11924 (time 1761121474.10 total 77.17 last 192.31) - link 13757 (time 1761121484.10 total 87.17 last 183.28) - link 15532 (time 1761121494.10 total 97.18 last 177.43) total: 16092 link in 100.00 seconds: 160.92 ops/second using seed 1021431020 running for 100 seconds - stat 10000 (time 1761121402 ; total 5 ; last 5) - stat 20000 (time 1761121407 ; total 10 ; last 5) - stat 30000 (time 1761121412 ; total 15 ; last 5) - stat 40000 (time 1761121416 ; total 19 ; last 4) - stat 50000 (time 1761121421 ; total 24 ; last 5) - stat 60000 (time 1761121426 ; total 29 ; last 5) - stat 70000 (time 1761121431 ; total 34 ; last 5) - stat 80000 (time 1761121435 ; total 38 ; last 4) - stat 90000 (time 1761121440 ; total 43 ; last 5) - stat 100000 (time 1761121445 ; total 48 ; last 5) - stat 110000 (time 1761121452 ; total 55 ; last 7) - stat 119013 (time 1761121463 ; total 66 ; last 11) - stat 120000 (time 1761121464 ; total 67 ; last 1) - stat 129998 (time 1761121475 ; total 78 ; last 11) - stat 130000 (time 1761121475 ; total 78 ; last 0) - stat 139799 (time 1761121486 ; total 89 ; last 11) - stat 140000 (time 1761121486 ; total 89 ; last 0) - stat 149684 (time 1761121497 ; total 100 ; last 11) total: 149685 stats in 100 seconds: 1496.849976 stats/second - unlinked 0 (time 1761121447 ; total 0 ; last 0) - unlinked 10000 (time 1761121509 ; total 62 ; last 62) unlink(/mnt/lustre2/lockdir/lockfile16092) error: No such file or directory total: 16091 unlinks in 76 seconds: 211.723679 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (19689) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (19691) - No such process PASS 12 (128s) == sanityn test 13: test directory page revocation ======= 04:25:25 (1761121525) f13 f13 PASS 13 (3s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 04:25:27 (1761121527) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 20859 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 04:25:29 (1761121529) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 21431 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (2s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 04:25:31 (1761121531) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 22003 Terminated $DIR1/$tdir/sleep 60 PASS 14b (3s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 04:25:34 (1761121534) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 22578 Terminated $DIR1/$tdir/sleep 60 PASS 14c (2s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 04:25:36 (1761121536) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 23153 Terminated $DIR1/$tdir/sleep 60 PASS 14d (3s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 04:25:39 (1761121539) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7522304 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7522304kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (12s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 04:25:51 (1761121551) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0867975 s, 24.2 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0833078 s, 25.2 MB/s Chance of close/open is 1 in 50 Seed set to 1552 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[1] 1761121552.495002 mapwrite 0x2c3ac1 thru 0x2d1549 (0xda89 bytes) truncating to largest ever: 0x1a68e1 000002[0] 1761121552.546250 trunc from 0x2d154a to 0x1a68e0 (0x12ac69 bytes) 000003[1] 1761121552.554024 write 0x931e6d thru 0x93e566 (0xc6fa bytes) 000005[1] 1761121552.567642 mapwrite 0x0b6354 thru 0x0c1287 (0xaf34 bytes) truncating to largest ever: 0x3c5bec 000006[0] 1761121552.596593 trunc from 0x93e567 to 0x3c5beb (0x57897b bytes) truncating to largest ever: 0x916042 000007[0] 1761121552.614798 trunc from 0x3c5bec to 0x916041 (0x550456 bytes) 000008[0] 1761121552.627048 read 0x14266e thru 0x14809b (0x5a2e bytes) 000010[0] 1761121552.636732 mapread 0x61be94 thru 0x62abed (0xed5a bytes) 000012[0] 1761121552.648591 write 0x143b7b thru 0x14deb1 (0xa337 bytes) 000013[1] 1761121552.655592 read 0x77a846 thru 0x77ff3a (0x56f5 bytes) 000014[1] 1761121552.666771 mapwrite 0x42ac0b thru 0x43a340 (0xf736 bytes) 000015[0] 1761121552.700678 trunc from 0x916042 to 0x6529a8 (0x2c3699 bytes) 000019[1] 1761121552.724105 read 0x602f1b thru 0x610dc1 (0xdea7 bytes) 000021[1] 1761121552.741180 write 0x07d052 thru 0x081366 (0x4315 bytes) 000022[1] 1761121552.751986 mapread 0x607142 thru 0x61407b (0xcf3a bytes) 000023[1] 1761121552.782850 mapwrite 0x332120 thru 0x33b641 (0x9522 bytes) 000027[1] 1761121552.834193 read 0x3d4b59 thru 0x3da591 (0x5a39 bytes) 000028[0] 1761121552.854029 mapread 0x488b2f thru 0x488fb1 (0x483 bytes) 000029[1] 1761121552.865516 mapread 0x34769a thru 0x356cf1 (0xf658 bytes) 000030[0] 1761121552.886425 write 0x948601 thru 0x9570d5 (0xead5 bytes) 000032[1] 1761121552.898781 mapwrite 0x850967 thru 0x85b24c (0xa8e6 bytes) 000033[0] 1761121552.994384 mapwrite 0x485f69 thru 0x490910 (0xa9a8 bytes) 000034[0] 1761121553.021122 write 0x92cc4e thru 0x92cddd (0x190 bytes) 000035[1] 1761121553.027743 mapread 0x1a3958 thru 0x1a98ed (0x5f96 bytes) 000039[1] 1761121553.064197 mapwrite 0x879bbd thru 0x88932a (0xf76e bytes) 000040[0] 1761121553.099352 mapread 0x7f2278 thru 0x7f6ac5 (0x484e bytes) 000041[1] 1761121553.111163 read 0x7d0d21 thru 0x7d81a6 (0x7486 bytes) 000042[0] 1761121553.111813 write 0x6a3f83 thru 0x6abb46 (0x7bc4 bytes) 000043[0] 1761121553.122153 mapwrite 0x821175 thru 0x82ffa9 (0xee35 bytes) 000045[0] 1761121553.152269 mapwrite 0x27e8d6 thru 0x286c4d (0x8378 bytes) 000046[0] 1761121553.171876 write 0x99b12d thru 0x9a3e0a (0x8cde bytes) 000047[1] 1761121553.177484 mapwrite 0x938a05 thru 0x93922f (0x82b bytes) 000048[0] 1761121553.203616 trunc from 0x9a3e0b to 0x502a89 (0x4a1381 bytes) 000049[0] 1761121553.217192 read 0x2e31da thru 0x2f06fb (0xd522 bytes) 000050[1] 1761121553.241528 write 0x59cf4b thru 0x5a7fda (0xb090 bytes) 000051[0] 1761121553.250487 mapread 0x55af0a thru 0x56240e (0x7505 bytes) 000052[1] 1761121553.269733 read 0x55db7b thru 0x56c059 (0xe4df bytes) 000054[1] 1761121553.282480 mapwrite 0x6bf418 thru 0x6cc690 (0xd279 bytes) 000056[1] 1761121553.329738 mapwrite 0x7b751a thru 0x7b96ed (0x21d4 bytes) 000057[1] 1761121553.353126 read 0x519af5 thru 0x52371c (0x9c28 bytes) 000058[0] 1761121553.368038 read 0x4626f7 thru 0x462e18 (0x722 bytes) 000059[1] 1761121553.384637 trunc from 0x7b96ee to 0x0efad5 (0x6c9c18 bytes) 000062[0] 1761121553.408776 trunc from 0x0efad6 to 0x6509bf (0x560eea bytes) 000064[1] 1761121553.429780 mapwrite 0x4853ee thru 0x489f84 (0x4b97 bytes) 000065[0] 1761121553.461323 mapwrite 0x988a87 thru 0x99245c (0x99d6 bytes) 000069[1] 1761121553.528536 mapwrite 0x115abd thru 0x11da94 (0x7fd8 bytes) 000070[0] 1761121553.569883 mapwrite 0x002a4a thru 0x00f4c2 (0xca79 bytes) 000071[0] 1761121553.609689 read 0x0111fb thru 0x019995 (0x879b bytes) 000074[0] 1761121553.632661 mapwrite 0x587ff0 thru 0x596d68 (0xed79 bytes) 000081[0] 1761121553.719859 mapwrite 0x925b42 thru 0x931756 (0xbc15 bytes) 000082[0] 1761121553.740748 write 0x243a1e thru 0x24d143 (0x9726 bytes) 000084[0] 1761121553.750422 write 0x943b8c thru 0x94bbc3 (0x8038 bytes) 000085[0] 1761121553.751141 mapread 0x5c66a3 thru 0x5cde4a (0x77a8 bytes) 000089[1] 1761121553.752171 trunc from 0x99245d to 0x8820aa (0x1103b2 bytes) 000090[0] 1761121553.782972 mapread 0x1234df thru 0x12acf0 (0x7812 bytes) 000093[0] 1761121553.809582 mapread 0x2d5b26 thru 0x2d873e (0x2c19 bytes) 000094[1] 1761121553.820615 mapread 0x72e93e thru 0x73c201 (0xd8c4 bytes) 000095[1] 1761121553.840799 read 0x5c8761 thru 0x5d1711 (0x8fb1 bytes) 000097[1] 1761121553.863431 read 0x58185c thru 0x58fd2f (0xe4d4 bytes) 000100[1] 1761121553.868802 mapread 0x2d34dd thru 0x2dee71 (0xb995 bytes) 000101[0] 1761121553.884314 mapread 0x341158 thru 0x342ef2 (0x1d9b bytes) 000102[0] 1761121553.887081 mapread 0x3d36aa thru 0x3e31a5 (0xfafc bytes) 000103[0] 1761121553.909137 mapwrite 0x1a2269 thru 0x1ab2e5 (0x907d bytes) 000105[0] 1761121553.942986 read 0x85d5b9 thru 0x861ed1 (0x4919 bytes) 000107[0] 1761121553.950524 trunc from 0x8820ab to 0x53faf7 (0x3425b3 bytes) 000110[0] 1761121553.967986 mapread 0x382cef thru 0x390f82 (0xe294 bytes) 000116[0] 1761121554.012276 mapread 0x49a677 thru 0x4aa3be (0xfd48 bytes) 000118[0] 1761121554.031449 trunc from 0x53faf8 to 0x0c2f8d (0x47cb6a bytes) 000122[0] 1761121554.050524 mapread 0x078c31 thru 0x082013 (0x93e3 bytes) truncating to largest ever: 0x9a94a6 000124[0] 1761121554.072751 trunc from 0x0c2f8e to 0x9a94a5 (0x8e6518 bytes) 000127[1] 1761121554.086714 mapwrite 0x1f7ccf thru 0x2076e3 (0xfa15 bytes) 000128[1] 1761121554.152903 mapread 0x087d73 thru 0x090623 (0x88b1 bytes) 000130[1] 1761121554.174825 mapwrite 0x8915ba thru 0x8a0ace (0xf515 bytes) 000131[0] 1761121554.196539 read 0x109bfe thru 0x10f9af (0x5db2 bytes) 000133[1] 1761121554.215398 read 0x526f52 thru 0x529b25 (0x2bd4 bytes) 000134[1] 1761121554.244793 mapread 0x6f9727 thru 0x6fed6e (0x5648 bytes) 000135[1] 1761121554.250674 trunc from 0x9a94a6 to 0x5f04b9 (0x3b8fec bytes) 000137[0] 1761121554.272629 mapwrite 0x7b76c4 thru 0x7c3bf1 (0xc52e bytes) 000138[0] 1761121554.333618 trunc from 0x7c3bf2 to 0x3d73db (0x3ec816 bytes) 000139[0] 1761121554.342854 write 0x9d053c thru 0x9d6b5d (0x6622 bytes) 000140[0] 1761121554.345084 mapwrite 0x8d9d92 thru 0x8dc448 (0x26b7 bytes) 000145[0] 1761121554.384692 read 0x2d54af thru 0x2d98b6 (0x4408 bytes) 000146[1] 1761121554.385080 mapread 0x0e6df9 thru 0x0e802d (0x1235 bytes) 000147[0] 1761121554.415583 trunc from 0x9d6b5e to 0x1d8025 (0x7feb38 bytes) 000148[1] 1761121554.427801 mapwrite 0x056ef8 thru 0x05b85f (0x4968 bytes) 000150[0] 1761121554.466349 mapread 0x054d32 thru 0x059b2f (0x4dfe bytes) 000151[0] 1761121554.477838 read 0x16d4cd thru 0x17c705 (0xf239 bytes) 000152[1] 1761121554.483472 write 0x5e3ffc thru 0x5e56df (0x16e4 bytes) 000155[1] 1761121554.493354 read 0x3514fc thru 0x35db8c (0xc691 bytes) 000157[0] 1761121554.519873 trunc from 0x5e56e0 to 0x5b637f (0x2f360 bytes) 000165[1] 1761121554.551166 read 0x0cef7c thru 0x0cf082 (0x107 bytes) 000166[0] 1761121554.561823 trunc from 0x5b6380 to 0x225a98 (0x3908e7 bytes) 000169[1] 1761121554.571401 trunc from 0x225a99 to 0x66be5c (0x4463c4 bytes) 000172[0] 1761121554.584058 trunc from 0x66be5d to 0x5d614f (0x95d0d bytes) 000173[1] 1761121554.593865 mapwrite 0x2245b8 thru 0x22d611 (0x905a bytes) 000176[1] 1761121554.637378 read 0x050167 thru 0x05e56d (0xe407 bytes) 000181[1] 1761121554.667160 read 0x2afe88 thru 0x2bb34e (0xb4c7 bytes) 000182[0] 1761121554.677561 read 0x1a7d12 thru 0x1b6b2f (0xee1e bytes) 000185[0] 1761121554.694881 mapwrite 0x912b68 thru 0x915e28 (0x32c1 bytes) 000187[0] 1761121554.772493 read 0x6e57c9 thru 0x6f1fae (0xc7e6 bytes) 000188[0] 1761121554.772987 mapread 0x63da93 thru 0x648156 (0xa6c4 bytes) 000189[0] 1761121554.777859 read 0x621410 thru 0x622d52 (0x1943 bytes) 000190[0] 1761121554.781980 mapwrite 0x2469b1 thru 0x2473c3 (0xa13 bytes) 000193[1] 1761121554.803879 trunc from 0x915e29 to 0x0c4519 (0x85190f bytes) 000194[0] 1761121554.824413 read 0x04dde2 thru 0x05acef (0xcf0e bytes) 000195[1] 1761121554.838543 trunc from 0x0c451a to 0x370ce0 (0x2ac7c7 bytes) 000196[0] 1761121554.852061 mapwrite 0x05bd5a thru 0x06181e (0x5ac5 bytes) 000198[0] 1761121554.893539 trunc from 0x370ce1 to 0x150d4c (0x21ff94 bytes) 000205[0] 1761121554.923556 trunc from 0x150d4d to 0x7e06b9 (0x68f96d bytes) 000209[1] 1761121554.947872 mapread 0x489caf thru 0x499bb3 (0xff05 bytes) 000212[1] 1761121554.979136 mapread 0x4a36a1 thru 0x4ad08e (0x99ee bytes) 000213[0] 1761121554.994393 write 0x0eaaff thru 0x0f00cb (0x55cd bytes) 000214[1] 1761121555.039353 trunc from 0x7e06ba to 0x79287b (0x4de3e bytes) 000217[1] 1761121555.065002 read 0x4878b6 thru 0x48fd85 (0x84d0 bytes) 000219[0] 1761121555.088365 mapwrite 0x0a2773 thru 0x0b0478 (0xdd06 bytes) 000221[1] 1761121555.138497 read 0x4ab1ab thru 0x4aedba (0x3c10 bytes) 000225[0] 1761121555.159918 mapwrite 0x7a9329 thru 0x7b1c89 (0x8961 bytes) 000226[0] 1761121555.206988 trunc from 0x7b1c8a to 0x0d8db8 (0x6d8ed1 bytes) 000227[0] 1761121555.216932 read 0x06762d thru 0x071f2a (0xa8fe bytes) 000230[1] 1761121555.231745 read 0x060c2b thru 0x06bc98 (0xb06e bytes) 000232[1] 1761121555.248255 read 0x09b28b thru 0x0a2832 (0x75a8 bytes) 000234[0] 1761121555.249907 mapread 0x0c34a2 thru 0x0cd06f (0x9bce bytes) 000235[1] 1761121555.262780 mapwrite 0x88692b thru 0x890f28 (0xa5fe bytes) 000236[1] 1761121555.306819 write 0x0da783 thru 0x0e594b (0xb1c9 bytes) 000237[0] 1761121555.315688 write 0x2fb8b7 thru 0x30463e (0x8d88 bytes) 000238[0] 1761121555.343586 mapread 0x279634 thru 0x285828 (0xc1f5 bytes) 000239[0] 1761121555.370566 read 0x0fc395 thru 0x10a95f (0xe5cb bytes) 000240[0] 1761121555.377965 trunc from 0x890f29 to 0x83de88 (0x530a0 bytes) 000242[0] 1761121555.385938 mapwrite 0x2fd1ed thru 0x301709 (0x451d bytes) 000243[1] 1761121555.406732 trunc from 0x83de89 to 0x33d24b (0x500c3d bytes) 000244[1] 1761121555.424682 read 0x0174cf thru 0x02019f (0x8cd1 bytes) 000245[0] 1761121555.448426 write 0x2c1f82 thru 0x2cfae4 (0xdb63 bytes) 000248[0] 1761121555.468990 mapwrite 0x87cc75 thru 0x87f4fe (0x288a bytes) 000249[1] 1761121555.504438 read 0x427524 thru 0x427e45 (0x922 bytes) 000253[1] 1761121555.530321 write 0x3bcfe9 thru 0x3bee9a (0x1eb2 bytes) 000254[0] 1761121555.560641 trunc from 0x87f4ff to 0x2e7791 (0x597d6d bytes) 000257[1] 1761121555.582428 read 0x1907cd thru 0x19ce8a (0xc6be bytes) 000260[0] 1761121555.595134 mapread 0x06916f thru 0x0700bb (0x6f4d bytes) 000261[0] 1761121555.608715 mapwrite 0x008291 thru 0x00ae5b (0x2bcb bytes) 000262[1] 1761121555.630847 mapwrite 0x0022bf thru 0x00aa52 (0x8794 bytes) 000263[0] 1761121555.669092 read 0x292d50 thru 0x29e880 (0xbb31 bytes) 000265[1] 1761121555.680997 trunc from 0x2e7792 to 0x68688c (0x39f0fb bytes) 000266[1] 1761121555.694568 read 0x3e1870 thru 0x3e887c (0x700d bytes) 000270[0] 1761121555.724853 trunc from 0x68688d to 0x46e93c (0x217f50 bytes) 000272[1] 1761121555.743482 read 0x2aff63 thru 0x2bee95 (0xef33 bytes) 000274[1] 1761121555.757334 write 0x6b74fc thru 0x6c0cf3 (0x97f8 bytes) 000277[1] 1761121555.771311 mapwrite 0x885afb thru 0x890143 (0xa649 bytes) 000282[1] 1761121555.844044 trunc from 0x890144 to 0x5cea81 (0x2c16c2 bytes) 000283[0] 1761121555.855862 mapwrite 0x544d0d thru 0x5507fe (0xbaf2 bytes) 000284[1] 1761121555.897821 mapwrite 0x5fbc12 thru 0x601dda (0x61c9 bytes) 000287[1] 1761121555.944938 read 0x0afb22 thru 0x0b02cd (0x7ac bytes) 000289[0] 1761121555.983679 read 0x0716a1 thru 0x07ccf8 (0xb658 bytes) 000290[0] 1761121555.995953 write 0x52fa09 thru 0x53075f (0xd57 bytes) 000291[1] 1761121556.033426 read 0x14676b thru 0x14afba (0x4850 bytes) 000295[0] 1761121556.072143 mapread 0x55f6c8 thru 0x56bf14 (0xc84d bytes) 000296[0] 1761121556.086853 mapwrite 0x72e5e6 thru 0x73bf5e (0xd979 bytes) 000298[1] 1761121556.136308 write 0x3ed125 thru 0x3f00b3 (0x2f8f bytes) 000302[1] 1761121556.156403 mapread 0x6a53b9 thru 0x6aaf82 (0x5bca bytes) 000304[1] 1761121556.176054 trunc from 0x73bf5f to 0x720930 (0x1b62e bytes) 000305[0] 1761121556.184934 write 0x5ac4ce thru 0x5af970 (0x34a3 bytes) 000307[0] 1761121556.199565 mapwrite 0x30772f thru 0x3124d4 (0xada6 bytes) 000310[1] 1761121556.242523 trunc from 0x720931 to 0x040c2c (0x6dfd04 bytes) 000312[0] 1761121556.251012 write 0x18d315 thru 0x194d45 (0x7a31 bytes) 000313[0] 1761121556.258896 read 0x0c4976 thru 0x0c9ff4 (0x567f bytes) 000314[0] 1761121556.271159 mapread 0x13428c thru 0x13d8a9 (0x961e bytes) 000315[0] 1761121556.275448 read 0x06c468 thru 0x07800c (0xbba5 bytes) 000318[1] 1761121556.287896 read 0x11755f thru 0x118305 (0xda7 bytes) 000319[0] 1761121556.303071 trunc from 0x194d46 to 0x899049 (0x704304 bytes) 000321[0] 1761121556.322478 read 0x50763f thru 0x507c10 (0x5d2 bytes) 000323[0] 1761121556.355072 write 0x0c14fb thru 0x0c9d73 (0x8879 bytes) 000324[1] 1761121556.362082 mapwrite 0x25f8d2 thru 0x26ac86 (0xb3b5 bytes) 000325[0] 1761121556.408411 mapwrite 0x126196 thru 0x128855 (0x26c0 bytes) 000326[0] 1761121556.432196 trunc from 0x89904a to 0x4f2ef4 (0x3a6155 bytes) 000329[1] 1761121556.457896 read 0x0c58ba thru 0x0cbede (0x6625 bytes) 000331[1] 1761121556.469858 read 0x306366 thru 0x314554 (0xe1ef bytes) 000332[0] 1761121556.493013 mapwrite 0x7565f6 thru 0x75f737 (0x9142 bytes) 000335[0] 1761121556.543868 trunc from 0x75f738 to 0x34f004 (0x410733 bytes) 000336[1] 1761121556.549291 write 0x68836a thru 0x694b06 (0xc79d bytes) 000338[1] 1761121556.560797 write 0x9769c9 thru 0x97a9ba (0x3ff2 bytes) 000339[1] 1761121556.566079 trunc from 0x97a9bb to 0x126e20 (0x853b9a bytes) 000340[0] 1761121556.576289 write 0x420717 thru 0x4216cf (0xfb9 bytes) 000345[1] 1761121556.603945 read 0x2323c6 thru 0x23ce21 (0xaa5c bytes) 000346[0] 1761121556.627749 mapread 0x06ed6c thru 0x075a43 (0x6cd8 bytes) 000349[1] 1761121556.650560 read 0x09a51e thru 0x0a13f0 (0x6ed3 bytes) 000350[0] 1761121556.665185 mapwrite 0x78a04b thru 0x78a53c (0x4f2 bytes) 000352[1] 1761121556.703442 trunc from 0x78a53d to 0x187186 (0x6033b6 bytes) 000353[1] 1761121556.716933 trunc from 0x187187 to 0x676fed (0x4efe67 bytes) 000354[1] 1761121556.724077 mapwrite 0x867dff thru 0x86af29 (0x312b bytes) 000355[1] 1761121556.778501 mapread 0x5cb62e thru 0x5d0db0 (0x5783 bytes) 000356[0] 1761121556.782785 mapwrite 0x568859 thru 0x56e9db (0x6183 bytes) 000358[0] 1761121556.821656 mapread 0x3ad45b thru 0x3b4f10 (0x7ab6 bytes) 000360[1] 1761121556.841028 mapread 0x413d5f thru 0x41f46d (0xb70f bytes) 000363[1] 1761121556.870313 trunc from 0x86af2a to 0x960380 (0xf5457 bytes) 000366[0] 1761121556.887928 read 0x40960d thru 0x4121af (0x8ba3 bytes) 000367[0] 1761121556.906916 write 0x056894 thru 0x058c8e (0x23fb bytes) 000370[0] 1761121556.925773 read 0x8a9ffb thru 0x8ac106 (0x210c bytes) 000374[1] 1761121556.947626 trunc from 0x960381 to 0x2d4fd1 (0x68b3af bytes) 000377[1] 1761121556.964090 trunc from 0x2d4fd2 to 0x74baa5 (0x476ad4 bytes) 000378[0] 1761121556.971335 mapwrite 0x031bf4 thru 0x03ffbb (0xe3c8 bytes) 000379[0] 1761121557.023331 trunc from 0x74baa6 to 0x0cc1dc (0x67f8c9 bytes) 000384[0] 1761121557.042681 write 0x505158 thru 0x505229 (0xd2 bytes) 000386[1] 1761121557.048988 write 0x8750e4 thru 0x880929 (0xb846 bytes) 000387[1] 1761121557.057480 trunc from 0x88092a to 0x75358a (0x12d39f bytes) 000391[1] 1761121557.080814 trunc from 0x75358b to 0x1e7593 (0x56bff7 bytes) 000392[1] 1761121557.087008 trunc from 0x1e7594 to 0x730e68 (0x5498d5 bytes) 000393[0] 1761121557.092893 trunc from 0x730e69 to 0x4532d8 (0x2ddb90 bytes) 000394[0] 1761121557.103439 mapread 0x3a267e thru 0x3b0db0 (0xe733 bytes) 000395[0] 1761121557.121864 mapwrite 0x6b82db thru 0x6b95dc (0x1302 bytes) 000397[1] 1761121557.143204 mapread 0x0ca5cd thru 0x0ce07a (0x3aae bytes) 000399[1] 1761121557.162933 mapwrite 0x7a85d6 thru 0x7b4d52 (0xc77d bytes) 000401[1] 1761121557.213583 read 0x2acea2 thru 0x2b81e4 (0xb343 bytes) 000402[1] 1761121557.254356 read 0x64f587 thru 0x65699c (0x7416 bytes) 000405[1] 1761121557.259560 mapread 0x62511e thru 0x62d7e1 (0x86c4 bytes) 000406[0] 1761121557.264404 mapwrite 0x4bb01b thru 0x4bd9e9 (0x29cf bytes) 000407[1] 1761121557.291377 mapread 0x1c1d5e thru 0x1cad5e (0x9001 bytes) 000408[0] 1761121557.308565 mapwrite 0x725d80 thru 0x72ba85 (0x5d06 bytes) 000410[1] 1761121557.346409 mapwrite 0x320906 thru 0x326458 (0x5b53 bytes) 000412[1] 1761121557.384580 read 0x73847e thru 0x73de8c (0x5a0f bytes) 000413[1] 1761121557.394413 read 0x241052 thru 0x24ad16 (0x9cc5 bytes) 000415[1] 1761121557.438460 read 0x12ff89 thru 0x13d816 (0xd88e bytes) 000417[0] 1761121557.451092 write 0x62b9aa thru 0x62f27d (0x38d4 bytes) 000418[1] 1761121557.463494 write 0x25f8b5 thru 0x26513b (0x5887 bytes) 000420[1] 1761121557.471476 write 0x619f51 thru 0x6212e0 (0x7390 bytes) 000422[1] 1761121557.480505 mapwrite 0x388614 thru 0x38e5ef (0x5fdc bytes) 000424[0] 1761121557.500840 trunc from 0x7b4d53 to 0x303a07 (0x4b134b bytes) 000425[1] 1761121557.518393 mapread 0x0be46c thru 0x0c02df (0x1e74 bytes) 000427[1] 1761121557.536546 read 0x2c5aef thru 0x2d4b59 (0xf06b bytes) 000429[0] 1761121557.548479 write 0x4f0a3b thru 0x4f7d21 (0x72e7 bytes) 000430[1] 1761121557.553498 trunc from 0x4f7d22 to 0x8b7895 (0x3bfb74 bytes) 000432[1] 1761121557.568498 read 0x0cf9ce thru 0x0dd78d (0xddc0 bytes) 000433[0] 1761121557.609047 mapwrite 0x815a23 thru 0x824614 (0xebf2 bytes) 000434[1] 1761121557.667425 trunc from 0x8b7896 to 0x4027f8 (0x4b509d bytes) 000437[1] 1761121557.686269 mapread 0x3810f4 thru 0x38a1cb (0x90d8 bytes) 000438[1] 1761121557.690716 trunc from 0x4027f9 to 0x8b586d (0x4b3075 bytes) 000439[1] 1761121557.701278 mapread 0x6ce23c thru 0x6dbc74 (0xda39 bytes) 000441[1] 1761121557.720990 mapwrite 0x43b824 thru 0x4478ae (0xc08b bytes) 000442[1] 1761121557.754675 mapwrite 0x1ed01c thru 0x1f4451 (0x7436 bytes) 000445[1] 1761121557.780106 read 0x7f5327 thru 0x7fe7ec (0x94c6 bytes) 000446[1] 1761121557.789269 write 0x9a62ba thru 0x9ac63b (0x6382 bytes) 000449[0] 1761121557.794805 trunc from 0x9ac63c to 0x9662e7 (0x46354 bytes) 000450[0] 1761121557.809950 trunc from 0x9662e8 to 0x053337 (0x912fb0 bytes) 000452[1] 1761121557.814707 read 0x04e3bb thru 0x04fdab (0x19f1 bytes) 000453[1] 1761121557.823881 mapread 0x0027b3 thru 0x00be79 (0x96c7 bytes) 000454[0] 1761121557.841705 mapread 0x04e166 thru 0x053337 (0x51d2 bytes) 000455[1] 1761121557.856062 trunc from 0x053338 to 0x2477b3 (0x1f447c bytes) 000457[1] 1761121557.872911 mapwrite 0x31ed3b thru 0x320eab (0x2171 bytes) 000458[1] 1761121557.923771 mapwrite 0x92c321 thru 0x935533 (0x9213 bytes) 000459[0] 1761121557.963594 mapwrite 0x7bb3eb thru 0x7c1c8c (0x68a2 bytes) 000460[0] 1761121557.999466 mapwrite 0x3628c4 thru 0x3648e5 (0x2022 bytes) 000461[1] 1761121558.024848 mapwrite 0x163818 thru 0x1688e4 (0x50cd bytes) 000463[0] 1761121558.056371 write 0x3f9629 thru 0x402026 (0x89fe bytes) 000464[1] 1761121558.067543 write 0x605929 thru 0x610770 (0xae48 bytes) 000465[0] 1761121558.078474 mapread 0x442daa thru 0x44f330 (0xc587 bytes) 000467[0] 1761121558.105298 trunc from 0x935534 to 0x7226c7 (0x212e6c bytes) 000469[1] 1761121558.121305 read 0x2fa17d thru 0x3033dd (0x9261 bytes) 000472[1] 1761121558.161569 trunc from 0x7226c8 to 0x95d4b3 (0x23adec bytes) 000473[1] 1761121558.177939 trunc from 0x95d4b4 to 0x215aca (0x7479e9 bytes) 000475[1] 1761121558.193132 write 0x46fc77 thru 0x47c765 (0xcaef bytes) 000476[0] 1761121558.199817 trunc from 0x47c766 to 0x2df131 (0x19d634 bytes) 000479[0] 1761121558.216683 mapwrite 0x80d3e4 thru 0x8106de (0x32fb bytes) 000482[0] 1761121558.250124 mapread 0x050ab8 thru 0x053c6f (0x31b8 bytes) 000484[1] 1761121558.265068 mapwrite 0x80796a thru 0x814b2a (0xd1c1 bytes) 000485[1] 1761121558.301538 read 0x0afb53 thru 0x0b8dd7 (0x9285 bytes) 000486[1] 1761121558.337921 trunc from 0x814b2b to 0x662d6e (0x1b1dbc bytes) 000491[1] 1761121558.363872 mapwrite 0x7a218b thru 0x7a7332 (0x51a8 bytes) 000496[0] 1761121558.411473 write 0x68031f thru 0x686893 (0x6575 bytes) 000497[1] 1761121558.422979 mapwrite 0x9824b5 thru 0x990a4b (0xe597 bytes) 000498[1] 1761121558.466722 read 0x7b51b3 thru 0x7b7307 (0x2155 bytes) 000499[0] 1761121558.473109 write 0x7c7e2f thru 0x7d5da7 (0xdf79 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 1558 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[1] 1761121558.742156 mapwrite 0x62b000 thru 0x639163 (0xe164 bytes) 000004[0] 1761121558.793723 mapread 0x406000 thru 0x4075f4 (0x15f5 bytes) 000005[1] 1761121558.801974 mapwrite 0x67a000 thru 0x685061 (0xb062 bytes) 000006[1] 1761121558.830879 mapwrite 0x921000 thru 0x921dc8 (0xdc9 bytes) truncating to largest ever: 0x2664e8 000007[1] 1761121558.848560 trunc from 0x921dc9 to 0x2664e7 (0x6bb8e1 bytes) 000010[1] 1761121558.857758 mapread 0x1c6000 thru 0x1d25e9 (0xc5ea bytes) 000011[1] 1761121558.869058 mapwrite 0x68e000 thru 0x6970d2 (0x90d3 bytes) truncating to largest ever: 0x8f04bf 000012[1] 1761121558.896170 trunc from 0x6970d3 to 0x8f04be (0x2593ec bytes) 000014[0] 1761121558.907306 write 0x925000 thru 0x926a9e (0x1a9f bytes) 000017[0] 1761121558.910451 write 0x18a000 thru 0x18b6e8 (0x16e9 bytes) 000018[1] 1761121558.912019 read 0x219000 thru 0x222f98 (0x9f99 bytes) 000020[0] 1761121558.919891 write 0x213000 thru 0x21a850 (0x7851 bytes) 000022[1] 1761121558.930596 mapread 0x673000 thru 0x67ed0b (0xbd0c bytes) 000023[1] 1761121558.944435 mapread 0x592000 thru 0x59681a (0x481b bytes) 000027[0] 1761121558.947923 mapread 0x6a4000 thru 0x6a8874 (0x4875 bytes) 000030[1] 1761121558.956267 write 0x856000 thru 0x85b2e5 (0x52e6 bytes) 000033[1] 1761121558.968939 write 0x533000 thru 0x53397a (0x97b bytes) 000034[1] 1761121558.971084 mapread 0x35b000 thru 0x35e26c (0x326d bytes) 000035[0] 1761121558.976817 mapwrite 0x989000 thru 0x989b29 (0xb2a bytes) 000036[0] 1761121559.001431 read 0x11e000 thru 0x125bde (0x7bdf bytes) 000037[1] 1761121559.004036 mapread 0x70c000 thru 0x71a456 (0xe457 bytes) 000038[0] 1761121559.016073 mapread 0x1c1000 thru 0x1c63a9 (0x53aa bytes) 000039[1] 1761121559.021890 write 0x026000 thru 0x02a0b6 (0x40b7 bytes) 000040[1] 1761121559.029546 trunc from 0x989b2a to 0x25485e (0x7352cb bytes) 000045[1] 1761121559.046446 write 0x38f000 thru 0x399acb (0xaacc bytes) 000046[1] 1761121559.051344 write 0x258000 thru 0x25c355 (0x4356 bytes) 000048[1] 1761121559.057408 trunc from 0x399acc to 0x62b51c (0x291a51 bytes) 000049[1] 1761121559.066781 trunc from 0x62b51d to 0x1e6eaa (0x444672 bytes) 000050[1] 1761121559.071983 read 0x082000 thru 0x0852be (0x32bf bytes) 000054[1] 1761121559.089977 trunc from 0x1e6eab to 0x45b4a2 (0x2745f8 bytes) 000056[1] 1761121559.096065 write 0x8e3000 thru 0x8ea1f0 (0x71f1 bytes) 000057[1] 1761121559.103314 mapwrite 0x83e000 thru 0x84594d (0x794e bytes) 000058[0] 1761121559.165407 write 0x5e5000 thru 0x5e64bd (0x14be bytes) 000059[1] 1761121559.177774 mapwrite 0x8a5000 thru 0x8affdc (0xafdd bytes) 000061[1] 1761121559.218787 trunc from 0x8ea1f1 to 0x518bfe (0x3d15f2 bytes) 000063[0] 1761121559.229139 mapwrite 0x9a0000 thru 0x9a02b1 (0x2b2 bytes) 000064[0] 1761121559.251402 mapread 0x853000 thru 0x8561e2 (0x31e3 bytes) 000065[0] 1761121559.261820 write 0x1a0000 thru 0x1acc84 (0xcc85 bytes) 000069[1] 1761121559.278310 write 0x47d000 thru 0x47ec49 (0x1c4a bytes) 000071[0] 1761121559.288521 mapread 0x1ab000 thru 0x1ac5e2 (0x15e3 bytes) 000072[1] 1761121559.299874 read 0x986000 thru 0x993be0 (0xdbe1 bytes) 000073[1] 1761121559.303419 trunc from 0x9a02b2 to 0x40ba3a (0x594877 bytes) 000077[0] 1761121559.319067 read 0x0e5000 thru 0x0f398d (0xe98e bytes) 000078[0] 1761121559.347105 read 0x1ed000 thru 0x1f5591 (0x8592 bytes) 000079[1] 1761121559.347484 read 0x376000 thru 0x37bda6 (0x5da7 bytes) 000080[0] 1761121559.353521 trunc from 0x40ba3b to 0x49347a (0x87a40 bytes) 000082[0] 1761121559.371491 read 0x039000 thru 0x03e168 (0x5169 bytes) 000083[0] 1761121559.403950 write 0x506000 thru 0x5071e3 (0x11e4 bytes) 000084[0] 1761121559.407811 trunc from 0x5071e4 to 0x20724a (0x2fff99 bytes) 000085[1] 1761121559.414678 write 0x595000 thru 0x5a4b08 (0xfb09 bytes) 000086[1] 1761121559.421699 mapread 0x126000 thru 0x132eb3 (0xceb4 bytes) 000090[0] 1761121559.460003 mapread 0x032000 thru 0x03b66b (0x966c bytes) 000092[1] 1761121559.492086 trunc from 0x5a4b09 to 0x3ab760 (0x1f93a8 bytes) 000094[1] 1761121559.505390 read 0x03b000 thru 0x04815d (0xd15e bytes) 000096[1] 1761121559.533607 trunc from 0x3ab761 to 0x658e53 (0x2ad6f3 bytes) 000098[0] 1761121559.540103 mapwrite 0x824000 thru 0x82c1b1 (0x81b2 bytes) 000102[0] 1761121559.603970 read 0x0d3000 thru 0x0dd960 (0xa961 bytes) 000103[0] 1761121559.642310 read 0x725000 thru 0x72df62 (0x8f63 bytes) 000105[0] 1761121559.642846 write 0x52c000 thru 0x52e13c (0x213d bytes) 000107[1] 1761121559.647565 write 0x618000 thru 0x6212fe (0x92ff bytes) 000108[1] 1761121559.682933 read 0x77f000 thru 0x7810d4 (0x20d5 bytes) 000109[1] 1761121559.694475 read 0x220000 thru 0x223085 (0x3086 bytes) 000113[1] 1761121559.727669 trunc from 0x82c1b2 to 0x8dbff2 (0xafe41 bytes) 000114[1] 1761121559.732764 write 0x5f7000 thru 0x5fe94a (0x794b bytes) 000115[0] 1761121559.738548 mapread 0x4ce000 thru 0x4dc101 (0xe102 bytes) 000118[0] 1761121559.772965 trunc from 0x8dbff3 to 0x4acca0 (0x42f352 bytes) 000119[1] 1761121559.780571 mapread 0x26b000 thru 0x2781fb (0xd1fc bytes) 000121[1] 1761121559.800901 read 0x0a0000 thru 0x0a76fd (0x76fe bytes) 000123[1] 1761121559.831464 mapwrite 0x9f8000 thru 0x9fb544 (0x3545 bytes) 000126[0] 1761121559.900106 trunc from 0x9fb545 to 0x6a6c0b (0x354939 bytes) 000127[0] 1761121559.915425 mapread 0x68b000 thru 0x6989cf (0xd9d0 bytes) 000129[0] 1761121559.941912 read_OD 0x56a000 thru 0x574fff (0xb000 bytes) 000131[0] 1761121559.946968 mapwrite 0x91d000 thru 0x92a42a (0xd42b bytes) 000132[0] 1761121559.981046 mapwrite 0x146000 thru 0x14780c (0x180d bytes) 000134[0] 1761121560.005209 trunc from 0x92a42b to 0x2378de (0x6f2b4c bytes) 000135[0] 1761121560.014605 read_OD 0x09a000 thru 0x09bfff (0x2000 bytes) 000136[1] 1761121560.018764 mapwrite 0x7d6000 thru 0x7de916 (0x8917 bytes) 000138[0] 1761121560.062601 trunc from 0x7de917 to 0x83c91a (0x5e004 bytes) 000139[0] 1761121560.070690 mapwrite 0x082000 thru 0x08438a (0x238b bytes) 000142[0] 1761121560.134806 mapwrite 0x5b3000 thru 0x5b66fe (0x36ff bytes) 000146[0] 1761121560.166012 trunc from 0x83c91b to 0x2529a2 (0x5e9f78 bytes) 000147[1] 1761121560.184615 trunc from 0x2529a3 to 0x5c7491 (0x374aef bytes) 000148[0] 1761121560.194892 mapread 0x189000 thru 0x1939ef (0xa9f0 bytes) 000152[0] 1761121560.216710 write_OD 0x9be000 thru 0x9cafff (0xd000 bytes) 000155[1] 1761121560.233458 trunc from 0x9cb000 to 0x4f96d3 (0x4d192c bytes) 000156[1] 1761121560.244855 trunc from 0x4f96d4 to 0x7ceb20 (0x2d544d bytes) 000157[0] 1761121560.253454 read_OD 0x36a000 thru 0x377fff (0xe000 bytes) 000160[0] 1761121560.261773 mapread 0x4bc000 thru 0x4c46bc (0x86bd bytes) 000161[1] 1761121560.294869 mapread 0x6d1000 thru 0x6dd421 (0xc422 bytes) 000166[1] 1761121560.312066 mapread 0x401000 thru 0x405b5d (0x4b5e bytes) 000167[0] 1761121560.318348 read_OD 0x2c8000 thru 0x2d3fff (0xc000 bytes) 000169[0] 1761121560.320896 read_OD 0x05c000 thru 0x063fff (0x8000 bytes) 000173[0] 1761121560.323895 read_OD 0x22f000 thru 0x239fff (0xb000 bytes) 000174[0] 1761121560.326872 trunc from 0x7ceb21 to 0x7fb1ed (0x2c6cd bytes) 000175[0] 1761121560.343065 mapread 0x4fc000 thru 0x5095b3 (0xd5b4 bytes) 000176[0] 1761121560.381197 read_OD 0x66c000 thru 0x676fff (0xb000 bytes) 000177[1] 1761121560.393428 mapread 0x17c000 thru 0x17f034 (0x3035 bytes) truncating to largest ever: 0x9bf54a 000178[1] 1761121560.399408 trunc from 0x7fb1ee to 0x9bf549 (0x1c435c bytes) 000179[0] 1761121560.407608 read_OD 0x52f000 thru 0x538fff (0xa000 bytes) 000181[0] 1761121560.416428 mapwrite 0x365000 thru 0x373c26 (0xec27 bytes) 000182[1] 1761121560.491438 write 0x8b1000 thru 0x8b78fc (0x68fd bytes) 000186[0] 1761121560.505118 write_OD 0x97c000 thru 0x980fff (0x5000 bytes) 000190[0] 1761121560.523215 read_OD 0x58c000 thru 0x592fff (0x7000 bytes) 000192[1] 1761121560.526072 trunc from 0x9bf54a to 0x1e1ec5 (0x7dd684 bytes) 000193[1] 1761121560.537512 read 0x0ae000 thru 0x0ae0d9 (0xda bytes) 000196[0] 1761121560.560617 read_OD 0x10b000 thru 0x110fff (0x6000 bytes) 000197[0] 1761121560.568886 write_OD 0x3c1000 thru 0x3c6fff (0x6000 bytes) 000199[0] 1761121560.586152 mapwrite 0x486000 thru 0x48d8e1 (0x78e2 bytes) 000200[1] 1761121560.638321 read 0x0f2000 thru 0x1005bf (0xe5c0 bytes) 000205[1] 1761121560.677699 read 0x109000 thru 0x10f4c5 (0x64c6 bytes) 000206[1] 1761121560.678072 write 0x4ea000 thru 0x4f49af (0xa9b0 bytes) 000208[1] 1761121560.687475 read 0x03e000 thru 0x042c47 (0x4c48 bytes) 000209[0] 1761121560.719476 trunc from 0x4f49b0 to 0x096485 (0x45e52a bytes) 000211[0] 1761121560.743636 read_OD 0x031000 thru 0x037fff (0x7000 bytes) 000213[0] 1761121560.751079 mapwrite 0x79c000 thru 0x79e57b (0x257c bytes) 000215[0] 1761121560.827811 mapwrite 0x910000 thru 0x9193c0 (0x93c1 bytes) 000220[1] 1761121560.871498 read 0x0ff000 thru 0x101ccc (0x2ccd bytes) 000222[1] 1761121560.929106 write 0x306000 thru 0x30c6c0 (0x66c1 bytes) 000223[0] 1761121560.945387 read_OD 0x569000 thru 0x575fff (0xd000 bytes) 000224[1] 1761121560.953524 read 0x29e000 thru 0x29f15e (0x115f bytes) 000228[1] 1761121560.997297 write 0x073000 thru 0x07b5b6 (0x85b7 bytes) 000229[1] 1761121561.014130 read 0x22a000 thru 0x234807 (0xa808 bytes) 000231[1] 1761121561.052854 read 0x122000 thru 0x122822 (0x823 bytes) 000233[0] 1761121561.067363 mapwrite 0x84e000 thru 0x857ecb (0x9ecc bytes) 000235[0] 1761121561.123045 write_OD 0x093000 thru 0x093fff (0x1000 bytes) 000238[0] 1761121561.141696 write_OD 0x847000 thru 0x84bfff (0x5000 bytes) 000240[1] 1761121561.155655 trunc from 0x9193c1 to 0x7f97cf (0x11fbf1 bytes) 000241[0] 1761121561.162985 write_OD 0x603000 thru 0x604fff (0x2000 bytes) 000242[1] 1761121561.175629 mapwrite 0x235000 thru 0x235eb1 (0xeb2 bytes) 000244[0] 1761121561.191303 trunc from 0x7f97d0 to 0x342e69 (0x4b6966 bytes) 000246[1] 1761121561.199323 mapwrite 0x921000 thru 0x930ff5 (0xfff6 bytes) 000248[0] 1761121561.232745 mapwrite 0x57b000 thru 0x57c630 (0x1631 bytes) 000249[1] 1761121561.257842 mapread 0x86d000 thru 0x87caf3 (0xfaf4 bytes) 000250[0] 1761121561.285433 mapread 0x0ee000 thru 0x0f4972 (0x6973 bytes) 000252[0] 1761121561.294665 write_OD 0x56a000 thru 0x56dfff (0x4000 bytes) 000253[1] 1761121561.308321 trunc from 0x930ff6 to 0x71465d (0x21c998 bytes) 000254[1] 1761121561.316298 write 0x928000 thru 0x92f087 (0x7088 bytes) 000255[0] 1761121561.322440 trunc from 0x92f088 to 0x1b90aa (0x775fdd bytes) 000256[1] 1761121561.335198 mapread 0x04c000 thru 0x04e227 (0x2228 bytes) 000257[0] 1761121561.343070 mapread 0x105000 thru 0x1081ba (0x31bb bytes) 000258[0] 1761121561.349418 mapread 0x179000 thru 0x18558d (0xc58e bytes) 000261[0] 1761121561.366520 write_OD 0x1cd000 thru 0x1d0fff (0x4000 bytes) 000262[0] 1761121561.381173 mapwrite 0x56a000 thru 0x57461f (0xa620 bytes) 000264[1] 1761121561.425717 mapread 0x54b000 thru 0x559300 (0xe301 bytes) 000267[1] 1761121561.450848 trunc from 0x574620 to 0x8088db (0x2942bc bytes) 000268[0] 1761121561.460781 mapread 0x1a9000 thru 0x1b26f7 (0x96f8 bytes) 000269[0] 1761121561.477865 trunc from 0x8088dc to 0x009422 (0x7ff4b9 bytes) 000271[0] 1761121561.494419 trunc from 0x009423 to 0x33c095 (0x332c73 bytes) 000274[0] 1761121561.504800 write_OD 0x482000 thru 0x48dfff (0xc000 bytes) 000280[1] 1761121561.521649 write 0x761000 thru 0x76369f (0x26a0 bytes) 000281[1] 1761121561.528740 mapwrite 0x642000 thru 0x647a4d (0x5a4e bytes) 000284[0] 1761121561.554559 write_OD 0x05f000 thru 0x05ffff (0x1000 bytes) 000286[1] 1761121561.569426 read 0x063000 thru 0x06e46d (0xb46e bytes) 000288[0] 1761121561.610262 mapread 0x6bb000 thru 0x6c2106 (0x7107 bytes) 000290[1] 1761121561.620648 mapread 0x750000 thru 0x757156 (0x7157 bytes) 000293[0] 1761121561.621507 mapread 0x4f8000 thru 0x4f881e (0x81f bytes) 000294[0] 1761121561.622877 mapread 0x44f000 thru 0x45d74b (0xe74c bytes) 000296[1] 1761121561.641209 mapwrite 0x1e7000 thru 0x1ef4c9 (0x84ca bytes) 000299[0] 1761121561.700505 mapread 0x65f000 thru 0x662dc3 (0x3dc4 bytes) 000300[1] 1761121561.710534 read 0x025000 thru 0x028037 (0x3038 bytes) 000301[0] 1761121561.743492 trunc from 0x7636a0 to 0x51d36d (0x246332 bytes) 000302[0] 1761121561.771382 read_OD 0x35e000 thru 0x361fff (0x4000 bytes) 000303[0] 1761121561.776521 read_OD 0x0b7000 thru 0x0b9fff (0x3000 bytes) 000304[1] 1761121561.779106 write 0x773000 thru 0x77d2b4 (0xa2b5 bytes) 000305[0] 1761121561.782982 trunc from 0x77d2b5 to 0x616468 (0x166e4c bytes) 000306[0] 1761121561.792097 mapread 0x3a4000 thru 0x3b1f2a (0xdf2b bytes) 000310[0] 1761121561.842433 mapwrite 0x9a0000 thru 0x9aae9c (0xae9d bytes) 000311[0] 1761121561.876118 mapwrite 0x8b5000 thru 0x8c4b3b (0xfb3c bytes) 000312[1] 1761121561.916186 mapread 0x047000 thru 0x0536eb (0xc6ec bytes) 000313[1] 1761121561.961950 mapread 0x16b000 thru 0x16b95c (0x95d bytes) 000316[0] 1761121561.963875 trunc from 0x9aae9d to 0x078545 (0x932957 bytes) 000318[0] 1761121561.974969 read_OD 0x047000 thru 0x050fff (0xa000 bytes) 000319[0] 1761121561.979497 mapread 0x018000 thru 0x01cfb1 (0x4fb2 bytes) 000320[0] 1761121561.987534 write_OD 0x094000 thru 0x0a1fff (0xe000 bytes) 000321[0] 1761121562.004218 read_OD 0x09f000 thru 0x0a0fff (0x2000 bytes) 000325[1] 1761121562.007438 mapread 0x036000 thru 0x04445a (0xe45b bytes) 000326[0] 1761121562.025940 mapwrite 0x014000 thru 0x01b990 (0x7991 bytes) 000327[1] 1761121562.053945 trunc from 0x0a2000 to 0x959521 (0x8b7522 bytes) 000329[1] 1761121562.069542 read 0x66d000 thru 0x66d3c8 (0x3c9 bytes) 000333[1] 1761121562.101894 mapwrite 0x48c000 thru 0x492555 (0x6556 bytes) 000335[0] 1761121562.131901 mapwrite 0x405000 thru 0x407065 (0x2066 bytes) 000336[1] 1761121562.220590 read 0x165000 thru 0x16acba (0x5cbb bytes) 000338[1] 1761121562.273203 mapwrite 0x4be000 thru 0x4c4bd4 (0x6bd5 bytes) 000339[0] 1761121562.323533 mapread 0x5f8000 thru 0x5fcfc3 (0x4fc4 bytes) 000343[0] 1761121562.336560 trunc from 0x959522 to 0x7d30c0 (0x186461 bytes) 000344[0] 1761121562.344604 mapwrite 0x9f4000 thru 0x9f9147 (0x5148 bytes) 000347[0] 1761121562.379908 mapread 0x63e000 thru 0x648337 (0xa338 bytes) 000348[0] 1761121562.397026 trunc from 0x9f9148 to 0x1eda74 (0x80b6d3 bytes) 000349[0] 1761121562.402358 read_OD 0x09b000 thru 0x0a5fff (0xb000 bytes) 000356[1] 1761121562.409301 mapread 0x0e1000 thru 0x0ed20c (0xc20d bytes) 000358[1] 1761121562.425764 write 0x067000 thru 0x072008 (0xb009 bytes) 000360[0] 1761121562.430912 read_OD 0x15b000 thru 0x15ffff (0x5000 bytes) 000361[1] 1761121562.440470 read 0x198000 thru 0x1a0737 (0x8738 bytes) 000362[0] 1761121562.446326 read_OD 0x13d000 thru 0x141fff (0x5000 bytes) 000363[1] 1761121562.449053 mapwrite 0x64e000 thru 0x64f2cb (0x12cc bytes) 000365[0] 1761121562.503267 trunc from 0x64f2cc to 0x43da3a (0x211891 bytes) 000366[1] 1761121562.515632 write 0x571000 thru 0x577b02 (0x6b03 bytes) 000367[1] 1761121562.523488 trunc from 0x577b03 to 0x605949 (0x8de47 bytes) 000369[0] 1761121562.536611 mapwrite 0x858000 thru 0x85b0a4 (0x30a5 bytes) 000375[0] 1761121562.615757 trunc from 0x85b0a5 to 0x06e0ca (0x7ecfda bytes) 000376[1] 1761121562.628694 trunc from 0x06e0cb to 0x41c14e (0x3ae084 bytes) 000377[1] 1761121562.639771 write 0x83c000 thru 0x840758 (0x4759 bytes) 000379[0] 1761121562.642819 read_OD 0x06a000 thru 0x06cfff (0x3000 bytes) 000380[1] 1761121562.651591 mapwrite 0x435000 thru 0x43d23b (0x823c bytes) 000381[0] 1761121562.681488 write_OD 0x7db000 thru 0x7e1fff (0x7000 bytes) 000382[1] 1761121562.697006 read 0x5e4000 thru 0x5f38c7 (0xf8c8 bytes) 000384[0] 1761121562.718330 trunc from 0x840759 to 0x01179c (0x82efbc bytes) 000385[0] 1761121562.728546 read_OD 0x002000 thru 0x00ffff (0xe000 bytes) 000386[0] 1761121562.733409 read_OD 0x00e000 thru 0x00ffff (0x2000 bytes) 000387[1] 1761121562.735063 write 0x609000 thru 0x617c59 (0xec5a bytes) 000388[0] 1761121562.743425 mapwrite 0x32c000 thru 0x335669 (0x966a bytes) 000389[1] 1761121562.810490 read 0x603000 thru 0x608f42 (0x5f43 bytes) 000393[0] 1761121562.825590 mapwrite 0x438000 thru 0x4444bd (0xc4be bytes) 000396[0] 1761121562.868703 mapwrite 0x998000 thru 0x9a1483 (0x9484 bytes) 000397[0] 1761121562.909643 mapread 0x6e5000 thru 0x6f2cb8 (0xdcb9 bytes) 000402[1] 1761121562.938943 mapread 0x921000 thru 0x929623 (0x8624 bytes) 000404[1] 1761121562.973865 write 0x8cc000 thru 0x8d4ab8 (0x8ab9 bytes) 000405[0] 1761121562.982096 trunc from 0x9a1484 to 0x2f2e85 (0x6ae5fe bytes) 000407[1] 1761121562.992095 mapread 0x1f3000 thru 0x1f712a (0x412b bytes) 000408[0] 1761121563.004929 mapread 0x0a6000 thru 0x0b06dd (0xa6de bytes) 000410[0] 1761121563.020026 read_OD 0x147000 thru 0x148fff (0x2000 bytes) 000414[1] 1761121563.022027 trunc from 0x2f2e86 to 0x11fdf5 (0x1d3090 bytes) 000415[0] 1761121563.030421 trunc from 0x11fdf6 to 0x7ac404 (0x68c60f bytes) 000416[0] 1761121563.044582 read_OD 0x5a7000 thru 0x5aafff (0x4000 bytes) 000418[0] 1761121563.050296 read_OD 0x583000 thru 0x590fff (0xe000 bytes) truncating to largest ever: 0x9cc2fe 000419[0] 1761121563.053775 trunc from 0x7ac405 to 0x9cc2fd (0x21fef9 bytes) 000420[1] 1761121563.063458 write 0x6ab000 thru 0x6b7058 (0xc059 bytes) 000422[1] 1761121563.069247 mapread 0x3e0000 thru 0x3ee971 (0xe972 bytes) 000424[0] 1761121563.087718 mapread 0x122000 thru 0x1271b2 (0x51b3 bytes) 000425[0] 1761121563.130596 trunc from 0x9cc2fe to 0x65e8b9 (0x36da44 bytes) 000427[1] 1761121563.143548 mapwrite 0x06e000 thru 0x075565 (0x7566 bytes) 000428[0] 1761121563.173915 trunc from 0x65e8ba to 0x4ed47e (0x17143b bytes) 000429[1] 1761121563.182596 read 0x43f000 thru 0x442ff9 (0x3ffa bytes) 000430[1] 1761121563.193886 trunc from 0x4ed47f to 0x8beb85 (0x3d1707 bytes) 000435[0] 1761121563.222879 read_OD 0x32a000 thru 0x32bfff (0x2000 bytes) 000436[0] 1761121563.228855 mapwrite 0x5aa000 thru 0x5b20d9 (0x80da bytes) 000438[0] 1761121563.291195 write_OD 0x9f1000 thru 0x9f9fff (0x9000 bytes) 000440[0] 1761121563.313038 read_OD 0x5a7000 thru 0x5b3fff (0xd000 bytes) 000443[1] 1761121563.317075 read 0x514000 thru 0x51ff3e (0xbf3f bytes) 000444[1] 1761121563.350979 write 0x010000 thru 0x01b109 (0xb10a bytes) 000445[1] 1761121563.364411 read 0x2f1000 thru 0x2fc62f (0xb630 bytes) 000447[0] 1761121563.406657 trunc from 0x9fa000 to 0x6dbfd1 (0x31e02e bytes) 000448[1] 1761121563.433557 mapwrite 0x84d000 thru 0x8508cc (0x38cd bytes) 000449[0] 1761121563.463177 write_OD 0x173000 thru 0x178fff (0x6000 bytes) 000452[1] 1761121563.483419 mapread 0x332000 thru 0x33b9b5 (0x99b6 bytes) 000454[1] 1761121563.496826 mapread 0x503000 thru 0x5063b5 (0x33b6 bytes) 000455[1] 1761121563.502294 mapread 0x208000 thru 0x213eb9 (0xbeba bytes) 000458[1] 1761121563.517859 write 0x1af000 thru 0x1b92bb (0xa2bc bytes) 000461[0] 1761121563.529640 read_OD 0x43e000 thru 0x448fff (0xb000 bytes) 000463[0] 1761121563.538251 mapwrite 0x7a1000 thru 0x7afc07 (0xec08 bytes) 000465[0] 1761121563.590095 trunc from 0x8508cd to 0x84455d (0xc36f bytes) 000466[0] 1761121563.599296 mapread 0x410000 thru 0x41a1c4 (0xa1c5 bytes) 000469[1] 1761121563.613974 read 0x229000 thru 0x232af1 (0x9af2 bytes) 000472[0] 1761121563.660346 mapwrite 0x992000 thru 0x99e8d3 (0xc8d4 bytes) 000474[1] 1761121563.713710 read 0x66a000 thru 0x66d9e5 (0x39e6 bytes) 000475[0] 1761121563.742790 write_OD 0x254000 thru 0x25efff (0xb000 bytes) 000476[0] 1761121563.759699 read_OD 0x343000 thru 0x34afff (0x8000 bytes) 000477[1] 1761121563.761973 read 0x99b000 thru 0x99df58 (0x2f59 bytes) 000479[0] 1761121563.764643 mapread 0x835000 thru 0x83b07d (0x607e bytes) 000481[0] 1761121563.814934 mapread 0x885000 thru 0x88ec74 (0x9c75 bytes) 000490[1] 1761121563.817090 read 0x48b000 thru 0x48f5b4 (0x45b5 bytes) 000491[0] 1761121563.845060 trunc from 0x99e8d4 to 0x644316 (0x35a5bd bytes) 000492[0] 1761121563.870848 trunc from 0x644317 to 0x61cbda (0x2773c bytes) 000494[1] 1761121563.877096 trunc from 0x61cbdb to 0x7ddbb9 (0x1c0fdf bytes) 000495[0] 1761121563.888813 read_OD 0x0ef000 thru 0x0fafff (0xc000 bytes) 000498[1] 1761121563.897162 trunc from 0x7ddbba to 0x435d0b (0x3a7eae bytes) 000499[0] 1761121563.904612 trunc from 0x435d0c to 0x6e3e76 (0x2ae16b bytes) 000500[0] 1761121563.913562 trunc from 0x6e3e77 to 0x6cb16f (0x18d07 bytes) All operations completed A-OK! PASS 16a (14s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 04:26:05 (1761121565) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0903473 s, 23.2 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0716215 s, 29.3 MB/s Chance of close/open is 1 in 50 Seed set to 1566 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x70e99a 000001[1] 1761121566.703906 trunc from 00000000 to 0x70e999 (0x70e99a bytes) 000002[0] 1761121566.714680 mapread 0x036c21 thru 0x037446 (0x826 bytes) 000003[0] 1761121566.726403 trunc from 0x70e99a to 0x4f0d4c (0x21dc4d bytes) 000006[0] 1761121566.742893 write 0x468555 thru 0x469ec2 (0x196e bytes) 000008[1] 1761121566.754336 mapread 0x0c6503 thru 0x0c736c (0xe6a bytes) 000010[0] 1761121566.768535 mapread 0x00b1ec thru 0x00bfa4 (0xdb9 bytes) 000014[0] 1761121566.781724 read 0x43a396 thru 0x43a531 (0x19c bytes) 000018[1] 1761121566.796490 mapwrite 0x11ca29 thru 0x11e2e5 (0x18bd bytes) 000021[0] 1761121566.837824 write 0x4590d5 thru 0x459a58 (0x984 bytes) 000022[1] 1761121566.846741 write 0x79a74f thru 0x79aa0e (0x2c0 bytes) 000023[1] 1761121566.853041 mapread 0x0f6645 thru 0x0f75d4 (0xf90 bytes) 000025[0] 1761121566.866459 read 0x069c88 thru 0x06b4ab (0x1824 bytes) 000028[1] 1761121566.883364 mapread 0x2a71c6 thru 0x2a80d6 (0xf11 bytes) 000033[0] 1761121566.902799 mapread 0x0f7346 thru 0x0f9203 (0x1ebe bytes) 000034[1] 1761121566.909270 write 0x1fd55e thru 0x1fe886 (0x1329 bytes) 000035[0] 1761121566.918860 mapwrite 0x95d2f0 thru 0x95d9b9 (0x6ca bytes) 000037[0] 1761121566.957009 write 0x116508 thru 0x1174b3 (0xfac bytes) 000038[1] 1761121566.964062 read 0x2bfa15 thru 0x2c06bf (0xcab bytes) 000040[1] 1761121566.973179 mapread 0x6d26ca thru 0x6d45fb (0x1f32 bytes) 000043[0] 1761121566.988819 write 0x95e410 thru 0x95f0ee (0xcdf bytes) 000045[1] 1761121566.997016 write 0x592ebf thru 0x593ecc (0x100e bytes) 000046[1] 1761121567.003943 trunc from 0x95f0ef to 0x0cc94f (0x89279f bytes) 000047[1] 1761121567.012367 trunc from 0x0cc950 to 0x5921a8 (0x4c5859 bytes) 000053[1] 1761121567.034824 mapwrite 0x8d8489 thru 0x8d9714 (0x128c bytes) 000055[0] 1761121567.065577 mapread 0x57802f thru 0x579d02 (0x1cd4 bytes) 000057[1] 1761121567.077585 mapwrite 0x921de0 thru 0x922481 (0x6a2 bytes) 000060[0] 1761121567.112836 read 0x721fe1 thru 0x72311d (0x113d bytes) truncating to largest ever: 0x8fcde4 000061[0] 1761121567.127200 trunc from 0x922482 to 0x8fcde3 (0x2569e bytes) 000062[0] 1761121567.137827 trunc from 0x8fcde4 to 0x21e52b (0x6de8b8 bytes) 000063[1] 1761121567.141498 mapread 0x13d10a thru 0x13da8d (0x984 bytes) 000064[1] 1761121567.151186 mapread 0x0cc624 thru 0x0cd8b2 (0x128f bytes) 000065[0] 1761121567.159173 mapwrite 0x597606 thru 0x597967 (0x362 bytes) 000070[0] 1761121567.202743 mapread 0x1a7d02 thru 0x1a80ef (0x3ee bytes) 000071[0] 1761121567.210694 mapwrite 0x923645 thru 0x924da0 (0x175c bytes) 000073[1] 1761121567.245626 trunc from 0x924da1 to 0x68585a (0x29f546 bytes) 000076[1] 1761121567.264815 trunc from 0x68585b to 0x12a22f (0x55b62b bytes) 000079[1] 1761121567.282793 trunc from 0x12a230 to 0x0a7db4 (0x8247b bytes) 000083[0] 1761121567.293583 write 0x49b49e thru 0x49d35f (0x1ec2 bytes) 000086[1] 1761121567.308360 read 0x12b780 thru 0x12b7f5 (0x76 bytes) 000088[0] 1761121567.318542 trunc from 0x49d360 to 0x510975 (0x73616 bytes) 000089[1] 1761121567.326101 write 0x2233d9 thru 0x2251a7 (0x1dcf bytes) 000090[0] 1761121567.338512 mapwrite 0x2dc970 thru 0x2de943 (0x1fd4 bytes) 000091[1] 1761121567.366836 trunc from 0x510976 to 0x689f39 (0x1795c4 bytes) 000094[0] 1761121567.383408 mapread 0x39ae25 thru 0x39ca1f (0x1bfb bytes) 000097[1] 1761121567.397533 mapread 0x4648fd thru 0x465508 (0xc0c bytes) 000098[1] 1761121567.400543 mapwrite 0x6b4c17 thru 0x6b6853 (0x1c3d bytes) 000099[0] 1761121567.424267 read 0x308abe thru 0x30a451 (0x1994 bytes) 000100[0] 1761121567.439155 mapwrite 0x1ec73e thru 0x1ed62f (0xef2 bytes) 000101[1] 1761121567.464689 trunc from 0x6b6854 to 0x5c25e8 (0xf426b bytes) 000106[0] 1761121567.482106 read 0x3fe96d thru 0x3ff5d1 (0xc65 bytes) 000107[1] 1761121567.487976 write 0x2f3791 thru 0x2f4062 (0x8d2 bytes) 000109[0] 1761121567.494554 read 0x2affba thru 0x2b1bed (0x1c34 bytes) 000112[1] 1761121567.524283 mapread 0x1d8b33 thru 0x1d9741 (0xc0f bytes) 000113[0] 1761121567.527704 write 0x4a7d2e thru 0x4a84d8 (0x7ab bytes) 000115[1] 1761121567.538367 mapread 0x294c5a thru 0x2953b3 (0x75a bytes) 000117[1] 1761121567.547975 write 0x712c19 thru 0x713908 (0xcf0 bytes) 000120[1] 1761121567.561773 mapread 0x43e560 thru 0x43fe35 (0x18d6 bytes) 000122[0] 1761121567.568861 write 0x01b6ac thru 0x01cb7d (0x14d2 bytes) 000125[1] 1761121567.585533 trunc from 0x713909 to 0x2aa302 (0x469606 bytes) 000129[1] 1761121567.608544 trunc from 0x2aa303 to 0x3b7111 (0x10ce0f bytes) 000130[0] 1761121567.614727 mapread 0x04a065 thru 0x04ae30 (0xdcc bytes) 000133[0] 1761121567.622720 mapwrite 0x80d82d thru 0x80f679 (0x1e4d bytes) 000134[1] 1761121567.681249 trunc from 0x80f67a to 0x2f595d (0x519d1c bytes) 000135[1] 1761121567.692892 write 0x041c0e thru 0x042f3e (0x1331 bytes) 000136[0] 1761121567.701135 write 0x3fd7e1 thru 0x3fef4a (0x176a bytes) 000140[0] 1761121567.719196 write 0x99ed14 thru 0x99fd71 (0x105e bytes) 000141[0] 1761121567.724915 mapwrite 0x2624d2 thru 0x2635d9 (0x1108 bytes) truncating to largest ever: 0x907272 000142[0] 1761121567.748850 trunc from 0x99fd72 to 0x907271 (0x98b00 bytes) 000144[0] 1761121567.759426 trunc from 0x907272 to 0x254c78 (0x6b25f9 bytes) 000147[0] 1761121567.770615 write 0x481ac0 thru 0x483469 (0x19aa bytes) 000148[1] 1761121567.771781 write 0x672488 thru 0x673b0e (0x1687 bytes) 000149[0] 1761121567.780349 trunc from 0x673b0f to 0x01cb62 (0x656fac bytes) 000152[0] 1761121567.796025 read 0x00a673 thru 0x00b42c (0xdba bytes) 000154[0] 1761121567.803963 trunc from 0x01cb63 to 0x3020d5 (0x2e5573 bytes) 000155[0] 1761121567.813897 write 0x370caf thru 0x3720b9 (0x140b bytes) 000157[0] 1761121567.820915 write 0x031ef9 thru 0x033bdd (0x1ce5 bytes) 000158[1] 1761121567.848838 read 0x22750a thru 0x22761c (0x113 bytes) 000161[0] 1761121567.869816 mapwrite 0x195ce1 thru 0x1973c0 (0x16e0 bytes) 000162[1] 1761121567.888361 mapwrite 0x97ea62 thru 0x97ed6e (0x30d bytes) 000166[1] 1761121567.961597 read 0x8973c7 thru 0x897aa6 (0x6e0 bytes) 000167[0] 1761121567.965614 mapread 0x249587 thru 0x24af8e (0x1a08 bytes) 000168[1] 1761121567.979395 read 0x07c63f thru 0x07d805 (0x11c7 bytes) 000169[0] 1761121568.012118 mapread 0x240d1f thru 0x242c30 (0x1f12 bytes) 000170[1] 1761121568.019123 mapread 0x64248b thru 0x64385a (0x13d0 bytes) 000171[0] 1761121568.023686 write 0x97b2c5 thru 0x97bd25 (0xa61 bytes) 000172[0] 1761121568.032663 read 0x1331c2 thru 0x1337e4 (0x623 bytes) 000173[0] 1761121568.072519 mapwrite 0x3921d6 thru 0x3931ab (0xfd6 bytes) 000176[0] 1761121568.098041 read 0x1f4896 thru 0x1f5604 (0xd6f bytes) 000177[1] 1761121568.098497 mapread 0x194cb7 thru 0x196aa0 (0x1dea bytes) 000179[0] 1761121568.114854 read 0x05721a thru 0x057cd5 (0xabc bytes) 000180[0] 1761121568.145001 write 0x8d6c85 thru 0x8d8a9c (0x1e18 bytes) 000181[1] 1761121568.165009 write 0x9758d8 thru 0x975f20 (0x649 bytes) 000182[1] 1761121568.178100 write 0x4049bd thru 0x4067af (0x1df3 bytes) 000190[0] 1761121568.203077 read 0x21aae2 thru 0x21c1eb (0x170a bytes) 000191[0] 1761121568.219045 mapread 0x765163 thru 0x7662dd (0x117b bytes) 000192[1] 1761121568.270767 read 0x3d48a5 thru 0x3d5f84 (0x16e0 bytes) 000193[0] 1761121568.303271 trunc from 0x97ed6f to 0x70845e (0x276910 bytes) 000194[1] 1761121568.331044 mapwrite 0x2bbceb thru 0x2bbd84 (0x9a bytes) 000197[1] 1761121568.363437 mapwrite 0x8263cc thru 0x827dc0 (0x19f5 bytes) 000198[0] 1761121568.386768 read 0x1e919b thru 0x1e9b41 (0x9a7 bytes) 000200[0] 1761121568.393584 trunc from 0x827dc1 to 0x2da45f (0x54d961 bytes) 000201[0] 1761121568.404344 write 0x247766 thru 0x2477f0 (0x8b bytes) 000202[0] 1761121568.413150 mapwrite 0x90107d thru 0x902fb1 (0x1f35 bytes) truncating to largest ever: 0x9a2ca1 000204[0] 1761121568.482254 trunc from 0x902fb2 to 0x9a2ca0 (0x9fcef bytes) 000207[1] 1761121568.493074 mapread 0x57f24f thru 0x5801d3 (0xf85 bytes) 000208[0] 1761121568.509804 mapwrite 0x31c11e thru 0x31da8a (0x196d bytes) 000210[1] 1761121568.527871 mapwrite 0x6fb8bf thru 0x6fcabe (0x1200 bytes) 000212[1] 1761121568.562102 write 0x1e40fe thru 0x1e416b (0x6e bytes) 000214[1] 1761121568.570793 trunc from 0x9a2ca1 to 0x6b6f70 (0x2ebd30 bytes) 000217[1] 1761121568.581895 trunc from 0x6b6f71 to 0x8e0b36 (0x229bc6 bytes) 000219[0] 1761121568.592133 trunc from 0x8e0b37 to 0x342173 (0x59e9c3 bytes) 000221[0] 1761121568.608306 mapread 0x179518 thru 0x179c57 (0x740 bytes) 000223[0] 1761121568.613991 read 0x0851dd thru 0x085465 (0x289 bytes) 000226[1] 1761121568.637906 read 0x2ecbb1 thru 0x2ed3d9 (0x829 bytes) 000229[0] 1761121568.653683 trunc from 0x342174 to 0x949040 (0x606ecd bytes) 000230[0] 1761121568.666979 read 0x49f7d8 thru 0x4a08dc (0x1105 bytes) 000235[0] 1761121568.704444 mapread 0x24309a thru 0x243e6a (0xdd1 bytes) 000236[1] 1761121568.706448 trunc from 0x949041 to 0x59bda7 (0x3ad299 bytes) 000237[1] 1761121568.726617 mapwrite 0x801a36 thru 0x80272c (0xcf7 bytes) 000240[0] 1761121568.786500 trunc from 0x80272d to 0x15387a (0x6aeeb2 bytes) 000241[0] 1761121568.800628 read 0x0c69ac thru 0x0c6d95 (0x3ea bytes) 000247[1] 1761121568.821035 trunc from 0x15387b to 0x1cd571 (0x79cf7 bytes) 000248[0] 1761121568.837279 mapwrite 0x9778ef thru 0x979401 (0x1b13 bytes) 000249[1] 1761121568.915505 mapread 0x0fae1f thru 0x0fc6f7 (0x18d9 bytes) 000250[0] 1761121568.932944 mapread 0x79b01c thru 0x79c1c7 (0x11ac bytes) 000252[0] 1761121568.942207 mapwrite 0x46f2e2 thru 0x470c26 (0x1945 bytes) 000253[0] 1761121568.962556 write 0x93ce87 thru 0x93e44c (0x15c6 bytes) 000254[1] 1761121568.964155 mapwrite 0x5b9a04 thru 0x5baa29 (0x1026 bytes) 000256[0] 1761121568.998450 mapread 0x92faa7 thru 0x930357 (0x8b1 bytes) 000260[1] 1761121569.010295 mapread 0x6f395b thru 0x6f39aa (0x50 bytes) 000261[1] 1761121569.017128 trunc from 0x979402 to 0x494c1d (0x4e47e4 bytes) 000266[1] 1761121569.033746 read 0x13f47a thru 0x140fc0 (0x1b47 bytes) 000270[0] 1761121569.072478 write 0x7f8618 thru 0x7fa36c (0x1d55 bytes) 000271[0] 1761121569.081473 trunc from 0x7fa36d to 0x023e96 (0x7d64d6 bytes) 000272[1] 1761121569.090580 read 0x002ec1 thru 0x0034b3 (0x5f3 bytes) 000273[1] 1761121569.100799 mapread 0x023b20 thru 0x023e96 (0x377 bytes) 000274[0] 1761121569.103939 trunc from 0x023e97 to 0x586c03 (0x562d6d bytes) 000277[0] 1761121569.113687 trunc from 0x586c04 to 0x3f992a (0x18d2d9 bytes) 000278[1] 1761121569.117757 trunc from 0x3f992b to 0x772a18 (0x3790ee bytes) 000279[0] 1761121569.128307 read 0x08fb29 thru 0x0917b6 (0x1c8e bytes) 000281[0] 1761121569.144779 mapread 0x46b020 thru 0x46b93a (0x91b bytes) 000282[1] 1761121569.151784 mapwrite 0x299e58 thru 0x29af87 (0x1130 bytes) 000283[0] 1761121569.173959 write 0x82820d thru 0x828ec7 (0xcbb bytes) 000284[1] 1761121569.180142 mapwrite 0x256f1d thru 0x258189 (0x126d bytes) 000288[0] 1761121569.217396 write 0x0b7355 thru 0x0b75a7 (0x253 bytes) 000289[0] 1761121569.224514 mapread 0x10ff6f thru 0x110b63 (0xbf5 bytes) 000290[1] 1761121569.272410 read 0x12dab8 thru 0x12e7ae (0xcf7 bytes) 000292[0] 1761121569.285275 mapwrite 0x387c7c thru 0x3898c6 (0x1c4b bytes) 000293[1] 1761121569.321834 read 0x4dd071 thru 0x4df021 (0x1fb1 bytes) 000294[0] 1761121569.335434 mapread 0x3d8e4e thru 0x3d8f8b (0x13e bytes) 000296[0] 1761121569.341587 mapread 0x34aece thru 0x34c743 (0x1876 bytes) 000298[1] 1761121569.354419 mapwrite 0x6a6560 thru 0x6a6f94 (0xa35 bytes) 000300[1] 1761121569.379445 mapread 0x3190f9 thru 0x3198fb (0x803 bytes) 000301[1] 1761121569.413565 read 0x45762f thru 0x457c6d (0x63f bytes) 000302[0] 1761121569.421592 write 0x66a02c thru 0x66bcfe (0x1cd3 bytes) 000303[1] 1761121569.429431 write 0x1b4d8f thru 0x1b56f0 (0x962 bytes) 000304[0] 1761121569.439786 trunc from 0x828ec8 to 0x83a0d8 (0x11211 bytes) 000305[0] 1761121569.448572 trunc from 0x83a0d9 to 0x4c4187 (0x375f51 bytes) 000306[1] 1761121569.452663 read 0x00938d thru 0x009b7e (0x7f2 bytes) 000309[0] 1761121569.472972 mapwrite 0x6bafa9 thru 0x6bb576 (0x5ce bytes) 000312[0] 1761121569.508642 mapwrite 0x76fc24 thru 0x771b94 (0x1f71 bytes) 000314[0] 1761121569.539608 read 0x05b47e thru 0x05bdb6 (0x939 bytes) 000315[0] 1761121569.590299 trunc from 0x771b95 to 0x2809c1 (0x4f11d3 bytes) 000316[0] 1761121569.601488 mapwrite 0x5f2157 thru 0x5f3419 (0x12c3 bytes) 000318[1] 1761121569.629474 trunc from 0x5f341a to 0x58d411 (0x66008 bytes) 000321[0] 1761121569.653429 mapread 0x40f10e thru 0x40fd87 (0xc7a bytes) 000322[0] 1761121569.666060 write 0x3d066b thru 0x3d204b (0x19e1 bytes) 000323[1] 1761121569.675459 trunc from 0x58d412 to 0x56f288 (0x1e189 bytes) 000325[0] 1761121569.692576 trunc from 0x56f289 to 0x4819f2 (0xed896 bytes) 000327[0] 1761121569.701907 trunc from 0x4819f3 to 0x6dcbeb (0x25b1f9 bytes) 000330[1] 1761121569.717835 trunc from 0x6dcbec to 0x1f0d67 (0x4ebe84 bytes) 000331[1] 1761121569.728465 mapread 0x10ebd8 thru 0x10ed93 (0x1bc bytes) 000332[0] 1761121569.738184 write 0x9302e0 thru 0x930d51 (0xa72 bytes) 000333[1] 1761121569.748268 mapwrite 0x8c6cbf thru 0x8c71d2 (0x514 bytes) 000334[1] 1761121569.774329 read 0x292396 thru 0x292f50 (0xbbb bytes) 000335[1] 1761121569.784304 mapread 0x4553aa thru 0x4554a1 (0xf8 bytes) 000337[0] 1761121569.795351 write 0x73f3c4 thru 0x73f490 (0xcd bytes) 000339[1] 1761121569.807992 trunc from 0x930d52 to 0x92823e (0x8b13 bytes) 000340[0] 1761121569.822432 mapread 0x16c908 thru 0x16e6f9 (0x1df2 bytes) 000341[0] 1761121569.834214 mapwrite 0x37b938 thru 0x37b9b5 (0x7e bytes) 000344[0] 1761121569.864494 mapwrite 0x62ec74 thru 0x62f148 (0x4d5 bytes) 000346[1] 1761121569.894206 write 0x86e8fb thru 0x86f29d (0x9a3 bytes) 000349[0] 1761121569.910763 trunc from 0x92823f to 0x75220d (0x1d6031 bytes) 000350[1] 1761121569.919649 write 0x0a0c64 thru 0x0a1d1f (0x10bc bytes) 000351[0] 1761121569.928636 mapread 0x2c4239 thru 0x2c4ff6 (0xdbe bytes) 000352[1] 1761121569.940438 trunc from 0x75220e to 0x1973ab (0x5bae62 bytes) 000354[1] 1761121569.956625 mapwrite 0x2be730 thru 0x2c0223 (0x1af4 bytes) 000357[0] 1761121569.987826 read 0x245675 thru 0x246ecb (0x1857 bytes) 000358[1] 1761121569.999274 mapread 0x270089 thru 0x271dc1 (0x1d39 bytes) 000359[1] 1761121570.003169 mapread 0x0e4e4c thru 0x0e5047 (0x1fc bytes) 000360[0] 1761121570.009416 write 0x54d6bf thru 0x54dad3 (0x415 bytes) 000363[0] 1761121570.020667 mapwrite 0x458f77 thru 0x45a1b4 (0x123e bytes) 000364[0] 1761121570.058392 read 0x3ad1f9 thru 0x3ad6f4 (0x4fc bytes) 000367[1] 1761121570.070938 mapwrite 0x704518 thru 0x704f4d (0xa36 bytes) 000372[0] 1761121570.135428 mapread 0x4ed265 thru 0x4ee74c (0x14e8 bytes) 000374[0] 1761121570.150376 mapwrite 0x0d72f2 thru 0x0d82f1 (0x1000 bytes) 000376[0] 1761121570.178148 mapread 0x43d8c9 thru 0x43d9ac (0xe4 bytes) 000377[0] 1761121570.185144 write 0x8ac741 thru 0x8ade7a (0x173a bytes) 000382[0] 1761121570.204373 trunc from 0x8ade7b to 0x8de51f (0x306a5 bytes) 000383[0] 1761121570.218544 write 0x4c5ddd thru 0x4c6595 (0x7b9 bytes) 000384[1] 1761121570.222513 mapread 0x0f323f thru 0x0f3988 (0x74a bytes) 000385[0] 1761121570.231613 trunc from 0x8de520 to 0x82fda2 (0xae77d bytes) 000386[1] 1761121570.238755 mapread 0x759c44 thru 0x75a337 (0x6f4 bytes) 000389[0] 1761121570.252411 mapwrite 0x52789d thru 0x529199 (0x18fd bytes) 000390[1] 1761121570.272545 mapread 0x241213 thru 0x2426fd (0x14eb bytes) 000391[0] 1761121570.283672 trunc from 0x82fda3 to 0x0f71e9 (0x738bb9 bytes) 000392[0] 1761121570.291282 trunc from 0x0f71ea to 0x7e23d6 (0x6eb1ed bytes) 000393[1] 1761121570.296754 mapread 0x6cafcc thru 0x6cbd84 (0xdb9 bytes) 000396[1] 1761121570.310093 read 0x54874b thru 0x5489a1 (0x257 bytes) 000400[0] 1761121570.334458 mapread 0x17c50b thru 0x17cf8c (0xa82 bytes) 000401[1] 1761121570.336498 write 0x3a2b05 thru 0x3a4a3b (0x1f37 bytes) 000402[1] 1761121570.343552 read 0x2f5187 thru 0x2f6817 (0x1691 bytes) 000403[0] 1761121570.372621 write 0x6a5e92 thru 0x6a77bb (0x192a bytes) 000408[0] 1761121570.399780 mapwrite 0x4eeb98 thru 0x4ef4fc (0x965 bytes) 000409[1] 1761121570.422495 write 0x7fbb1f thru 0x7fc15e (0x640 bytes) 000410[1] 1761121570.426505 read 0x031afc thru 0x0322d3 (0x7d8 bytes) 000411[1] 1761121570.466293 write 0x7a2698 thru 0x7a2e52 (0x7bb bytes) 000412[0] 1761121570.470408 read 0x1f5820 thru 0x1f7388 (0x1b69 bytes) 000413[0] 1761121570.520527 read 0x0f0aee thru 0x0f0d11 (0x224 bytes) 000416[1] 1761121570.528440 mapread 0x29c0d0 thru 0x29d4d7 (0x1408 bytes) 000417[0] 1761121570.529172 mapwrite 0x73158c thru 0x731974 (0x3e9 bytes) 000418[1] 1761121570.570499 mapread 0x2f7511 thru 0x2f85d9 (0x10c9 bytes) 000419[1] 1761121570.572882 trunc from 0x7fc15f to 0x572cfe (0x289460 bytes) 000420[1] 1761121570.592768 read 0x36619b thru 0x367b52 (0x19b8 bytes) 000422[1] 1761121570.611502 read 0x52ba2a thru 0x52ccfa (0x12d1 bytes) 000424[1] 1761121570.612132 trunc from 0x572cff to 0x8ad057 (0x33a359 bytes) 000426[1] 1761121570.617123 write 0x4b5409 thru 0x4b5eb9 (0xab1 bytes) 000427[0] 1761121570.622584 trunc from 0x8ad058 to 0x4ce337 (0x3ded20 bytes) 000428[1] 1761121570.639496 mapread 0x49ac96 thru 0x49b19d (0x508 bytes) 000430[1] 1761121570.653171 mapread 0x3e4a72 thru 0x3e5a3a (0xfc9 bytes) 000431[0] 1761121570.659745 write 0x06a47a thru 0x06b3c3 (0xf4a bytes) 000434[1] 1761121570.674942 mapread 0x37dc61 thru 0x37f8b7 (0x1c57 bytes) 000436[0] 1761121570.688277 trunc from 0x4ce338 to 0x0559a3 (0x478994 bytes) 000437[1] 1761121570.696535 mapwrite 0x21f6a7 thru 0x220da9 (0x1703 bytes) 000440[0] 1761121570.732420 trunc from 0x220daa to 0x617afa (0x3f6d51 bytes) 000442[0] 1761121570.743592 write 0x66112a thru 0x661de4 (0xcbb bytes) 000443[0] 1761121570.749501 write 0x58d928 thru 0x58ebf3 (0x12cc bytes) 000444[1] 1761121570.752106 mapwrite 0x43d55c thru 0x43f549 (0x1fee bytes) truncating to largest ever: 0x9f98c5 000446[0] 1761121570.780335 trunc from 0x661de5 to 0x9f98c4 (0x397ae0 bytes) 000447[1] 1761121570.788674 trunc from 0x9f98c5 to 0x8d1744 (0x128180 bytes) 000448[1] 1761121570.799881 read 0x06e94c thru 0x06fadf (0x1194 bytes) 000449[0] 1761121570.852027 mapread 0x03e25c thru 0x03fafe (0x18a3 bytes) 000450[0] 1761121570.865742 mapwrite 0x70697b thru 0x707181 (0x807 bytes) 000452[1] 1761121570.894545 write 0x5b64fd thru 0x5b7d73 (0x1877 bytes) 000453[0] 1761121570.899830 read 0x446333 thru 0x446bd3 (0x8a1 bytes) 000456[0] 1761121570.911126 trunc from 0x8d1745 to 0x30c2b5 (0x5c548f bytes) 000457[1] 1761121570.927755 mapread 0x0dd41a thru 0x0de3c8 (0xfaf bytes) 000458[1] 1761121570.938916 read 0x02dbec thru 0x02fa15 (0x1e2a bytes) 000459[1] 1761121570.950605 mapread 0x1dd7ee thru 0x1decea (0x14fd bytes) 000460[0] 1761121570.977621 read 0x294d04 thru 0x295139 (0x436 bytes) 000461[1] 1761121570.987839 mapread 0x1fc96c thru 0x1fc97a (0xf bytes) 000462[0] 1761121570.988514 trunc from 0x30c2b6 to 0x43a937 (0x12e682 bytes) 000464[1] 1761121571.007990 trunc from 0x43a938 to 0x79d954 (0x36301d bytes) 000466[1] 1761121571.024750 read 0x4275a1 thru 0x4285da (0x103a bytes) 000469[0] 1761121571.060014 trunc from 0x79d955 to 0x5b0da0 (0x1ecbb4 bytes) 000470[0] 1761121571.073749 trunc from 0x5b0da1 to 0x1653ea (0x44b9b6 bytes) 000472[1] 1761121571.077806 trunc from 0x1653eb to 0x2c18db (0x15c4f1 bytes) 000473[0] 1761121571.088329 trunc from 0x2c18dc to 0x541a2d (0x280152 bytes) 000475[1] 1761121571.103853 mapread 0x3ab621 thru 0x3abad9 (0x4b9 bytes) 000476[0] 1761121571.113079 read 0x4cf70f thru 0x4cfa10 (0x302 bytes) 000481[1] 1761121571.121539 read 0x069ce0 thru 0x06b6c4 (0x19e5 bytes) 000483[1] 1761121571.158287 read 0x1cb5bc thru 0x1cd363 (0x1da8 bytes) 000484[1] 1761121571.158784 mapread 0x0523c2 thru 0x052abc (0x6fb bytes) 000488[1] 1761121571.161913 mapwrite 0x3345a6 thru 0x33606f (0x1aca bytes) 000489[1] 1761121571.197717 trunc from 0x541a2e to 0x680ffc (0x13f5cf bytes) 000491[1] 1761121571.215631 mapwrite 0x8857a3 thru 0x886e3c (0x169a bytes) 000492[0] 1761121571.239058 write 0x576394 thru 0x57649a (0x107 bytes) 000493[0] 1761121571.257200 read 0x50c7db thru 0x50e6a4 (0x1eca bytes) 000494[0] 1761121571.261317 mapwrite 0x1a21a9 thru 0x1a3434 (0x128c bytes) 000497[1] 1761121571.292991 read 0x77303e thru 0x773dce (0xd91 bytes) 000498[0] 1761121571.308533 write 0x00728a thru 0x0088f3 (0x166a bytes) 000499[0] 1761121571.318272 read 0x2df159 thru 0x2e0f00 (0x1da8 bytes) All operations completed A-OK! PASS 16b (7s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 04:26:12 (1761121572) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (2s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 04:26:14 (1761121574) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0922143 s, 22.7 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0605867 s, 34.6 MB/s ldlm.namespaces.MGC192.168.204.115@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8fb4c84a4000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8fb4c8790000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8fb4c84a4000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8fb4c8790000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8fb4c84a4000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8fb4c8790000.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.59387 s, 40.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.70746 s, 148 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.784534 s, 134 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.46859 s, 71.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.66059 s, 63.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.12532 s, 93.2 MB/s PASS 16d (12s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 04:26:26 (1761121586) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.0998822 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321793 s, 32.6 MB/s PASS 16e (2s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 04:26:28 (1761121588) TIMEOUT PASS 16f (23s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 04:26:51 (1761121611) TIMEOUT PASS 16g (22s) == sanityn test 16i: read after truncate file ============ 04:27:14 (1761121634) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (2s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 04:27:16 (1761121636) fail_loc=0x8000030a PASS 17 (4s) == sanityn test 18: mmap sanity check =========================================================================================== 04:27:20 (1761121640) mmap test1: basic mmap operation (PASS, 0.018941s) mmap test2: MAP_PRIVATE not write back (PASS, 0.015188s) mmap test3: concurrent mmap ops on two nodes (PASS, 2.4284s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.086s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.02708s) mmap test6: check mmap write/read content on two nodes (PASS, 0.035637s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0145s) PASS 18 (9s) == sanityn test 19: test concurrent uncached read races ========================================================================= 04:27:29 (1761121649) oleg415-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg415-client: oleg415-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 04:27:30 (1761121650) PASS 20 (2s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 04:27:32 (1761121652) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) == sanityn test 23: others should see updated atime while another read============================================================== 04:27:34 (1761121654) now is 1761121655 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 new atime is 1761121716 PASS 23 (64s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 04:28:38 (1761121718) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.3K 376 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.2K 368 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 376 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 368 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 5.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 11.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118072 376 117696 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235728 368 235360 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 376 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 368 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118072 376 117696 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre[OST:1] filesystem_summary: 235728 368 235360 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118072 376 117696 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235728 368 235360 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 PASS 24a (2s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 04:28:40 (1761121720) PASS 24b (2s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 04:28:42 (1761121722) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (2s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 04:28:44 (1761121724) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 04:28:45 (1761121725) utime: good mknod times 1761121725 <= 1761121726 <= 1761121726 for /mnt/lustre/f26a utime: good mknod times 1761121725 <= 1761121726 <= 1761121726 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (3s) == sanityn test 26b: sync mtime between ost and mds ====== 04:28:48 (1761121728) PASS 26b (4s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 04:28:53 (1761121733) 0 PASS 30 (4s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 04:28:57 (1761121737) fail_loc=0x314 PASS 31a (3s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 04:29:00 (1761121740) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg415-server mds-ost sync done. sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.115@tcp:/lustre 7542784 12288 7524352 1% /mnt/lustre2 PASS 31b (19s) == sanityn test 31r: open-rename(replace) race =========== 04:29:19 (1761121759) -rw-r--r-- 1 root root 0 Oct 22 04:29 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (5s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 04:29:24 (1761121764) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 04:29:25 (1761121765) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 04:29:26 (1761121766) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 04:29:27 (1761121767) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 04:29:28 (1761121768) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg415-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8fb4c84a4000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8fb4c8790000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8fb4c84a4000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8fb4c8790000.ost_server_uuid in FULL state after 0 sec oleg415-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8fb4c84a4000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8fb4c8790000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8fb4c84a4000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8fb4c8790000.ost_server_uuid in FULL state after 0 sec oleg415-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8fb4c84a4000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8fb4c8790000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8fb4c84a4000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8fb4c8790000.ost_server_uuid in IDLE state after 0 sec oleg415-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8fb4c84a4000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8fb4c8790000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8fb4c84a4000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8fb4c8790000.ost_server_uuid in IDLE state after 0 sec oleg415-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8fb4c84a4000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8fb4c8790000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8fb4c84a4000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8fb4c8790000.ost_server_uuid in IDLE state after 0 sec oleg415-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8fb4c84a4000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8fb4c8790000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8fb4c84a4000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8fb4c8790000.ost_server_uuid in IDLE state after 0 sec PASS 34 (78s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 04:30:46 (1761121846) fail_loc=0x80000317 Race attempt 0 Wait for 44698 44826 for 60 sec... total: 4000 open/close in 8.13 seconds: 492.17 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.204.115@tcp:/lustre 7.2G 12M 7.2G 1% /mnt/lustre 192.168.204.115@tcp:/lustre 7.2G 12M 7.2G 1% /mnt/lustre2 PASS 35 (65s) == sanityn test 36: handle ESTALE/open-unlink correctly == 04:31:51 (1761121911) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg415-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.374185 s, 140 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.36719 s, 143 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.382803 s, 137 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.359258 s, 146 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.359639 s, 146 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.472647 s, 111 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.409029 s, 128 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.379253 s, 138 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.443212 s, 118 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.367984 s, 142 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.393023 s, 133 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7526400) after_dd(7475200) after(7526400) PASS 36 (251s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 04:36:02 (1761122162) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 - create 9106 (time 1761122174.15 total 10.00 last 910.51) total: 10000 create in 10.88 seconds: 919.05 ops/second PASS 37 (15s) == sanityn test 39a: test from 11063 ============================================================================================ 04:36:17 (1761122177) repeat after cancel_lru_locks PASS 39a (2s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 04:36:19 (1761122179) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 04:36:22 (1761122182) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 04:36:25 (1761122185) fail_loc=0x411 fail_loc=0 PASS 39d (2s) == sanityn test 40a: pdirops: create vs others ======================================================================== 04:36:27 (1761122187) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (8s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 04:36:35 (1761122195) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (8s) == sanityn test 40c: pdirops: link and others ======================================================================== 04:36:43 (1761122203) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (8s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 04:36:51 (1761122211) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (8s) == sanityn test 40e: pdirops: rename and others ======================================================================== 04:36:59 (1761122219) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (7s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 04:37:07 (1761122227) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (5s) == sanityn test 41b: pdirops: create vs create ======================================================================== 04:37:12 (1761122232) open: File exists Conflict PASS 41b (5s) == sanityn test 41c: pdirops: create vs link ======================================================================== 04:37:17 (1761122237) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (5s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 04:37:22 (1761122242) Conflict PASS 41d (5s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 04:37:27 (1761122247) Conflict PASS 41e (5s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 04:37:32 (1761122252) Conflict PASS 41f (5s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 04:37:37 (1761122257) Conflict PASS 41g (5s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 04:37:42 (1761122262) Conflict PASS 41h (5s) == sanityn test 41i: reint_open: create vs create ======== 04:37:47 (1761122267) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (807s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 04:51:14 (1761123074) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (8s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 04:51:23 (1761123083) open: File exists Conflict PASS 42b (7s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 04:51:30 (1761123090) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (8s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 04:51:38 (1761123098) Conflict PASS 42d (7s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 04:51:45 (1761123105) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (8s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 04:51:53 (1761123113) Conflict PASS 42f (8s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 04:52:01 (1761123121) Conflict PASS 42g (8s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 04:52:09 (1761123129) Conflict PASS 42h (7s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 04:52:16 (1761123136) PASS 43a (27s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 04:52:43 (1761123163) Conflict PASS 43b (8s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 04:52:51 (1761123171) Conflict PASS 43c (8s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 04:52:59 (1761123179) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (7s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 04:53:06 (1761123186) Conflict PASS 43e (7s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 04:53:13 (1761123193) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (8s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 04:53:21 (1761123201) stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (7s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 04:53:28 (1761123208) Conflict PASS 43h (7s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 04:53:35 (1761123215) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (2s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 04:53:37 (1761123217) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (72s) == sanityn test 43k: unlink vs create ==================== 04:54:49 (1761123289) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (699s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 05:06:28 (1761123988) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Wed Oct 22 05:06:34 EDT 2025 PASS 44a (9s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 05:06:37 (1761123997) open: File exists Conflict PASS 44b (9s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 05:06:46 (1761124006) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (11s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 05:06:57 (1761124017) Conflict PASS 44d (11s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 05:07:08 (1761124028) Conflict PASS 44e (11s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 05:07:19 (1761124039) Conflict PASS 44f (11s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 05:07:30 (1761124050) Conflict PASS 44g (11s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 05:07:41 (1761124061) Conflict PASS 44h (10s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 05:07:51 (1761124071) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (2s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 05:07:53 (1761124073) PASS 45a (84s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 05:09:17 (1761124157) Conflict PASS 45b (9s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 05:09:26 (1761124166) Conflict PASS 45c (9s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 05:09:36 (1761124176) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (8s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 05:09:44 (1761124184) Conflict PASS 45e (9s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 05:09:53 (1761124193) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (8s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 05:10:01 (1761124201) stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (8s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 05:10:09 (1761124209) Conflict PASS 45h (8s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 05:10:17 (1761124217) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (2s) == sanityn test 45j: read vs rename ====================== 05:10:19 (1761124219) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (668s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 05:21:27 (1761124887) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (6s) == sanityn test 46b: pdirops: link vs create ======================================================================== 05:21:33 (1761124893) open: File exists Conflict PASS 46b (7s) == sanityn test 46c: pdirops: link vs link ======================================================================== 05:21:40 (1761124900) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (6s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 05:21:46 (1761124906) Conflict PASS 46d (7s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 05:21:53 (1761124913) Conflict PASS 46e (7s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 05:22:00 (1761124920) Conflict PASS 46f (6s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 05:22:06 (1761124926) Conflict PASS 46g (7s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 05:22:13 (1761124933) Conflict PASS 46h (7s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 05:22:20 (1761124940) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 05:22:21 (1761124941) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (2s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 05:22:23 (1761124943) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 05:22:24 (1761124944) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 05:22:26 (1761124946) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (2s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 05:22:27 (1761124947) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 05:22:28 (1761124948) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (2s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 05:22:30 (1761124950) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 05:22:31 (1761124951) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.0134645 s, 761 kB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (8s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 05:22:39 (1761124959) 0+1 records in 0+1 records out 158 bytes copied, 0.0142843 s, 11.1 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (243172) - No such process PASS 51a (5s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 05:22:44 (1761124964) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00830867 s, 123 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0410708 s, 25.5 MB/s 3145728 PASS 51b (15s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 05:22:59 (1761124979) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00569746 s, 180 kB/s PASS 51c (8s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 05:23:07 (1761124987) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174471 s, 60.1 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (4s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 05:23:11 (1761124991) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2433 0x981 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2433 0x981 0 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 05:23:15 (1761124995) llite.lustre-ffff8fb4c84a4000.xattr_cache=1 llite.lustre-ffff8fb4c8790000.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504962 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 05:23:16.000000000 -0400 Modify: 2025-10-22 05:23:16.000000000 -0400 Change: 2025-10-22 05:23:16.000000000 -0400 Birth: 2025-10-22 05:23:15.000000000 -0400 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504965 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 05:23:22.000000000 -0400 Modify: 2025-10-22 05:23:22.000000000 -0400 Change: 2025-10-22 05:23:22.000000000 -0400 Birth: 2025-10-22 05:23:21.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 05:23:27.000000000 -0400 Modify: 2025-10-22 05:23:27.000000000 -0400 Change: 2025-10-22 05:23:27.000000000 -0400 Birth: 2025-10-22 05:23:27.000000000 -0400 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504967 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 05:23:27.000000000 -0400 Modify: 2025-10-22 05:23:28.000000000 -0400 Change: 2025-10-22 05:23:28.000000000 -0400 Birth: 2025-10-22 05:23:27.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504970 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 05:23:33.000000000 -0400 Modify: 2025-10-22 05:23:33.000000000 -0400 Change: 2025-10-22 05:23:33.000000000 -0400 Birth: 2025-10-22 05:23:32.000000000 -0400 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 05:23:33.000000000 -0400 Modify: 2025-10-22 05:23:33.000000000 -0400 Change: 2025-10-22 05:23:33.000000000 -0400 Birth: 2025-10-22 05:23:32.000000000 -0400 PASS 54 (25s) == sanityn test 55a: rename vs unlink target dir ========= 05:23:40 (1761125020) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (9s) == sanityn test 55b: rename vs unlink source dir ========= 05:23:49 (1761125029) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (8s) == sanityn test 55c: rename vs unlink orphan target dir == 05:23:57 (1761125037) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (13s) == sanityn test 55d: rename file vs link ================= 05:24:10 (1761125050) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (11s) == sanityn test 60: Verify data_version behaviour ======== 05:24:21 (1761125061) PASS 60 (3s) == sanityn test 70a: cd directory && rm directory ======== 05:24:24 (1761125064) PASS 70a (2s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 05:24:26 (1761125066) PASS 70b (3s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 05:24:29 (1761125069) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0042367 s, 15.5 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (2s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 05:24:31 (1761125071) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0069161 s, 9.5 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 05:24:32 (1761125072) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (2s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 05:24:34 (1761125074) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00558949 s, 11.7 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 05:24:35 (1761125075) llite.lustre-ffff8fb4c84a4000.xattr_cache=1 llite.lustre-ffff8fb4c8790000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (3s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 05:24:38 (1761125078) llite.lustre-ffff8fb4c84a4000.xattr_cache=1 llite.lustre-ffff8fb4c8790000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (3s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 05:24:41 (1761125081) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00552765 s, 185 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0191962 s, 53.3 kB/s 253206: get lock2 253207: get lock1 253206: done 253206: get lock1 253207: done 253207: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 253207: cannot get lock2: Resource deadlock avoided 253207: exit rc=0 fcntl cmd 7 failed: Resource deadlock avoided 253206: cannot get lock1: Resource deadlock avoided 253206: done 253206: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 05:24:47 (1761125087) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.027104 s, 77.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128842 s, 81.4 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 253781 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (8s) == sanityn test 76: Verify MDT open_files listing ======== 05:24:55 (1761125095) opening files: ........ closing files: ........ PASS 76 (24s) == sanityn test 77a: check FIFO NRS policy =============== 05:25:19 (1761125119) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.14832 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0532368 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0456436 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0445335 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0614827 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0667706 s, 15.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823362 s, 12.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0515727 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0842689 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0866266 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0697756 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0933514 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0569014 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0957076 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0924282 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0912924 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0873078 s, 12.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8fb4c8790000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8fb4c8790000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0527549 s, 19.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0458711 s, 22.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0474155 s, 22.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0420388 s, 24.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0493116 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.057927 s, 18.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0676641 s, 15.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0568458 s, 18.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0710389 s, 14.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0677786 s, 15.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0777688 s, 13.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0465453 s, 22.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.052827 s, 19.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0494446 s, 21.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0543638 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0551862 s, 19.0 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg415-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg415-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (4s) == sanityn test 77b: check CRR-N NRS policy ============== 05:25:23 (1761125123) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.151332 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0815293 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0799306 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0972393 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110367 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135598 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140185 s, 7.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129036 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126755 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122251 s, 8.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132324 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0972109 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147372 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157362 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128885 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15588 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138009 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130538 s, 80.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232747 s, 45.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164656 s, 63.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178209 s, 58.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202011 s, 51.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236757 s, 44.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203657 s, 51.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230516 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263988 s, 39.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259292 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114194 s, 91.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115979 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198122 s, 52.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743817 s, 141 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290237 s, 36.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8fb4c8790000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0343749 s, 30.5 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg415-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg415-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.152779 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0751832 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100752 s, 10.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140518 s, 7.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129081 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130683 s, 8.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127461 s, 8.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132707 s, 7.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146292 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129284 s, 8.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135713 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146735 s, 7.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138278 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170631 s, 6.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129108 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148845 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155496 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0351402 s, 29.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147101 s, 71.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0353341 s, 29.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297699 s, 35.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293992 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0404951 s, 25.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248447 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0418543 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124186 s, 84.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205671 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.03165 s, 33.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015035 s, 69.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210245 s, 49.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0502175 s, 20.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0436465 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143717 s, 73.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8fb4c8790000.lock_unused_count=1 removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg415-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg415-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (5s) == sanityn test 77c: check ORR NRS policy ================ 05:25:28 (1761125128) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.142359 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0499246 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0725027 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0761789 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0723588 s, 14.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0967803 s, 10.8 MB/sldlm.namespaces.lustre-OST0000-osc-ffff8fb4c8790000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102342 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0940401 s, 11.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0997635 s, 10.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105328 s, 10.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100828 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0913317 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0831792 s, 12.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0992697 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127934 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0948611 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100966 s, 10.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238627 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169757 s, 61.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193636 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223918 s, 46.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261879 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272617 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175783 s, 59.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271857 s, 38.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021986 s, 47.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015723 s, 66.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116499 s, 90.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251442 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117558 s, 89.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006555 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00358324 s, 293 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683297 s, 153 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8fb4c8790000.lock_unused_count=1 removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg415-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg415-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.141995 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336252 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365579 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0677056 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101318 s, 10.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0805494 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0945345 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103748 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121981 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105255 s, 10.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119599 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111352 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12502 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114012 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104611 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117402 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138975 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104218 s, 101 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0395306 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254449 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984466 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795833 s, 132 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106683 s, 98.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105419 s, 99.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279141 s, 37.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286743 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277575 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00440747 s, 238 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341362 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0468211 s, 22.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0597779 s, 17.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289017 s, 36.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293855 s, 35.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8fb4c8790000.lock_unused_count=5 removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg415-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg415-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.198846 s, 84.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0406177 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0349941 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0487483 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0593923 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0400214 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0611983 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0388833 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.069292 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117073 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118844 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102039 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0926542 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0959714 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127086 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0989316 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125445 s, 8.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309712 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231903 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308936 s, 33.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305113 s, 34.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191448 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201779 s, 52.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207404 s, 50.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258385 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269509 s, 38.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233873 s, 44.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211518 s, 49.6 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff8fb4c8790000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff8fb4c8790000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.040671 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973003 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144996 s, 72.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0355486 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674163 s, 156 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg415-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg415-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (7s) == sanityn test 77d: check TRR nrs policy ================ 05:25:35 (1761125135) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.187199 s, 89.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0649443 s, 16.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0546307 s, 19.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372697 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0857661 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0719928 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0891867 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0968895 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0909984 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104346 s, 10.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8fb4c8790000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13048 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273856 s, 38.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146385 s, 71.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131301 s, 79.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0404518 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301844 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154932 s, 6.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585456 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165388 s, 63.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322159 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155872 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128159 s, 81.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029152 s, 36.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024852 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184839 s, 5.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326213 s, 32.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123519 s, 84.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194814 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206791 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00925369 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19741 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114117 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209271 s, 5.0 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg415-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg415-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.147563 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0440782 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0352407 s, 29.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0489385 s, 21.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0429393 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0471154 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0376986 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0669706 s, 15.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0506559 s, 20.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.063542 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108998 s, 9.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8fb4c8790000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0973843 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100684 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0963554 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13378 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109735 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143704 s, 7.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287519 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330291 s, 31.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335093 s, 31.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271983 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021682 s, 48.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315086 s, 33.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153357 s, 68.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302315 s, 34.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273045 s, 38.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275468 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309344 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330893 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305155 s, 34.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307856 s, 34.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8fb4c8790000.lock_unused_count=4 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320207 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0343745 s, 30.5 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg415-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg415-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.139793 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314327 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0378081 s, 27.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8fb4c8790000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.087861 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0779035 s, 13.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0870905 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103788 s, 10.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105326 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111278 s, 9.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105426 s, 9.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114449 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104359 s, 10.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112655 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102913 s, 10.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0981043 s, 10.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138257 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139065 s, 7.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00423573 s, 248 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00446505 s, 235 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082798 s, 127 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238708 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266995 s, 39.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017487 s, 60.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178561 s, 58.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026973 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564599 s, 186 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315203 s, 33.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00321014 s, 327 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00321019 s, 327 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948506 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00532235 s, 197 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850306 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00301954 s, 347 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8fb4c8790000.lock_unused_count=2 removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg415-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg415-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (6s) == sanityn test 77e: check TBF NID nrs policy ============ 05:25:41 (1761125141) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.204.15@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.119086 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328983 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0364737 s, 28.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0885141 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0761906 s, 13.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0931404 s, 11.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102176 s, 10.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.080144 s, 13.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0904582 s, 11.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100786 s, 10.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0772395 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101297 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0924358 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0968236 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102353 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0900722 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0964472 s, 10.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944133 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018632 s, 56.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184616 s, 56.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174002 s, 60.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124424 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192906 s, 54.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224857 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237456 s, 44.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258487 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00301412 s, 348 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205011 s, 51.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234845 s, 44.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235445 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212303 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0342148 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247705 s, 42.3 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg415-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg415-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.153924 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282825 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365268 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0911527 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0861589 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0794523 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0852982 s, 12.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0820304 s, 12.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113309 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10329 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115128 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131283 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0977378 s, 10.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114591 s, 9.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130056 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113102 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118882 s, 8.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235911 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248488 s, 42.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205427 s, 51.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211267 s, 49.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249964 s, 41.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171158 s, 61.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192364 s, 54.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214658 s, 48.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173423 s, 60.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210159 s, 49.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210355 s, 49.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277174 s, 37.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253601 s, 41.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298106 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00403183 s, 260 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00510431 s, 205 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg415-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg415-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.201214 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330682 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0367177 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0944077 s, 11.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105589 s, 9.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112554 s, 9.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0987916 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111184 s, 9.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106381 s, 9.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8fb4c8790000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff8fb4c8790000.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109051 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117942 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10557 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109723 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127791 s, 8.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114285 s, 9.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110882 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0972586 s, 10.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170001 s, 61.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214828 s, 48.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211793 s, 49.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026734 s, 39.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285968 s, 36.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200513 s, 52.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285067 s, 36.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221263 s, 47.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234342 s, 44.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249848 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236083 s, 44.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260977 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030345 s, 34.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248458 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282924 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319688 s, 32.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8fb4c8790000.lock_unused_count=3 removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg415-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg415-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.133692 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0592931 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0675577 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0862158 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0753888 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0849406 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.094579 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102615 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1069 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0930148 s, 11.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104493 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0941204 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0894825 s, 11.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0792237 s, 13.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.071037 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0780925 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100612 s, 10.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159523 s, 65.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280395 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165193 s, 63.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177983 s, 58.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182336 s, 57.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286545 s, 36.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238358 s, 44.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215546 s, 48.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237854 s, 44.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310584 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027228 s, 38.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278883 s, 37.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298582 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031942 s, 32.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312687 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0318676 s, 32.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8fb4c8790000.lock_unused_count=2 removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg415-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg415-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (9s) == sanityn test 77f: check TBF JobID nrs policy ========== 05:25:50 (1761125150) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg415-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.156807 s, 107 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0431652 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0366635 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296965 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298321 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0597944 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0442793 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0476268 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0661028 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.042389 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0923883 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104114 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104451 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109303 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0891825 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111063 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10281 s, 10.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194019 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121547 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158594 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180687 s, 58.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205493 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157374 s, 66.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280238 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197624 s, 53.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210565 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224159 s, 46.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236478 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240688 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025357 s, 41.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259918 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025464 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261462 s, 40.1 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg415-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg415-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg415-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.136546 s, 123 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0450839 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0621647 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0835868 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.090765 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0881485 s, 11.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0786438 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0888756 s, 11.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0673589 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0775575 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.089231 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0760047 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0992294 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0851921 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118854 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0949393 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108994 s, 9.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020686 s, 50.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177326 s, 59.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159211 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141087 s, 74.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175342 s, 59.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181724 s, 57.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265437 s, 39.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224084 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025127 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234527 s, 44.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205014 s, 51.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242083 s, 43.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255579 s, 41.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226365 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219286 s, 47.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272553 s, 38.5 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg415-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg415-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg415-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.146224 s, 115 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0436124 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0689916 s, 15.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0766232 s, 13.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0722951 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0621677 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0662347 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0784336 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.091535 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0912807 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0953207 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109267 s, 9.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0845656 s, 12.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0853943 s, 12.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100827 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.095872 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0956027 s, 11.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107664 s, 97.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269094 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144261 s, 72.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147914 s, 70.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01649 s, 63.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195826 s, 53.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253705 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199383 s, 52.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208269 s, 50.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241341 s, 43.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240893 s, 43.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267824 s, 39.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259018 s, 40.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263212 s, 39.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255872 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276869 s, 37.9 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg415-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg415-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg415-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.161245 s, 104 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0363453 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0364993 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205701 s, 51.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158368 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164555 s, 6.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167498 s, 6.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167771 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181395 s, 5.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0348689 s, 30.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022117 s, 47.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272073 s, 38.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304657 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314059 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316192 s, 33.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0737048 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0693773 s, 15.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0408442 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0391918 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0743265 s, 14.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0495359 s, 21.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0454606 s, 23.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0455233 s, 23.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.219832 s, 4.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22032 s, 4.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.219297 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.225359 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220004 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.235752 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237954 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.034071 s, 30.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22797 s, 4.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219598 s, 47.7 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg415-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg415-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (9s) == sanityn test 77g: Change TBF type directly ============ 05:25:59 (1761125159) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (5s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 05:26:04 (1761125164) oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg415-client: oleg415-server: ssh exited with exit code 2 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg415-client: oleg415-server: ssh exited with exit code 12 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg415-client: oleg415-server: ssh exited with exit code 12 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg415-client: oleg415-server: ssh exited with exit code 12 PASS 77h (4s) == sanityn test 77i: Change rank of TBF rule ============= 05:26:08 (1761125168) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (9s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 05:26:17 (1761125177) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.126717 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296982 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033273 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0350133 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0735084 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0883125 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0865002 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.081753 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0719599 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0957073 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0688579 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0981268 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0887345 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0895274 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0792314 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108108 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0918392 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159664 s, 65.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127956 s, 81.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123025 s, 85.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163242 s, 64.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180142 s, 58.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184036 s, 57.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205501 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160117 s, 65.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00375509 s, 279 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00376765 s, 278 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760449 s, 138 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235962 s, 44.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011336 s, 92.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264618 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628635 s, 167 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00162388 s, 646 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg415-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg415-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.95125 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4438 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (56s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 05:27:13 (1761125233) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg415-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.668159 s, 25.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141853 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167538 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153185 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166504 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163971 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166564 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183786 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170543 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.341465 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.34268 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.370226 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.380626 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.369892 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.386428 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.380436 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.392975 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139901 s, 7.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136293 s, 7.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137036 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136346 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141037 s, 7.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145334 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138758 s, 7.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139355 s, 7.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147315 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143833 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142253 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144761 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146452 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150114 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145019 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157112 s, 6.7 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg415-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg415-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.7101 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9744 s, 5.2 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg415-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.660993 s, 25.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13819 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158496 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181513 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177137 s, 5.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172851 s, 6.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167699 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181106 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171981 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.339071 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.355459 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.347592 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.369891 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.37719 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.382798 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.366401 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.363709 s, 2.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132864 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125006 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128575 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129633 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128948 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135374 s, 7.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13434 s, 7.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130226 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138161 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140642 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138228 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140764 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137294 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136361 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136221 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139873 s, 7.5 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg415-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg415-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.7244 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.03 s, 5.2 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (147s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 05:29:40 (1761125380) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg415-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.587145 s, 28.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0526523 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0390032 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0556892 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0926571 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.287798 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.284569 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.302713 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.319204 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.299428 s, 3.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.407231 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.414448 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.393263 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.400213 s, 2.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.435509 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.443546 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.511813 s, 2.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8fb4c8790000.lock_unused_count=1 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365149 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0404025 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0763059 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151844 s, 69.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024818 s, 42.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282891 s, 37.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275929 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0509663 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0377023 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142073 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778955 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243075 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00239112 s, 439 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309752 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129306 s, 81.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171753 s, 61.1 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg415-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg415-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.59093 s, 12.2 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.91652 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.15@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.15@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.351839 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220684 s, 47.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0626308 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0635819 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.047153 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0939166 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144545 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155479 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147818 s, 7.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188075 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13171 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177018 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106637 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188739 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209018 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164767 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230134 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.060005 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372968 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338138 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0844641 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107879 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0467422 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0551789 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860245 s, 12.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0724653 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0876268 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0806059 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0846407 s, 12.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0694809 s, 15.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0814061 s, 12.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0733064 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0764611 s, 13.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg415-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg415-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.32682 s, 16.6 MB/s Write runtime is 8 s, speed is 12.500000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.9244 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.15@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg415-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.289332 s, 58.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0429212 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0563646 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0413265 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0954304 s, 11.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150173 s, 7.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127755 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174757 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180363 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170114 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186077 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208654 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17425 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222767 s, 4.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185178 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199154 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.210927 s, 5.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00258639 s, 405 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0859422 s, 12.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.067006 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101758 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0746057 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0826473 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554592 s, 189 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0832839 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0669383 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0465765 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0436097 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0542099 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0397203 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0342131 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275849 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230861 s, 45.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg415-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg415-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.23325 s, 16.8 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.09389 s, 20.6 MB/s Read runtime is 7 s, speed is 14.285714 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg415-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.427294 s, 39.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0508461 s, 20.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0839622 s, 12.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103002 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0944488 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10697 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0902847 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115481 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107528 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130128 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160987 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141628 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181145 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183945 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255538 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.284007 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.28916 s, 3.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00330532 s, 317 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202445 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237493 s, 44.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289978 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0367015 s, 28.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0510857 s, 20.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796583 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00209437 s, 501 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013909 s, 75.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0451472 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0619669 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0522422 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0473936 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0352577 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0535522 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.065083 s, 16.1 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg415-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg415-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.0234 s, 10.5 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.8508 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.1657 s, 17.0 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.89752 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg415-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.783039 s, 21.4 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0656242 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106389 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125887 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147433 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189158 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154672 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.252456 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.219778 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.268283 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.359315 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.398374 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.410179 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.476773 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.500817 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.507086 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.546305 s, 1.9 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167508 s, 62.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0048614 s, 216 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0932744 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11201 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00211608 s, 496 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0949302 s, 11.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0949817 s, 11.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116838 s, 9.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0917538 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0485887 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145473 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105824 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0517721 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326464 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0408168 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0842553 s, 12.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg415-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg415-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8393 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6436 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg415-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.480811 s, 34.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0675188 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0615568 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0788687 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0877793 s, 11.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165466 s, 6.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150973 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132461 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118193 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183007 s, 5.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182452 s, 5.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211954 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228459 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251699 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255052 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.288571 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.269486 s, 3.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244688 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832051 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.041687 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292504 s, 35.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341153 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0489478 s, 21.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197691 s, 53.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961878 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0640731 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158237 s, 66.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.004187 s, 250 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199605 s, 52.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0577052 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657614 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156958 s, 66.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00987891 s, 106 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg415-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg415-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 11.0706 s, 9.5 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.0575 s, 10.4 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg415-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.505896 s, 33.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0571131 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0730619 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0739808 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101243 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173596 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152287 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207332 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220045 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189902 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182441 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.246216 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187404 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224419 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248596 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.278505 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.263484 s, 4.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0493171 s, 21.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0380097 s, 27.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0540862 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.055251 s, 19.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0623914 s, 16.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.062033 s, 16.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0478907 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0944954 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0496357 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0521106 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0527772 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0511284 s, 20.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0524138 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0703429 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0773907 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0889575 s, 11.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg415-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg415-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.3544 s, 10.1 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.93879 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.0426 s, 10.4 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.91565 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (422s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 05:36:42 (1761125802) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (7s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 05:36:49 (1761125809) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg415-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.34973 s, 313 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg415-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.1149 s, 261 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (55s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 05:37:44 (1761125864) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.269628 s, 62.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0665684 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.068232 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0590277 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131573 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208906 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.231927 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224014 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223892 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.293915 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.314365 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.298742 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.307129 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.331344 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.331139 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.333754 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163232 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.412196 s, 2.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8fb4c8790000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff8fb4c8790000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00324433 s, 323 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060622 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643738 s, 163 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00299728 s, 350 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171134 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0796863 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131545 s, 8.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713461 s, 147 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013556 s, 77.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.051293 s, 20.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0404324 s, 25.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182876 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319675 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265383 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253692 s, 41.3 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg415-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg415-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.14914 s, 17.1 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.04451 s, 20.8 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.368762 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0393214 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0485284 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033047 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138547 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130102 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13231 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177772 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168341 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173019 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189117 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222171 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201926 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239179 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23158 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239034 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248851 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0331286 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0630406 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0591336 s, 17.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0445788 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0745904 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.050873 s, 20.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0719122 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0786279 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0619157 s, 16.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0703296 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0638687 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0834025 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0692642 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0689453 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0610627 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111992 s, 9.4 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg415-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg415-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.19889 s, 16.9 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.08396 s, 20.6 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (80s) == sanityn test 77o: Changing rank should not panic ====== 05:39:04 (1761125944) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (10s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 05:39:14 (1761125954) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (87s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 05:40:42 (1761126042) mds.MDS.mdt.nrs_policies=tbf oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg415-client: oleg415-server: ssh exited with exit code 36 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg415-client: oleg415-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (27s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 05:41:08 (1761126068) oleg415-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: Resource temporarily unavailable pdsh@oleg415-client: oleg415-server: ssh exited with exit code 11 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (7s) == sanityn test 79: xattr: intent error ================== 05:41:15 (1761126075) File: /mnt/lustre/d79.sanityn Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505430 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 05:41:16.000000000 -0400 Modify: 2025-10-22 05:41:16.000000000 -0400 Change: 2025-10-22 05:41:16.000000000 -0400 Birth: 2025-10-22 05:41:16.000000000 -0400 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (49s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 05:42:05 (1761126125) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (3s) == sanityn test 80b: Accessing directory during migration ========================================================== 05:42:07 (1761126127) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (4s) == sanityn test 81a: rename and stat under striped directory ========================================================== 05:42:11 (1761126131) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (3s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 05:42:14 (1761126134) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (2s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 05:42:16 (1761126136) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (2s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 05:42:18 (1761126138) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7132 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7132 PASS 82 (5s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 05:42:23 (1761126143) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (2s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 05:42:26 (1761126146) fail_loc=0x8000060b PASS 84 (11s) == sanityn test 90: open/create and unlink striped directory ========================================================== 05:42:37 (1761126157) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (3s) == sanityn test 91: chmod and unlink striped directory === 05:42:39 (1761126159) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (3s) == sanityn test 92: create remote directory under orphan directory ========================================================== 05:42:42 (1761126162) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (2s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 05:42:45 (1761126165) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2623 0xa3f 0 0 2643 0xa53 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2624 0xa40 0 0 2644 0xa54 0 rc1=0 and rc2=0 PASS 93 (14s) == sanityn test 94: signal vs CP callback race =========== 05:42:58 (1761126178) osc.lustre-OST0000-osc-ffff8fb4c84a4000.idle_timeout=0 osc.lustre-OST0000-osc-ffff8fb4c8790000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8fb4c84a4000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8fb4c8790000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.179164 s, 45.7 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 2.98163 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 284187 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.043973 s, 93.1 kB/s osc.lustre-OST0000-osc-ffff8fb4c84a4000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8fb4c8790000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8fb4c84a4000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8fb4c8790000.idle_timeout=debug PASS 94 (14s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 05:43:12 (1761126192) llite.lustre-ffff8fb4c84a4000.fast_read=0 llite.lustre-ffff8fb4c8790000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.00733556 s, 1.7 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8fb4c84a4000.fast_read=1 llite.lustre-ffff8fb4c8790000.fast_read=1 PASS 95b (12s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 05:43:24 (1761126204) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (3s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 05:43:27 (1761126207) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00760673 s, 538 kB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (5s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 05:43:32 (1761126212) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0596498 s, 35.2 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (5s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 05:43:37 (1761126217) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.086751 s, 24.2 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (5s) == sanityn test 100e: DoM: read on open and file size ==== 05:43:42 (1761126222) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.00640126 s, 1.9 MB/s PASS 100e (6s) == sanityn test 101a: Discard DoM data on unlink ========= 05:43:48 (1761126228) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00437531 s, 936 kB/s PASS 101a (6s) == sanityn test 101b: Discard DoM data on rename ========= 05:43:54 (1761126234) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0118988 s, 344 kB/s PASS 101b (5s) == sanityn test 101c: Discard DoM data on close-unlink === 05:43:59 (1761126239) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00065568 s, 6.2 MB/s PASS 101c (7s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 05:44:07 (1761126247) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1761126250 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1761126253 Finishing test test23 at 1761126253 PASS 103 (12s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 05:44:20 (1761126260) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (3s) == sanityn test 105: Glimpse and lock cancel race ======== 05:44:23 (1761126263) fail_loc=0x416 PASS 105 (51s) == sanityn test 106a: Verify the btime via statx() ======= 05:45:14 (1761126314) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (3s) == sanityn test 106b: Glimpse RPCs test for statx ======== 05:45:17 (1761126317) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0102623 s, 99.8 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505458 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 05:45:19.000000000 -0400 Modify: 2025-10-22 05:45:19.000000000 -0400 Change: 2025-10-22 05:45:19.000000000 -0400 Birth: 2025-10-22 05:45:18.000000000 -0400 /mnt/lustre/f106b.sanityn 144115205272505458 -rw-r--r-- 1 0 0 1761126318 1761126319 1761126319 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505458 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 05:45:19.000000000 -0400 Modify: 2025-10-22 05:45:19.000000000 -0400 Change: 2025-10-22 05:45:19.000000000 -0400 Birth: 2025-10-22 05:45:18.000000000 -0400 1761126319 1024 1 PASS 106b (7s) == sanityn test 106c: Verify statx attributes mask ======= 05:45:24 (1761126324) PASS 106c (7s) == sanityn test 107a: Basic grouplock conflict =========== 05:45:31 (1761126331) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.155159 s, 67.6 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2630 0xa46 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 First grouplock blocks second one PASS 107a (7s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 05:45:39 (1761126339) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.154619 s, 67.8 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2651 0xa5b 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7132 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (12s) == sanityn test 108a: lseek: parallel updates ============ 05:45:51 (1761126351) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.00758905 s, 1.3 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 4.9311 s, 1.7 MB/s PASS 108a (11s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 05:46:02 (1761126362) 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Iteration 37 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: detected 4 online CPUs by sysfs oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: Force libcfs to create 2 CPU partitions oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Iteration 38 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: detected 4 online CPUs by sysfs oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 192.168.204.115@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.204.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: Force libcfs to create 2 CPU partitions Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre Starting client: oleg415-client.virtnet: -o user_xattr,flock oleg415-server@tcp:/lustre /mnt/lustre2 PASS 109 (1139s) == sanityn test 110: do not grant another lock on resend ========================================================== 06:05:01 (1761127501) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-22 06:05:02.000000000 -0400 Modify: 2025-10-22 06:05:02.000000000 -0400 Change: 2025-10-22 06:05:02.000000000 -0400 Birth: 2025-10-22 06:05:01.000000000 -0400 fail_loc=0 fail_val=0 Connected clients: oleg415-client.virtnet oleg415-client.virtnet PASS 110 (46s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 06:05:47 (1761127547) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (2s) == sanityn test 112: update max-inherit in default LMV === 06:05:49 (1761127549) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (3s) == sanityn test 113: check servers of specified fs ======= 06:05:52 (1761127552) lustre-OST0000-osc-ffff8fb4c40a0000 active. lustre-OST0001-osc-ffff8fb4c40a0000 active. lustre-OST0000-osc-ffff8fb4c40a0000 active. lustre-OST0001-osc-ffff8fb4c40a0000 active. MGC192.168.204.115@tcp active. PASS 113 (5s) cleanup: ====================================================== == sanityn test complete, duration 6224 sec ============== 06:05:58 (1761127558) Stopping clients: oleg415-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg415-client.virtnet /mnt/lustre2 opts: