-----============= acceptance-small: sanityn ============----- Fri Oct 24 11:09:56 EDT 2025 oleg343-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 Started clients oleg343-client.virtnet: 192.168.203.143@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt) oleg343-client.virtnet: executing check_config_client /mnt/lustre oleg343-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg343-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffffa0ec47e05000.idle_timeout=debug osc.lustre-OST0000-osc-ffffa0ec56019000.idle_timeout=debug osc.lustre-OST0001-osc-ffffa0ec47e05000.idle_timeout=debug osc.lustre-OST0001-osc-ffffa0ec56019000.idle_timeout=debug disable quota as required oleg343-server: oleg343-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137349 s, 76.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f7123] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 11:10:33 (1761318633) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (6s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 11:10:39 (1761318639) -rw-r--r-- 1 root root 0 Oct 24 11:10 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (5s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 11:10:44 (1761318644) -rw-r--r-- 1 root root 0 Oct 24 11:10 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (6s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 11:10:50 (1761318650) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (4s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 11:10:54 (1761318654) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (5s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 11:10:59 (1761318659) total 1 -rwxrwxrwx 1 root root 0 Oct 24 11:10 f2a -rwxrwxrwx 1 root root 0 Oct 24 11:10 f2b total 1 -rwxrwxrwx 1 root root 0 Oct 24 11:10 f2a -rwxrwxrwx 1 root root 0 Oct 24 11:10 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.0223733 s, 22.9 kB/s PASS 2e (6s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 11:11:05 (1761318665) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (3s) == sanityn test 2g: check blocks update on sync write ==== 11:11:08 (1761318668) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.235826 s, 8.9 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (4s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 11:11:12 (1761318672) PASS 3 (5s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 11:11:17 (1761318677) PASS 4 (6s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 11:11:24 (1761318684) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (7s) == sanityn test 6: remove of open file on other node ============================================================================ 11:11:30 (1761318690) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (5s) == sanityn test 7: remove of open directory on other node ======================================================================= 11:11:35 (1761318695) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (5s) == sanityn test 8: remove of open special file on other node ==================================================================== 11:11:40 (1761318700) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (5s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 11:11:45 (1761318705) PASS 9a (6s) == sanityn test 9b: append to striped sparse file ======== 11:11:51 (1761318711) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0410706 s, 25.5 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.0453679 s, 0.1 kB/s Data read (expecting 'foo'): foo PASS 9b (6s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 11:11:57 (1761318717) 1+0 records in 1+0 records out 1 byte copied, 0.0202453 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0168646 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0122109 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0243482 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0159552 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0150411 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.015576 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00608072 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00920446 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00508065 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00536266 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00596176 s, 0.2 kB/s PASS 10a (7s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 11:12:04 (1761318724) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.0227238 s, 135 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0272411 s, 150 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.00024971 s, 12.3 MB/s PASS 10b (5s) == sanityn test 11: execution of file opened for write should return error ============================================================== 11:12:09 (1761318729) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7123 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (5s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 11:12:14 (1761318734) warning: '-runtime' deprecated, use '-t runtime' instead - link 486 (time 1761318746.39 total 10.01 last 48.56) - link 919 (time 1761318756.39 total 20.01 last 43.29) - link 1403 (time 1761318766.40 total 30.02 last 48.36) - link 1862 (time 1761318776.41 total 40.03 last 45.86) - link 2360 (time 1761318786.41 total 50.03 last 49.79) - link 2745 (time 1761318796.42 total 60.03 last 38.49) - link 3167 (time 1761318806.44 total 70.06 last 42.09) - link 3561 (time 1761318816.46 total 80.08 last 39.31) - link 3952 (time 1761318826.48 total 90.10 last 39.04) total: 4375 link in 100.01 seconds: 43.74 ops/second using seed 3085095175 running for 100 seconds - stat 10000 (time 1761318744 ; total 7 ; last 7) - stat 20000 (time 1761318751 ; total 14 ; last 7) - stat 30000 (time 1761318759 ; total 22 ; last 8) - stat 40000 (time 1761318765 ; total 28 ; last 6) - stat 50000 (time 1761318772 ; total 35 ; last 7) - stat 60000 (time 1761318779 ; total 42 ; last 7) - stat 70000 (time 1761318785 ; total 48 ; last 6) - stat 78359 (time 1761318796 ; total 59 ; last 11) - stat 80000 (time 1761318798 ; total 61 ; last 2) - stat 88029 (time 1761318809 ; total 72 ; last 11) - stat 90000 (time 1761318811 ; total 74 ; last 2) - stat 96426 (time 1761318822 ; total 85 ; last 11) - stat 100000 (time 1761318827 ; total 90 ; last 5) total: 106799 stats in 100 seconds: 1067.989990 stats/second - unlinked 0 (time 1761318787 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile4375) error: No such file or directory total: 4374 unlinks in 85 seconds: 51.458824 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (19697) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (19700) - No such process PASS 12 (141s) == sanityn test 13: test directory page revocation ======= 11:14:36 (1761318876) f13 f13 PASS 13 (6s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 11:14:41 (1761318881) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7123 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 20873 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (5s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 11:14:46 (1761318886) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 21447 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (4s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 11:14:50 (1761318890) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 22019 Terminated $DIR1/$tdir/sleep 60 PASS 14b (5s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 11:14:55 (1761318895) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 22595 Terminated $DIR1/$tdir/sleep 60 PASS 14c (5s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 11:15:00 (1761318900) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 23172 Terminated $DIR1/$tdir/sleep 60 PASS 14d (6s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 11:15:06 (1761318906) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7529472 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7529472kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (16s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 11:15:22 (1761318922) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.213205 s, 9.8 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.117312 s, 17.9 MB/s Chance of close/open is 1 in 50 Seed set to 8924 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x7c9648 000004[0] 1761318925.422479 trunc from 00000000 to 0x7c9647 (0x7c9648 bytes) 000005[1] 1761318925.494008 read 0x3e11a9 thru 0x3e2279 (0x10d1 bytes) 000006[0] 1761318925.515565 mapread 0x7b7a46 thru 0x7bcb34 (0x50ef bytes) 000007[0] 1761318925.544171 mapread 0x15cde9 thru 0x1600b3 (0x32cb bytes) 000008[1] 1761318925.570145 read 0x42b52a thru 0x436141 (0xac18 bytes) 000009[1] 1761318925.669733 read 0x230f25 thru 0x23958b (0x8667 bytes) 000010[0] 1761318925.717494 read 0x52267c thru 0x522e36 (0x7bb bytes) 000012[0] 1761318925.725908 mapread 0x36722d thru 0x3706b5 (0x9489 bytes) 000013[1] 1761318925.749621 mapread 0x4f7955 thru 0x4fafca (0x3676 bytes) 000014[1] 1761318925.751308 trunc from 0x7c9648 to 0x3ed560 (0x3dc0e7 bytes) 000016[1] 1761318925.811983 write 0x11a993 thru 0x11b3dd (0xa4b bytes) 000019[0] 1761318925.841555 write 0x7f0a30 thru 0x7f467b (0x3c4c bytes) 000020[1] 1761318925.873944 read 0x298a36 thru 0x2a5be4 (0xd1af bytes) 000022[0] 1761318925.926173 trunc from 0x7f467c to 0x241e9f (0x5b27dc bytes) 000024[0] 1761318925.990004 mapread 0x0ab2c1 thru 0x0b7485 (0xc1c5 bytes) 000025[0] 1761318926.023379 mapread 0x080952 thru 0x089d2a (0x93d9 bytes) 000029[0] 1761318926.079955 trunc from 0x241ea0 to 0x4c7548 (0x2856a9 bytes) 000032[0] 1761318926.152995 mapwrite 0x59e087 thru 0x5a4db8 (0x6d32 bytes) 000033[1] 1761318926.265210 trunc from 0x5a4db9 to 0x0a009a (0x504d1e bytes) 000036[0] 1761318926.317786 mapwrite 0x349dbc thru 0x359967 (0xfbac bytes) 000038[0] 1761318926.410402 read 0x21fa0e thru 0x22b09d (0xb690 bytes) 000039[0] 1761318926.427965 mapwrite 0x0e8845 thru 0x0f467c (0xbe38 bytes) 000041[1] 1761318926.544173 trunc from 0x359968 to 0x7bf8bf (0x465f58 bytes) 000043[1] 1761318926.604330 trunc from 0x7bf8c0 to 0x708ff4 (0xb68cb bytes) 000044[1] 1761318926.622151 mapread 0x206859 thru 0x21138b (0xab33 bytes) 000047[0] 1761318926.778013 mapwrite 0x87de0b thru 0x882543 (0x4739 bytes) 000048[1] 1761318926.915407 write 0x85d82a thru 0x8608f3 (0x30ca bytes) 000049[1] 1761318926.942212 read 0x43cb74 thru 0x43f90b (0x2d98 bytes) 000051[1] 1761318926.980896 write 0x117622 thru 0x1179b9 (0x398 bytes) 000055[1] 1761318927.043341 mapwrite 0x0a5c8f thru 0x0b3d95 (0xe107 bytes) 000056[0] 1761318927.176852 read 0x66fd16 thru 0x67d660 (0xd94b bytes) 000057[0] 1761318927.222484 mapread 0x0b5cf8 thru 0x0c11d4 (0xb4dd bytes) 000062[1] 1761318927.310923 write 0x1ccb8c thru 0x1d115f (0x45d4 bytes) truncating to largest ever: 0x83d725 000064[0] 1761318927.360281 trunc from 0x882544 to 0x83d724 (0x44e1f bytes) 000065[1] 1761318927.400410 mapread 0x15f13e thru 0x15f30f (0x1d2 bytes) 000066[0] 1761318927.425870 mapwrite 0x8d655f thru 0x8d9a8c (0x352e bytes) 000067[1] 1761318927.498405 mapread 0x4e965c thru 0x4f4362 (0xad07 bytes) 000070[1] 1761318927.620003 write 0x580a42 thru 0x588353 (0x7912 bytes) 000072[1] 1761318927.670021 write 0x9f4696 thru 0x9ff028 (0xa993 bytes) 000073[1] 1761318927.673239 write 0x28d3a6 thru 0x292c8a (0x58e5 bytes) 000074[0] 1761318927.701920 mapread 0x769c72 thru 0x77359a (0x9929 bytes) 000075[1] 1761318927.792633 trunc from 0x9ff029 to 0x24c30d (0x7b2d1b bytes) 000078[0] 1761318927.861825 mapread 0x067784 thru 0x06d052 (0x58cf bytes) 000079[1] 1761318927.942655 trunc from 0x24c30e to 0x32e8ac (0xe259f bytes) 000081[1] 1761318927.972338 read 0x0034c4 thru 0x004567 (0x10a4 bytes) truncating to largest ever: 0x95a72e 000084[0] 1761318928.068865 trunc from 0x32e8ad to 0x95a72d (0x62be81 bytes) 000086[1] 1761318928.106702 trunc from 0x95a72e to 0x728b66 (0x231bc7 bytes) 000087[1] 1761318928.164673 mapwrite 0x29bde1 thru 0x2a56e0 (0x9900 bytes) 000088[0] 1761318928.287856 trunc from 0x728b67 to 0x2eadd5 (0x43dd91 bytes) 000089[1] 1761318928.348074 mapwrite 0x1ac58f thru 0x1b271f (0x6191 bytes) 000090[1] 1761318928.485279 read 0x002bfd thru 0x0051f1 (0x25f5 bytes) 000093[0] 1761318928.573678 mapwrite 0x9254a2 thru 0x93319c (0xdcfb bytes) 000094[1] 1761318928.704936 mapwrite 0x1ae289 thru 0x1b13fa (0x3172 bytes) 000098[1] 1761318928.861805 mapread 0x46b26f thru 0x4792a9 (0xe03b bytes) 000099[0] 1761318928.970015 mapread 0x62ca1e thru 0x62e5fa (0x1bdd bytes) 000100[0] 1761318929.024994 read 0x8dd958 thru 0x8e5394 (0x7a3d bytes) 000102[0] 1761318929.069840 write 0x8663c6 thru 0x870cd4 (0xa90f bytes) 000103[1] 1761318929.122066 mapread 0x2e3fb6 thru 0x2e6079 (0x20c4 bytes) 000104[1] 1761318929.185632 read 0x5aaf4a thru 0x5b6aca (0xbb81 bytes) 000105[0] 1761318929.229917 mapread 0x3ece7c thru 0x3f30f1 (0x6276 bytes) 000107[1] 1761318929.295952 mapwrite 0x9bd130 thru 0x9c8707 (0xb5d8 bytes) 000108[1] 1761318929.469345 mapwrite 0x71e32d thru 0x724d09 (0x69dd bytes) 000109[0] 1761318929.598298 mapwrite 0x57ca3d thru 0x5873ac (0xa970 bytes) 000110[0] 1761318929.771287 trunc from 0x9c8708 to 0x58dc54 (0x43aab3 bytes) 000111[1] 1761318929.828948 read 0x1fc865 thru 0x1fc8d1 (0x6d bytes) 000114[0] 1761318929.881140 write 0x2db732 thru 0x2de267 (0x2b36 bytes) 000115[1] 1761318929.892702 mapread 0x12f146 thru 0x13aab0 (0xb96b bytes) 000118[0] 1761318929.970611 write 0x39a699 thru 0x39aeaa (0x812 bytes) 000119[1] 1761318930.005396 write 0x1c2577 thru 0x1cfc67 (0xd6f1 bytes) 000120[1] 1761318930.036253 read 0x568a10 thru 0x56d370 (0x4961 bytes) 000121[0] 1761318930.055998 mapread 0x4d0b35 thru 0x4df420 (0xe8ec bytes) 000122[0] 1761318930.153702 mapread 0x0826ed thru 0x0864f1 (0x3e05 bytes) 000123[0] 1761318930.205942 write 0x20f609 thru 0x210bfc (0x15f4 bytes) 000125[1] 1761318930.267068 read 0x278767 thru 0x27d421 (0x4cbb bytes) 000126[1] 1761318930.387702 mapread 0x221894 thru 0x22977f (0x7eec bytes) 000128[1] 1761318930.449614 write 0x25cb35 thru 0x25d7ff (0xccb bytes) 000130[1] 1761318930.483277 read 0x220afe thru 0x22b971 (0xae74 bytes) 000131[0] 1761318930.518952 read 0x3cab83 thru 0x3d7828 (0xcca6 bytes) 000134[1] 1761318930.582996 trunc from 0x58dc55 to 0x5b9698 (0x2ba44 bytes) 000135[0] 1761318930.618911 mapread 0x372db1 thru 0x3755a6 (0x27f6 bytes) 000136[1] 1761318930.671650 read 0x210a79 thru 0x214565 (0x3aed bytes) 000137[1] 1761318930.692046 mapwrite 0x3e08c4 thru 0x3e65ca (0x5d07 bytes) 000138[1] 1761318930.783239 mapread 0x4a4426 thru 0x4a9316 (0x4ef1 bytes) 000141[1] 1761318930.824201 mapread 0x29ecbb thru 0x2a2a7e (0x3dc4 bytes) 000143[0] 1761318930.842584 mapwrite 0x120a3c thru 0x121589 (0xb4e bytes) 000145[0] 1761318930.923048 trunc from 0x5b9699 to 0x847fac (0x28e914 bytes) 000146[1] 1761318930.967846 mapwrite 0x2f4c38 thru 0x2f652d (0x18f6 bytes) 000147[0] 1761318931.060983 mapwrite 0x6da746 thru 0x6e0be5 (0x64a0 bytes) 000150[0] 1761318931.263727 mapread 0x643003 thru 0x650d3b (0xdd39 bytes) 000151[0] 1761318931.264487 mapread 0x162cbf thru 0x17022b (0xd56d bytes) 000152[1] 1761318931.317729 trunc from 0x847fad to 0x71dcd1 (0x12a2db bytes) 000153[1] 1761318931.388296 mapread 0x4a3ff5 thru 0x4a43c4 (0x3d0 bytes) 000155[1] 1761318931.469665 write 0x2b6304 thru 0x2c1130 (0xae2d bytes) 000156[0] 1761318931.501013 trunc from 0x71dcd2 to 0x6f5063 (0x28c6e bytes) 000157[0] 1761318931.573160 mapread 0x0bfa3d thru 0x0cee2e (0xf3f2 bytes) 000159[1] 1761318931.685789 write 0x91d05a thru 0x92bc18 (0xebbf bytes) 000163[1] 1761318931.759436 read 0x789e3e thru 0x79958c (0xf74f bytes) 000164[0] 1761318931.792426 mapwrite 0x747860 thru 0x751d77 (0xa518 bytes) 000167[0] 1761318931.935466 mapwrite 0x02efe1 thru 0x03844d (0x946d bytes) 000168[1] 1761318932.043179 mapwrite 0x1c1f2d thru 0x1cb3d6 (0x94aa bytes) 000169[1] 1761318932.219150 read 0x6a526d thru 0x6a854f (0x32e3 bytes) 000171[1] 1761318932.277901 mapwrite 0x904611 thru 0x911903 (0xd2f3 bytes) 000173[0] 1761318932.342490 read 0x46cb82 thru 0x47c59d (0xfa1c bytes) 000177[1] 1761318932.398761 mapread 0x4a7361 thru 0x4aec8a (0x792a bytes) 000178[0] 1761318932.464597 mapread 0x61829a thru 0x61adf1 (0x2b58 bytes) 000179[1] 1761318932.498289 trunc from 0x92bc19 to 0x6ec551 (0x23f6c7 bytes) 000180[1] 1761318932.528032 mapwrite 0x7d1f35 thru 0x7d95da (0x76a6 bytes) 000181[0] 1761318932.640086 trunc from 0x7d95db to 0x7c58b4 (0x13d26 bytes) 000183[1] 1761318932.695774 mapwrite 0x896d76 thru 0x8a273d (0xb9c8 bytes) 000184[0] 1761318932.875134 read 0x59786a thru 0x5a4fb1 (0xd748 bytes) 000185[1] 1761318932.959911 read 0x2d6e71 thru 0x2e0bc5 (0x9d55 bytes) 000186[1] 1761318932.986951 mapread 0x883968 thru 0x88e9c8 (0xb061 bytes) 000190[0] 1761318933.121303 write 0x1f0a76 thru 0x1f88f6 (0x7e81 bytes) 000191[0] 1761318933.164639 mapwrite 0x71f7bd thru 0x72173f (0x1f83 bytes) 000193[0] 1761318933.251438 write 0x4f9ef6 thru 0x50188b (0x7996 bytes) 000197[0] 1761318933.387027 mapwrite 0x42d629 thru 0x432e37 (0x580f bytes) 000201[0] 1761318933.513370 write 0x4eb767 thru 0x4f6146 (0xa9e0 bytes) 000202[0] 1761318933.520645 mapwrite 0x81e203 thru 0x820101 (0x1eff bytes) 000203[0] 1761318933.583895 mapwrite 0x4e3b68 thru 0x4efd62 (0xc1fb bytes) 000210[1] 1761318933.772896 write 0x1db56d thru 0x1eae16 (0xf8aa bytes) 000213[1] 1761318933.829535 read 0x408fa7 thru 0x413924 (0xa97e bytes) 000217[1] 1761318933.934581 trunc from 0x8a273e to 0x4fc277 (0x3a64c6 bytes) 000218[1] 1761318933.980734 read 0x4b6ee0 thru 0x4bc0ad (0x51ce bytes) 000219[1] 1761318934.033666 mapread 0x31ea08 thru 0x32df1c (0xf515 bytes) 000221[0] 1761318934.130300 mapwrite 0x708d28 thru 0x710aff (0x7dd8 bytes) 000222[1] 1761318934.296575 read 0x1c7f98 thru 0x1d4e93 (0xcefc bytes) 000223[1] 1761318934.331423 write 0x42d2cf thru 0x4324da (0x520c bytes) 000224[0] 1761318934.424298 mapwrite 0x628138 thru 0x636158 (0xe021 bytes) 000226[0] 1761318934.629694 trunc from 0x710b00 to 0x5e59fd (0x12b102 bytes) 000227[1] 1761318934.707217 mapwrite 0x62313e thru 0x630662 (0xd525 bytes) 000228[0] 1761318934.893011 trunc from 0x630663 to 0x0f2211 (0x53e451 bytes) 000229[1] 1761318934.941630 read 0x04dce5 thru 0x05a026 (0xc342 bytes) 000230[0] 1761318935.002600 mapread 0x06bf36 thru 0x07ae89 (0xef54 bytes) 000231[1] 1761318935.094603 mapwrite 0x0a72d4 thru 0x0b186f (0xa59c bytes) 000232[0] 1761318935.236640 write 0x9a3b5e thru 0x9a60c3 (0x2566 bytes) 000233[0] 1761318935.251364 trunc from 0x9a60c4 to 0x7f7455 (0x1aec6e bytes) 000236[1] 1761318935.303912 trunc from 0x7f7456 to 0x8138c7 (0x1c472 bytes) 000238[1] 1761318935.380237 read 0x3a6723 thru 0x3a9253 (0x2b31 bytes) 000239[1] 1761318935.447053 mapwrite 0x7b1002 thru 0x7b34c0 (0x24bf bytes) 000242[1] 1761318935.519708 write 0x5c786e thru 0x5c78cf (0x62 bytes) 000245[1] 1761318935.545317 trunc from 0x8138c8 to 0x7aad72 (0x68b55 bytes) 000248[0] 1761318935.620283 write 0x7dee1e thru 0x7e89f4 (0x9bd7 bytes) 000250[0] 1761318935.664045 trunc from 0x7e89f5 to 0x324f78 (0x4c3a7c bytes) 000251[0] 1761318935.704639 write 0x7db121 thru 0x7e9ebc (0xed9c bytes) 000252[1] 1761318935.707094 read 0x25ffbd thru 0x260f39 (0xf7d bytes) 000253[0] 1761318935.739448 mapread 0x113ca8 thru 0x11c577 (0x88d0 bytes) 000254[1] 1761318935.771413 mapwrite 0x1611fb thru 0x1677f0 (0x65f6 bytes) 000255[0] 1761318935.962017 read 0x32b56f thru 0x33036c (0x4dfe bytes) 000256[1] 1761318936.035314 read 0x755c4d thru 0x75c065 (0x6419 bytes) 000258[0] 1761318936.051943 write 0x170318 thru 0x17e783 (0xe46c bytes) 000259[0] 1761318936.100708 write 0x55c193 thru 0x562021 (0x5e8f bytes) 000261[1] 1761318936.115664 read 0x086a5c thru 0x090559 (0x9afe bytes) 000262[1] 1761318936.134544 trunc from 0x7e9ebd to 0x45d695 (0x38c827 bytes) 000263[0] 1761318936.234174 mapwrite 0x66f89f thru 0x673c7b (0x43dd bytes) 000264[1] 1761318936.333565 mapwrite 0x1eee94 thru 0x1fe415 (0xf582 bytes) 000265[1] 1761318936.430996 read 0x1d7d11 thru 0x1e5560 (0xd850 bytes) 000266[0] 1761318936.455001 mapread 0x21419e thru 0x215f1c (0x1d7f bytes) 000267[0] 1761318936.500382 read 0x4d7945 thru 0x4d7b25 (0x1e1 bytes) 000272[1] 1761318936.597899 mapwrite 0x688223 thru 0x698061 (0xfe3f bytes) 000279[0] 1761318936.833414 trunc from 0x698062 to 0x6d9b26 (0x41ac5 bytes) 000282[1] 1761318936.930212 mapread 0x320dad thru 0x32da35 (0xcc89 bytes) 000284[0] 1761318937.034413 read 0x5eec32 thru 0x5fa5a4 (0xb973 bytes) 000286[1] 1761318937.053943 mapread 0x2068df thru 0x20aba5 (0x42c7 bytes) 000287[0] 1761318937.131413 mapwrite 0x5ef065 thru 0x5f9997 (0xa933 bytes) 000288[0] 1761318937.282453 write 0x5b4854 thru 0x5c1f17 (0xd6c4 bytes) 000290[0] 1761318937.335618 read 0x5fa512 thru 0x5fdc9e (0x378d bytes) 000291[1] 1761318937.355104 read 0x1f7ddb thru 0x1f9a1d (0x1c43 bytes) 000293[0] 1761318937.429939 trunc from 0x6d9b27 to 0x639a00 (0xa0126 bytes) 000294[0] 1761318937.502631 trunc from 0x639a01 to 0x1ad962 (0x48c09e bytes) 000295[0] 1761318937.525598 read 0x01bdd1 thru 0x028451 (0xc681 bytes) 000297[1] 1761318937.576033 read 0x1226e8 thru 0x125487 (0x2da0 bytes) 000298[1] 1761318937.602083 mapwrite 0x16a8ef thru 0x17922a (0xe93c bytes) 000304[0] 1761318937.847839 trunc from 0x1ad963 to 0x30b809 (0x15dea7 bytes) 000307[1] 1761318937.919977 read 0x25235a thru 0x256ffe (0x4ca5 bytes) 000309[0] 1761318937.984882 trunc from 0x30b80a to 0x4d524b (0x1c9a42 bytes) 000310[1] 1761318938.026233 mapwrite 0x43f5ff thru 0x447131 (0x7b33 bytes) 000312[1] 1761318938.154180 read 0x40d35e thru 0x412809 (0x54ac bytes) 000313[0] 1761318938.180552 trunc from 0x4d524c to 0x350051 (0x1851fa bytes) 000314[0] 1761318938.204986 write 0x4440ff thru 0x4486cc (0x45ce bytes) 000315[0] 1761318938.206068 read 0x1c23c3 thru 0x1ca71d (0x835b bytes) 000319[0] 1761318938.307676 read 0x2a7300 thru 0x2ab17c (0x3e7d bytes) 000320[0] 1761318938.329967 mapread 0x19fe1f thru 0x1ad966 (0xdb48 bytes) 000321[0] 1761318938.403972 write 0x17b45a thru 0x1889a6 (0xd54d bytes) 000323[1] 1761318938.432997 trunc from 0x4486cd to 0x566d7c (0x11e6b0 bytes) 000326[0] 1761318938.547535 mapwrite 0x170a92 thru 0x173d2b (0x329a bytes) 000327[0] 1761318938.620729 write 0x7e4bbf thru 0x7ea2e6 (0x5728 bytes) 000332[0] 1761318938.712307 mapread 0x37557e thru 0x3763c8 (0xe4b bytes) 000333[0] 1761318938.740954 trunc from 0x7ea2e7 to 0x29ae5f (0x54f487 bytes) 000334[1] 1761318938.802650 read 0x1133de thru 0x11ff3c (0xcb5f bytes) 000335[1] 1761318938.842628 mapwrite 0x185329 thru 0x1878bc (0x2594 bytes) 000336[0] 1761318938.958021 write 0x7fdb3e thru 0x801b5c (0x401f bytes) 000337[0] 1761318938.998579 trunc from 0x801b5d to 0x65818d (0x1a99cf bytes) 000339[1] 1761318939.049415 write 0x450ec9 thru 0x45bfe9 (0xb121 bytes) 000343[0] 1761318939.131022 read 0x139f6b thru 0x13c569 (0x25ff bytes) truncating to largest ever: 0x9c8cf5 000344[1] 1761318939.187979 trunc from 0x65818e to 0x9c8cf4 (0x370b67 bytes) 000350[1] 1761318939.305483 write 0x262787 thru 0x26a106 (0x7980 bytes) 000351[0] 1761318939.324396 write 0x73a58f thru 0x73b95b (0x13cd bytes) 000353[1] 1761318939.398873 trunc from 0x9c8cf5 to 0x67eb8b (0x34a169 bytes) 000354[0] 1761318939.439259 write 0x3e3409 thru 0x3f1e81 (0xea79 bytes) 000355[0] 1761318939.488357 mapread 0x51a680 thru 0x527b4b (0xd4cc bytes) 000357[1] 1761318939.612390 read 0x2d420c thru 0x2dcec8 (0x8cbd bytes) 000359[1] 1761318939.649740 trunc from 0x67eb8c to 0x4505b4 (0x22e5d7 bytes) 000361[0] 1761318939.718018 mapread 0x1bd8a3 thru 0x1cd4d3 (0xfc31 bytes) 000362[0] 1761318939.860265 mapread 0x05db22 thru 0x06ace1 (0xd1c0 bytes) 000363[0] 1761318940.035076 write 0x57fdb1 thru 0x583eca (0x411a bytes) 000364[1] 1761318940.072389 read 0x128de1 thru 0x12cbb9 (0x3dd9 bytes) 000365[0] 1761318940.177886 write 0x02a77a thru 0x02fc92 (0x5519 bytes) 000366[1] 1761318940.252386 write 0x995977 thru 0x995b38 (0x1c2 bytes) 000367[0] 1761318940.295004 mapwrite 0x8a2a9a thru 0x8a4901 (0x1e68 bytes) 000368[1] 1761318940.441567 mapwrite 0x1ef585 thru 0x1fded6 (0xe952 bytes) 000369[0] 1761318940.601206 mapwrite 0x55dc7a thru 0x563de1 (0x6168 bytes) 000373[1] 1761318940.802369 mapread 0x2f6558 thru 0x2fa404 (0x3ead bytes) 000375[0] 1761318940.891317 mapread 0x75c05a thru 0x764db4 (0x8d5b bytes) 000376[1] 1761318940.892570 trunc from 0x995b39 to 0x386f01 (0x60ec37 bytes) 000380[1] 1761318941.025744 write 0x6984f5 thru 0x6a1601 (0x910d bytes) 000382[1] 1761318941.037025 mapread 0x480100 thru 0x482f30 (0x2e31 bytes) 000383[1] 1761318941.058725 mapwrite 0x6a4eaa thru 0x6ad652 (0x87a9 bytes) 000384[1] 1761318941.210097 mapread 0x5603d6 thru 0x5674a8 (0x70d3 bytes) 000385[0] 1761318941.291665 mapwrite 0x218463 thru 0x21c1a3 (0x3d41 bytes) 000386[0] 1761318941.423150 trunc from 0x6ad653 to 0x80018e (0x152b3c bytes) 000387[0] 1761318941.487352 write 0x250483 thru 0x253e5c (0x39da bytes) 000388[0] 1761318941.534355 write 0x1d3996 thru 0x1d6588 (0x2bf3 bytes) 000389[1] 1761318941.570635 trunc from 0x80018f to 0x852fe3 (0x52e55 bytes) 000394[0] 1761318941.684401 write 0x49ac65 thru 0x4a77ce (0xcb6a bytes) 000396[1] 1761318941.766039 mapread 0x683e41 thru 0x68d1b9 (0x9379 bytes) 000401[0] 1761318941.954237 mapwrite 0x7d782f thru 0x7da4be (0x2c90 bytes) 000403[1] 1761318942.084688 mapwrite 0x55d690 thru 0x55f796 (0x2107 bytes) 000404[0] 1761318942.203112 write 0x9b155c thru 0x9bad58 (0x97fd bytes) 000405[1] 1761318942.218216 mapread 0x5f6125 thru 0x5f9443 (0x331f bytes) 000406[1] 1761318942.310141 read 0x2a0c20 thru 0x2a0ca0 (0x81 bytes) 000407[1] 1761318942.357256 mapread 0x470a24 thru 0x479ed2 (0x94af bytes) 000409[0] 1761318942.358381 write 0x6e7fc5 thru 0x6f4c41 (0xcc7d bytes) 000411[1] 1761318942.416054 mapread 0x36bd86 thru 0x36ce23 (0x109e bytes) 000414[1] 1761318942.447036 mapread 0x59a3c0 thru 0x59f93e (0x557f bytes) 000415[0] 1761318942.466402 write 0x06bf84 thru 0x06ebe4 (0x2c61 bytes) 000419[1] 1761318942.515033 write 0x6ffac1 thru 0x7016b0 (0x1bf0 bytes) 000420[0] 1761318942.573635 write 0x1fd91f thru 0x20c7bf (0xeea1 bytes) 000422[0] 1761318942.620825 write 0x2ca636 thru 0x2cfe66 (0x5831 bytes) 000423[0] 1761318942.652037 read 0x027f0e thru 0x035fd5 (0xe0c8 bytes) 000424[0] 1761318942.723832 mapread 0x1c38b9 thru 0x1c85ef (0x4d37 bytes) 000425[1] 1761318942.724459 write 0x60c328 thru 0x616256 (0x9f2f bytes) 000426[0] 1761318942.781398 write 0x58b482 thru 0x597e72 (0xc9f1 bytes) 000427[1] 1761318942.798256 mapwrite 0x0d7d68 thru 0x0ddb14 (0x5dad bytes) 000428[1] 1761318942.922589 mapread 0x7f12ae thru 0x800ee2 (0xfc35 bytes) 000429[0] 1761318943.051363 read 0x475203 thru 0x47cac8 (0x78c6 bytes) 000431[1] 1761318943.129345 mapwrite 0x8ca24b thru 0x8d07a6 (0x655c bytes) 000436[1] 1761318943.236828 mapread 0x7b4a85 thru 0x7bf41e (0xa99a bytes) 000437[0] 1761318943.307843 read 0x50dbf8 thru 0x50eee5 (0x12ee bytes) 000439[0] 1761318943.343092 trunc from 0x9bad59 to 0x636f47 (0x383e11 bytes) 000440[0] 1761318943.398843 read 0x35efea thru 0x365be7 (0x6bfe bytes) 000441[0] 1761318943.443940 read 0x1d6ca7 thru 0x1dd69a (0x69f4 bytes) 000442[0] 1761318943.464750 read 0x3ff3a5 thru 0x3ff647 (0x2a3 bytes) 000444[0] 1761318943.508911 trunc from 0x636f48 to 0x60678a (0x307bd bytes) 000445[1] 1761318943.544255 trunc from 0x60678b to 0x02ecef (0x5d7a9b bytes) 000446[0] 1761318943.611043 mapwrite 0x99fbd0 thru 0x9a6b1d (0x6f4e bytes) 000447[0] 1761318943.813756 mapread 0x13fcc0 thru 0x147c28 (0x7f69 bytes) 000449[1] 1761318943.885714 mapwrite 0x98fd0a thru 0x99efd4 (0xf2cb bytes) 000451[0] 1761318944.073164 write 0x36d9b9 thru 0x378656 (0xac9e bytes) 000452[1] 1761318944.093060 trunc from 0x9a6b1e to 0x8b7943 (0xef1da bytes) 000454[1] 1761318944.147955 read 0x136376 thru 0x136f21 (0xbac bytes) 000457[0] 1761318944.322138 mapwrite 0x749920 thru 0x7510d6 (0x77b7 bytes) 000458[1] 1761318944.475962 mapread 0x7297dd thru 0x736438 (0xcc5c bytes) 000460[0] 1761318944.571096 mapwrite 0x172ec6 thru 0x17d3d0 (0xa50b bytes) 000461[1] 1761318944.695604 mapread 0x4ffca9 thru 0x509ac6 (0x9e1e bytes) 000462[0] 1761318944.768417 trunc from 0x8b7944 to 0x6e2cb4 (0x1d4c8f bytes) 000463[0] 1761318944.833287 mapwrite 0x882d98 thru 0x887071 (0x42da bytes) 000465[0] 1761318944.967223 trunc from 0x887072 to 0x26c1a1 (0x61aed0 bytes) 000467[1] 1761318945.010038 mapwrite 0x3a7346 thru 0x3abb11 (0x47cc bytes) 000472[1] 1761318945.178073 read 0x15ffe0 thru 0x1650ce (0x50ef bytes) 000476[0] 1761318945.285242 read 0x354463 thru 0x35a38b (0x5f29 bytes) 000478[1] 1761318945.338553 mapwrite 0x243e86 thru 0x24eed0 (0xb04b bytes) 000480[0] 1761318945.423021 read 0x37646a thru 0x384936 (0xe4cd bytes) 000481[1] 1761318945.424372 mapwrite 0x22fbf8 thru 0x23cf4d (0xd356 bytes) 000482[1] 1761318945.530324 mapread 0x0c96f4 thru 0x0ca910 (0x121d bytes) 000483[0] 1761318945.543163 trunc from 0x3abb12 to 0x542b0b (0x196ffa bytes) 000486[0] 1761318945.613794 read 0x2fe882 thru 0x308cb8 (0xa437 bytes) 000487[1] 1761318945.684643 mapwrite 0x30f628 thru 0x3139a7 (0x4380 bytes) 000488[0] 1761318945.782665 trunc from 0x542b0c to 0x1dca0f (0x3660fc bytes) 000491[1] 1761318945.834844 mapread 0x0b8361 thru 0x0bb9fc (0x369c bytes) 000493[0] 1761318945.881198 read 0x18ec7d thru 0x193730 (0x4ab4 bytes) 000499[0] 1761318945.910400 read 0x1bbd2f thru 0x1c40f4 (0x83c6 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 8945 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[1] 1761318946.453024 mapwrite 0x467000 thru 0x46928c (0x228d bytes) 000002[1] 1761318946.627184 write 0x8fb000 thru 0x9048af (0x98b0 bytes) 000004[1] 1761318946.663820 mapread 0x265000 thru 0x2705b7 (0xb5b8 bytes) 000005[1] 1761318946.705852 read 0x75e000 thru 0x76db75 (0xfb76 bytes) 000006[1] 1761318946.714580 mapread 0x710000 thru 0x7144c6 (0x44c7 bytes) 000008[1] 1761318946.764059 read 0x329000 thru 0x3334aa (0xa4ab bytes) 000010[0] 1761318946.852339 write 0x4fb000 thru 0x5067fe (0xb7ff bytes) 000012[0] 1761318946.927723 write 0x7e2000 thru 0x7ef951 (0xd952 bytes) 000013[1] 1761318946.966021 mapwrite 0x5c1000 thru 0x5c250a (0x150b bytes) truncating to largest ever: 0x3171b4 000014[0] 1761318947.073655 trunc from 0x9048b0 to 0x3171b3 (0x5ed6fc bytes) 000016[0] 1761318947.148902 write 0x047000 thru 0x04a79c (0x379d bytes) 000017[0] 1761318947.170201 mapread 0x0bb000 thru 0x0bd583 (0x2584 bytes) 000020[0] 1761318947.241203 mapread 0x094000 thru 0x09da63 (0x9a64 bytes) 000021[1] 1761318947.286791 mapwrite 0x6c3000 thru 0x6c9896 (0x6897 bytes) 000022[1] 1761318947.405939 read 0x39c000 thru 0x3a55f2 (0x95f3 bytes) 000025[1] 1761318947.491674 write 0x338000 thru 0x33edb9 (0x6dba bytes) 000028[1] 1761318947.526852 write 0x9bf000 thru 0x9c0541 (0x1542 bytes) 000031[1] 1761318947.539673 mapread 0x72b000 thru 0x72d41a (0x241b bytes) 000032[0] 1761318947.585976 read 0x2b3000 thru 0x2b5073 (0x2074 bytes) truncating to largest ever: 0x8ec58c 000033[1] 1761318947.623712 trunc from 0x9c0542 to 0x8ec58b (0xd3fb6 bytes) 000034[0] 1761318947.652654 trunc from 0x8ec58c to 0x7dfce0 (0x10c8ab bytes) 000036[0] 1761318947.690467 read 0x741000 thru 0x74ef24 (0xdf25 bytes) 000037[0] 1761318947.713394 read 0x30a000 thru 0x315606 (0xb607 bytes) 000039[0] 1761318947.762050 read 0x0ee000 thru 0x0f4e25 (0x6e26 bytes) 000041[1] 1761318947.809692 mapread 0x678000 thru 0x67a78c (0x278d bytes) 000043[1] 1761318947.860912 mapwrite 0x040000 thru 0x0471bc (0x71bd bytes) 000044[1] 1761318947.961205 mapwrite 0x78d000 thru 0x792277 (0x5278 bytes) 000045[0] 1761318948.024649 read 0x056000 thru 0x061810 (0xb811 bytes) 000046[1] 1761318948.079884 read 0x58e000 thru 0x58f346 (0x1347 bytes) 000047[0] 1761318948.144679 read 0x166000 thru 0x170770 (0xa771 bytes) 000052[1] 1761318948.145867 write 0x66f000 thru 0x67c97a (0xd97b bytes) 000053[1] 1761318948.179945 write 0x747000 thru 0x74f294 (0x8295 bytes) 000054[0] 1761318948.189695 read 0x418000 thru 0x41926b (0x126c bytes) 000055[0] 1761318948.272357 write 0x983000 thru 0x991876 (0xe877 bytes) 000057[0] 1761318948.296484 mapwrite 0x958000 thru 0x958c53 (0xc54 bytes) 000059[0] 1761318948.439171 write 0x05e000 thru 0x062bc2 (0x4bc3 bytes) 000061[1] 1761318948.491857 trunc from 0x991877 to 0x40524f (0x58c627 bytes) 000062[1] 1761318948.530747 write 0x573000 thru 0x580f74 (0xdf75 bytes) 000063[0] 1761318948.532211 mapwrite 0x76f000 thru 0x779d20 (0xad21 bytes) 000066[0] 1761318948.734313 mapread 0x458000 thru 0x463f9f (0xbfa0 bytes) 000067[0] 1761318948.829858 mapread 0x366000 thru 0x3697bd (0x37be bytes) 000068[0] 1761318948.847529 mapwrite 0x9d0000 thru 0x9d6a60 (0x6a61 bytes) 000069[1] 1761318948.980229 write 0x3e8000 thru 0x3ee51f (0x6520 bytes) 000073[0] 1761318949.062186 read 0x103000 thru 0x107ae1 (0x4ae2 bytes) 000075[0] 1761318949.178409 trunc from 0x9d6a61 to 0x69a692 (0x33c3ce bytes) 000077[0] 1761318949.289037 write 0x919000 thru 0x919a2b (0xa2c bytes) 000079[0] 1761318949.312448 mapwrite 0x17c000 thru 0x18141d (0x541e bytes) 000081[1] 1761318949.400251 mapwrite 0x3c1000 thru 0x3c7024 (0x6025 bytes) 000083[1] 1761318949.504791 mapwrite 0x26e000 thru 0x279c9f (0xbca0 bytes) 000084[0] 1761318949.611268 trunc from 0x919a2c to 0x7e6b2e (0x132efd bytes) 000085[0] 1761318949.647664 mapwrite 0x6f0000 thru 0x6f3f44 (0x3f45 bytes) 000087[0] 1761318949.705430 trunc from 0x7e6b2f to 0x5beab6 (0x228078 bytes) truncating to largest ever: 0x99bf43 000089[0] 1761318949.741372 trunc from 0x5beab7 to 0x99bf42 (0x3dd48c bytes) 000090[1] 1761318949.777381 trunc from 0x99bf43 to 0x95112c (0x4ae16 bytes) 000092[1] 1761318949.818849 write 0x26a000 thru 0x278278 (0xe279 bytes) 000093[1] 1761318949.851066 write 0x5ad000 thru 0x5b7244 (0xa245 bytes) 000097[1] 1761318949.895946 mapwrite 0x91f000 thru 0x92daad (0xeaae bytes) 000098[1] 1761318950.017578 mapread 0x67c000 thru 0x6868b8 (0xa8b9 bytes) 000100[1] 1761318950.124773 mapwrite 0x5c7000 thru 0x5c9455 (0x2456 bytes) 000101[0] 1761318950.219430 trunc from 0x95112d to 0x03520c (0x91bf20 bytes) 000102[1] 1761318950.266451 mapwrite 0x571000 thru 0x57f1d0 (0xe1d1 bytes) 000103[1] 1761318950.430142 trunc from 0x57f1d1 to 0x634aca (0xb58fa bytes) 000104[1] 1761318950.462298 read 0x3d9000 thru 0x3e616a (0xd16b bytes) 000105[0] 1761318950.505296 write 0x7b0000 thru 0x7b1397 (0x1398 bytes) 000106[0] 1761318950.536316 read 0x46a000 thru 0x477011 (0xd012 bytes) 000108[1] 1761318950.622745 mapwrite 0x1a5000 thru 0x1a743b (0x243c bytes) 000109[0] 1761318950.772971 write 0x07c000 thru 0x07d7e2 (0x17e3 bytes) 000112[0] 1761318950.819295 mapwrite 0x973000 thru 0x973b0e (0xb0f bytes) 000118[1] 1761318950.944406 write 0x51f000 thru 0x52b84d (0xc84e bytes) 000120[0] 1761318951.042145 write 0x043000 thru 0x04969f (0x66a0 bytes) 000121[1] 1761318951.105029 mapwrite 0x74b000 thru 0x75aedb (0xfedc bytes) 000123[0] 1761318951.313820 mapread 0x621000 thru 0x62f4a3 (0xe4a4 bytes) 000125[0] 1761318951.409329 mapwrite 0x71f000 thru 0x729f7e (0xaf7f bytes) 000126[1] 1761318951.504408 mapwrite 0x899000 thru 0x8a49fa (0xb9fb bytes) 000127[1] 1761318951.636340 mapread 0x077000 thru 0x084461 (0xd462 bytes) 000128[1] 1761318951.735078 trunc from 0x973b0f to 0x3f5610 (0x57e4fe bytes) 000130[1] 1761318951.769837 mapwrite 0x728000 thru 0x731adf (0x9ae0 bytes) 000135[1] 1761318951.926563 write 0x4e1000 thru 0x4ec775 (0xb776 bytes) 000140[0] 1761318951.981661 mapread 0x100000 thru 0x10025a (0x25b bytes) 000144[1] 1761318952.018105 write 0x941000 thru 0x94aafc (0x9afd bytes) 000145[1] 1761318952.045637 write 0x5ef000 thru 0x5ef87d (0x87e bytes) 000147[0] 1761318952.089138 read 0x4c8000 thru 0x4d515e (0xd15f bytes) 000149[0] 1761318952.163220 write 0x9b0000 thru 0x9b17ed (0x17ee bytes) 000150[1] 1761318952.212046 mapread 0x676000 thru 0x6843a4 (0xe3a5 bytes) 000151[1] 1761318952.333806 mapwrite 0x64d000 thru 0x64e331 (0x1332 bytes) 000156[0] 1761318952.421018 mapwrite 0x1d2000 thru 0x1e1898 (0xf899 bytes) 000161[1] 1761318952.627779 mapread 0x82d000 thru 0x83c8ac (0xf8ad bytes) 000164[1] 1761318952.734017 mapwrite 0x12d000 thru 0x136fae (0x9faf bytes) 000166[0] 1761318952.869010 write 0x9ea000 thru 0x9f009c (0x609d bytes) 000167[1] 1761318952.888070 mapwrite 0x87f000 thru 0x88a94a (0xb94b bytes) 000169[1] 1761318953.054161 mapread 0x550000 thru 0x55a392 (0xa393 bytes) 000174[1] 1761318953.162564 mapwrite 0x47c000 thru 0x47e908 (0x2909 bytes) 000176[1] 1761318953.233662 trunc from 0x9f009d to 0x8bcf26 (0x133176 bytes) 000178[0] 1761318953.283380 read 0x420000 thru 0x42cefd (0xcefe bytes) 000180[1] 1761318953.357259 mapwrite 0x504000 thru 0x50e493 (0xa494 bytes) 000181[1] 1761318953.487839 trunc from 0x8bcf27 to 0x7c0fa9 (0xfbf7d bytes) 000184[1] 1761318953.534067 trunc from 0x7c0faa to 0x64b0b7 (0x175ef2 bytes) 000185[1] 1761318953.576123 mapread 0x587000 thru 0x58f373 (0x8374 bytes) truncating to largest ever: 0x9dd236 000188[1] 1761318953.661540 trunc from 0x64b0b8 to 0x9dd235 (0x39217e bytes) 000189[0] 1761318953.696943 read 0x488000 thru 0x48cdb6 (0x4db7 bytes) 000190[0] 1761318953.759625 mapread 0x69a000 thru 0x6a7d92 (0xdd93 bytes) 000192[0] 1761318953.778871 read 0x8dc000 thru 0x8e1a80 (0x5a81 bytes) 000193[1] 1761318953.779839 mapwrite 0x6c6000 thru 0x6c9eed (0x3eee bytes) 000194[1] 1761318953.891192 mapread 0x4af000 thru 0x4b8f7b (0x9f7c bytes) 000196[0] 1761318954.001828 trunc from 0x9dd236 to 0x1f0a87 (0x7ec7ae bytes) 000197[0] 1761318954.049128 read 0x10c000 thru 0x118878 (0xc879 bytes) 000198[1] 1761318954.083116 read 0x147000 thru 0x147876 (0x877 bytes) 000200[1] 1761318954.141283 read 0x001000 thru 0x009e9c (0x8e9d bytes) 000206[1] 1761318954.168211 mapwrite 0x90e000 thru 0x9154f6 (0x74f7 bytes) 000209[1] 1761318954.377407 mapwrite 0x965000 thru 0x966ef8 (0x1ef9 bytes) 000211[1] 1761318954.480134 read 0x7e8000 thru 0x7f19ff (0x9a00 bytes) 000212[1] 1761318954.493479 mapwrite 0x597000 thru 0x5a642b (0xf42c bytes) 000213[1] 1761318954.541580 mapwrite 0x04b000 thru 0x057335 (0xc336 bytes) 000214[0] 1761318954.680890 mapread 0x317000 thru 0x32193a (0xa93b bytes) 000217[0] 1761318954.816926 mapread 0x7d4000 thru 0x7e13ae (0xd3af bytes) 000218[0] 1761318954.818913 trunc from 0x966ef9 to 0x8025bd (0x16493b bytes) 000219[0] 1761318954.878418 write 0x361000 thru 0x361a07 (0xa08 bytes) 000221[0] 1761318954.886803 write 0x500000 thru 0x50d39f (0xd3a0 bytes) 000224[0] 1761318954.895095 mapwrite 0x3ba000 thru 0x3c9742 (0xf743 bytes) 000225[0] 1761318955.038859 trunc from 0x8025be to 0x48a446 (0x378177 bytes) 000226[0] 1761318955.077555 mapread 0x114000 thru 0x11abb1 (0x6bb2 bytes) 000228[1] 1761318955.139843 read 0x1e3000 thru 0x1ee53f (0xb540 bytes) 000229[1] 1761318955.183312 read 0x047000 thru 0x053b8f (0xcb90 bytes) 000233[1] 1761318955.232556 write 0x814000 thru 0x816b61 (0x2b62 bytes) 000234[1] 1761318955.254230 mapwrite 0x3e0000 thru 0x3ef762 (0xf763 bytes) 000236[1] 1761318955.493023 read 0x130000 thru 0x13464a (0x464b bytes) 000238[0] 1761318955.562903 trunc from 0x816b62 to 0x8a11ca (0x8a669 bytes) 000240[0] 1761318955.628750 mapwrite 0x86d000 thru 0x876a44 (0x9a45 bytes) 000241[1] 1761318955.760239 write 0x0c8000 thru 0x0d3946 (0xb947 bytes) 000242[1] 1761318955.785769 mapread 0x0d3000 thru 0x0e0d77 (0xdd78 bytes) 000244[0] 1761318955.845818 read 0x54a000 thru 0x54fff4 (0x5ff5 bytes) 000246[0] 1761318955.917866 read 0x0cc000 thru 0x0dac43 (0xec44 bytes) 000247[1] 1761318955.959900 read 0x282000 thru 0x28b85c (0x985d bytes) 000248[0] 1761318956.038179 write 0x823000 thru 0x830200 (0xd201 bytes) 000249[1] 1761318956.072156 trunc from 0x8a11cb to 0x794e7d (0x10c34d bytes) 000250[0] 1761318956.096539 mapwrite 0x7f5000 thru 0x8043c5 (0xf3c6 bytes) 000253[0] 1761318956.294566 mapwrite 0x504000 thru 0x505f15 (0x1f16 bytes) 000254[0] 1761318956.354137 read 0x0b3000 thru 0x0b84bb (0x54bc bytes) 000258[0] 1761318956.491172 read 0x77d000 thru 0x782be9 (0x5bea bytes) 000260[1] 1761318956.525991 mapwrite 0x5d2000 thru 0x5de037 (0xc038 bytes) 000262[1] 1761318956.781780 mapread 0x064000 thru 0x07301a (0xf01b bytes) 000266[1] 1761318956.897298 trunc from 0x8043c6 to 0x5bb1ba (0x24920b bytes) 000268[1] 1761318956.939315 write 0x1c3000 thru 0x1c9efe (0x6eff bytes) 000270[0] 1761318956.944253 trunc from 0x5bb1bb to 0x7ccfd7 (0x211e1d bytes) 000271[0] 1761318956.989367 read 0x2e0000 thru 0x2e1b51 (0x1b52 bytes) 000272[0] 1761318957.058333 trunc from 0x7ccfd8 to 0x1a89bf (0x624618 bytes) 000274[1] 1761318957.078753 trunc from 0x1a89c0 to 0x394225 (0x1eb866 bytes) 000277[0] 1761318957.124630 mapwrite 0x879000 thru 0x883f5c (0xaf5d bytes) 000278[0] 1761318957.227308 read 0x620000 thru 0x62efda (0xefdb bytes) 000279[0] 1761318957.271635 mapread 0x445000 thru 0x44f786 (0xa787 bytes) 000283[1] 1761318957.356799 mapread 0x7f3000 thru 0x7f70ae (0x40af bytes) 000287[1] 1761318957.408811 mapread 0x5bd000 thru 0x5c2426 (0x5427 bytes) 000290[1] 1761318957.430943 write 0x3f6000 thru 0x3ff859 (0x985a bytes) 000291[0] 1761318957.459845 mapwrite 0x048000 thru 0x056bb4 (0xebb5 bytes) 000292[0] 1761318957.584347 trunc from 0x883f5d to 0x0ebdba (0x7981a2 bytes) 000294[1] 1761318957.638021 mapwrite 0x584000 thru 0x58bb6d (0x7b6e bytes) 000296[1] 1761318957.777269 mapread 0x521000 thru 0x5262b7 (0x52b8 bytes) 000298[0] 1761318957.827656 mapwrite 0x30f000 thru 0x31b9cb (0xc9cc bytes) 000299[0] 1761318957.977016 mapwrite 0x9d5000 thru 0x9e3c41 (0xec42 bytes) 000301[0] 1761318958.110807 mapread 0x881000 thru 0x8864fb (0x54fc bytes) 000303[1] 1761318958.134389 trunc from 0x9e3c42 to 0x21aa58 (0x7c91e9 bytes) 000307[1] 1761318958.234136 trunc from 0x21aa59 to 0x937aa3 (0x71d04b bytes) 000309[1] 1761318958.268302 mapread 0x76a000 thru 0x7778bd (0xd8be bytes) 000311[0] 1761318958.375996 trunc from 0x937aa4 to 0x3058fe (0x6321a5 bytes) 000314[0] 1761318958.409673 mapwrite 0x055000 thru 0x060875 (0xb876 bytes) 000315[1] 1761318958.523592 write 0x400000 thru 0x40261e (0x261f bytes) 000319[1] 1761318958.546722 read 0x222000 thru 0x222f18 (0xf19 bytes) 000320[0] 1761318958.612342 read 0x3b6000 thru 0x3b7362 (0x1363 bytes) 000321[1] 1761318958.652032 read 0x3ba000 thru 0x3c145a (0x745b bytes) 000322[0] 1761318958.676941 read 0x2c6000 thru 0x2ce2d6 (0x82d7 bytes) 000323[0] 1761318958.703617 mapread 0x389000 thru 0x398e5b (0xfe5c bytes) 000325[1] 1761318958.704443 mapread 0x170000 thru 0x17b239 (0xb23a bytes) 000326[0] 1761318958.755111 write 0x3c3000 thru 0x3c6019 (0x301a bytes) 000327[1] 1761318958.772669 mapread 0x023000 thru 0x02f2a7 (0xc2a8 bytes) 000328[0] 1761318958.858179 read 0x01e000 thru 0x02509e (0x709f bytes) 000329[1] 1761318958.912482 write 0x464000 thru 0x470aee (0xcaef bytes) 000331[0] 1761318958.960434 mapread 0x15b000 thru 0x169cda (0xecdb bytes) 000334[1] 1761318959.070696 write 0x6b1000 thru 0x6ba497 (0x9498 bytes) 000336[1] 1761318959.105207 write 0x454000 thru 0x4594c7 (0x54c8 bytes) 000339[0] 1761318959.144120 write 0x371000 thru 0x375cf9 (0x4cfa bytes) 000340[0] 1761318959.192052 read 0x199000 thru 0x1a0f58 (0x7f59 bytes) 000341[0] 1761318959.230788 mapread 0x636000 thru 0x63fa83 (0x9a84 bytes) 000342[0] 1761318959.233118 mapread 0x600000 thru 0x60f18b (0xf18c bytes) 000343[1] 1761318959.245273 read 0x2f7000 thru 0x2fe6d5 (0x76d6 bytes) 000344[0] 1761318959.314039 trunc from 0x6ba498 to 0x2603f7 (0x45a0a0 bytes) 000345[0] 1761318959.369017 read 0x0ab000 thru 0x0aef15 (0x3f16 bytes) 000348[1] 1761318959.430028 write 0x066000 thru 0x069589 (0x358a bytes) 000349[0] 1761318959.445577 trunc from 0x2603f8 to 0x98dc7f (0x72d888 bytes) 000354[0] 1761318959.534908 write 0x004000 thru 0x00da8d (0x9a8e bytes) 000356[1] 1761318959.599773 trunc from 0x98dc80 to 0x67abb9 (0x3130c6 bytes) 000359[1] 1761318959.677897 mapwrite 0x8fc000 thru 0x90a491 (0xe492 bytes) 000361[1] 1761318959.815361 mapwrite 0x219000 thru 0x21f873 (0x6874 bytes) 000364[0] 1761318959.907402 mapread 0x541000 thru 0x54548f (0x4490 bytes) 000365[1] 1761318959.961901 read 0x82c000 thru 0x8318b3 (0x58b4 bytes) 000366[1] 1761318959.979122 read 0x8cc000 thru 0x8d53f8 (0x93f9 bytes) 000367[0] 1761318959.979682 mapread 0x7c3000 thru 0x7cca65 (0x9a66 bytes) 000369[1] 1761318960.016960 trunc from 0x90a492 to 0x5618d8 (0x3a8bb9 bytes) 000372[1] 1761318960.080582 write 0x0eb000 thru 0x0f8c9c (0xdc9d bytes) truncating to largest ever: 0x9f9081 000374[1] 1761318960.100526 trunc from 0x5618d9 to 0x9f9080 (0x4977a8 bytes) 000377[0] 1761318960.155579 read 0x7a3000 thru 0x7aa0f6 (0x70f7 bytes) 000380[1] 1761318960.216305 mapread 0x828000 thru 0x828a7f (0xa80 bytes) 000385[0] 1761318960.221764 trunc from 0x9f9081 to 0x0f1abd (0x9075c3 bytes) 000386[0] 1761318960.252811 mapread 0x063000 thru 0x06b6da (0x86db bytes) 000387[0] 1761318960.320043 trunc from 0x0f1abe to 0x8a3f8b (0x7b24ce bytes) 000388[0] 1761318960.359929 mapread 0x01d000 thru 0x02bbaf (0xebb0 bytes) 000389[0] 1761318960.457947 mapread 0x177000 thru 0x17fb00 (0x8b01 bytes) 000391[0] 1761318960.507436 trunc from 0x8a3f8c to 0x7f4413 (0xafb78 bytes) 000392[0] 1761318960.527952 mapread 0x368000 thru 0x36e4c7 (0x64c8 bytes) 000394[0] 1761318960.594338 read 0x42d000 thru 0x432df7 (0x5df8 bytes) 000395[0] 1761318960.638453 trunc from 0x7f4414 to 0x619d09 (0x1da70a bytes) 000397[1] 1761318960.655423 mapread 0x010000 thru 0x01feb2 (0xfeb3 bytes) 000398[0] 1761318960.739838 mapread 0x4d5000 thru 0x4d9b71 (0x4b72 bytes) 000399[0] 1761318960.765099 mapwrite 0x6b3000 thru 0x6bd80b (0xa80c bytes) 000400[0] 1761318960.893404 trunc from 0x6bd80c to 0x2a7c2c (0x415bdf bytes) 000401[1] 1761318960.931406 mapwrite 0x3b0000 thru 0x3b3fc8 (0x3fc9 bytes) 000402[0] 1761318961.034865 read 0x1d2000 thru 0x1e0a78 (0xea79 bytes) 000405[0] 1761318961.108760 read_OD 0x0ec000 thru 0x0eefff (0x3000 bytes) 000408[0] 1761318961.113562 read_OD 0x1f1000 thru 0x1fcfff (0xc000 bytes) 000410[0] 1761318961.125003 mapwrite 0x653000 thru 0x653f75 (0xf76 bytes) 000412[1] 1761318961.241968 read 0x351000 thru 0x356d19 (0x5d1a bytes) 000414[1] 1761318961.293105 read 0x2b1000 thru 0x2b1cf6 (0xcf7 bytes) 000415[1] 1761318961.307735 mapread 0x135000 thru 0x13c13b (0x713c bytes) 000416[1] 1761318961.337808 write 0x39e000 thru 0x3a4513 (0x6514 bytes) 000418[0] 1761318961.370025 read_OD 0x5a2000 thru 0x5a4fff (0x3000 bytes) 000421[1] 1761318961.401106 mapread 0x5c4000 thru 0x5c6265 (0x2266 bytes) 000422[1] 1761318961.418033 write 0x4fd000 thru 0x504314 (0x7315 bytes) 000423[0] 1761318961.459607 read_OD 0x56e000 thru 0x57afff (0xd000 bytes) 000425[1] 1761318961.502792 trunc from 0x653f76 to 0x1f09e9 (0x46358c bytes) 000426[1] 1761318961.553634 mapread 0x097000 thru 0x0a1aad (0xaaae bytes) 000428[0] 1761318961.616004 trunc from 0x1f09ea to 0x579ea2 (0x3894b9 bytes) 000430[1] 1761318961.658853 read 0x112000 thru 0x11fdb8 (0xddb9 bytes) 000431[0] 1761318961.709016 mapread 0x344000 thru 0x353bf6 (0xfbf7 bytes) 000432[0] 1761318961.790684 write_OD 0x6d3000 thru 0x6d9fff (0x7000 bytes) 000433[1] 1761318961.839004 trunc from 0x6da000 to 0x345fa3 (0x39405c bytes) 000435[0] 1761318961.869895 mapwrite 0x142000 thru 0x146ce9 (0x4cea bytes) 000438[0] 1761318961.955032 write_OD 0x586000 thru 0x587fff (0x2000 bytes) 000440[1] 1761318962.004126 write 0x9e1000 thru 0x9e97d8 (0x87d9 bytes) 000441[1] 1761318962.013445 mapread 0x264000 thru 0x2687d1 (0x47d2 bytes) 000443[1] 1761318962.074720 trunc from 0x9e97d9 to 0x51aab0 (0x4ced28 bytes) 000446[0] 1761318962.118769 mapwrite 0x2ad000 thru 0x2b7f35 (0xaf36 bytes) 000447[0] 1761318962.209014 write_OD 0x26b000 thru 0x273fff (0x9000 bytes) 000449[1] 1761318962.243948 trunc from 0x51aab1 to 0x2e07f0 (0x23a2c0 bytes) 000451[0] 1761318962.268841 read_OD 0x1c9000 thru 0x1d7fff (0xf000 bytes) 000452[1] 1761318962.286954 read 0x103000 thru 0x10ea7f (0xba80 bytes) 000453[0] 1761318962.313491 mapread 0x190000 thru 0x19285e (0x285f bytes) 000455[0] 1761318962.347348 trunc from 0x2e07f1 to 0x73efd0 (0x45e7e0 bytes) 000456[0] 1761318962.368701 mapwrite 0x22e000 thru 0x237244 (0x9245 bytes) 000458[1] 1761318962.506603 trunc from 0x73efd1 to 0x6c52aa (0x79d26 bytes) 000461[1] 1761318962.534086 read 0x329000 thru 0x330085 (0x7086 bytes) 000462[1] 1761318962.568192 mapwrite 0x062000 thru 0x06fd31 (0xdd32 bytes) 000463[1] 1761318962.688526 write 0x9e5000 thru 0x9e8f3a (0x3f3b bytes) 000465[0] 1761318962.690247 read_OD 0x73c000 thru 0x746fff (0xb000 bytes) 000468[1] 1761318962.725056 write 0x1ad000 thru 0x1b5fe7 (0x8fe8 bytes) 000470[0] 1761318962.782944 read_OD 0x62c000 thru 0x62cfff (0x1000 bytes) 000471[0] 1761318962.816159 mapwrite 0x72e000 thru 0x73c9c6 (0xe9c7 bytes) 000473[0] 1761318962.928247 read_OD 0x3f2000 thru 0x3fafff (0x9000 bytes) 000476[1] 1761318962.955787 trunc from 0x9e8f3b to 0x9036c3 (0xe5877 bytes) 000477[0] 1761318962.976008 read_OD 0x33e000 thru 0x34afff (0xd000 bytes) 000480[1] 1761318962.994774 mapwrite 0x796000 thru 0x7a06d0 (0xa6d1 bytes) 000484[0] 1761318963.122800 mapwrite 0x527000 thru 0x5357a5 (0xe7a6 bytes) 000485[1] 1761318963.248917 trunc from 0x9036c4 to 0x1ecfd5 (0x7166ee bytes) 000487[1] 1761318963.293812 write 0x2de000 thru 0x2e8879 (0xa87a bytes) 000488[1] 1761318963.312299 mapread 0x2b1000 thru 0x2bb1d4 (0xa1d5 bytes) 000489[0] 1761318963.374826 read_OD 0x232000 thru 0x23dfff (0xc000 bytes) 000490[1] 1761318963.405454 write 0x3c5000 thru 0x3d2762 (0xd763 bytes) 000492[0] 1761318963.427272 write_OD 0x7cd000 thru 0x7d8fff (0xc000 bytes) 000493[0] 1761318963.474541 mapwrite 0x8e6000 thru 0x8e78cd (0x18ce bytes) 000497[1] 1761318963.656048 mapwrite 0x9d2000 thru 0x9d222f (0x230 bytes) 000498[1] 1761318963.736862 read 0x622000 thru 0x631a30 (0xfa31 bytes) All operations completed A-OK! PASS 16a (44s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 11:16:06 (1761318966) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.165448 s, 12.7 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.126628 s, 16.6 MB/s Chance of close/open is 1 in 50 Seed set to 8967 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x9990e7 000003[1] 1761318967.927743 trunc from 00000000 to 0x9990e6 (0x9990e7 bytes) 000005[0] 1761318967.964886 read 0x5bf201 thru 0x5bfd9d (0xb9d bytes) 000006[1] 1761318967.989976 mapwrite 0x6d2fc4 thru 0x6d4815 (0x1852 bytes) 000007[1] 1761318968.111622 mapread 0x74a3ce thru 0x74c1a5 (0x1dd8 bytes) 000008[0] 1761318968.132539 trunc from 0x9990e7 to 0x86e334 (0x12adb2 bytes) 000011[1] 1761318968.167340 mapread 0x6f69cf thru 0x6f6dfb (0x42d bytes) 000012[1] 1761318968.193586 mapwrite 0x7218fa thru 0x722366 (0xa6d bytes) 000013[0] 1761318968.263394 write 0x60ba0a thru 0x60bc98 (0x28f bytes) 000015[0] 1761318968.301146 read 0x4d8f26 thru 0x4da358 (0x1433 bytes) 000016[1] 1761318968.318916 trunc from 0x86e335 to 0x47dee1 (0x3f0453 bytes) 000017[0] 1761318968.359793 mapread 0x1bdfb4 thru 0x1bed19 (0xd66 bytes) 000019[1] 1761318968.412952 read 0x0eaf63 thru 0x0ec7b7 (0x1855 bytes) 000020[1] 1761318968.419643 read 0x225baf thru 0x2268c5 (0xd17 bytes) 000024[1] 1761318968.458493 mapwrite 0x0b1f55 thru 0x0b2d4d (0xdf9 bytes) 000026[0] 1761318968.520253 mapread 0x0ca27a thru 0x0cbcc5 (0x1a4c bytes) 000027[0] 1761318968.562085 read 0x2688cf thru 0x269607 (0xd39 bytes) 000029[0] 1761318968.598166 mapread 0x38afe6 thru 0x38c68d (0x16a8 bytes) 000030[0] 1761318968.598834 write 0x61852d thru 0x61a3ea (0x1ebe bytes) 000031[0] 1761318968.614779 write 0x107d62 thru 0x109d49 (0x1fe8 bytes) 000032[0] 1761318968.656862 read 0x0fedfe thru 0x0ff9f6 (0xbf9 bytes) 000033[0] 1761318968.755463 write 0x98604c thru 0x98732a (0x12df bytes) 000034[0] 1761318968.774612 write 0x090ecc thru 0x090f8c (0xc1 bytes) 000036[0] 1761318968.834895 read 0x1d847e thru 0x1d95b5 (0x1138 bytes) 000038[1] 1761318968.872388 mapwrite 0x2ecc89 thru 0x2ed59a (0x912 bytes) 000042[0] 1761318968.996680 read 0x414b96 thru 0x415f08 (0x1373 bytes) 000044[1] 1761318969.061983 write 0x9bdd2d thru 0x9becbf (0xf93 bytes) 000045[0] 1761318969.085679 write 0x6c4790 thru 0x6c5bbb (0x142c bytes) 000046[0] 1761318969.118149 read 0x2438c2 thru 0x245819 (0x1f58 bytes) 000047[0] 1761318969.148835 mapwrite 0x0767ef thru 0x077f3f (0x1751 bytes) 000049[0] 1761318969.227951 mapwrite 0x51327b thru 0x515156 (0x1edc bytes) 000050[1] 1761318969.381160 write 0x75a206 thru 0x75b992 (0x178d bytes) 000052[1] 1761318969.443298 mapwrite 0x7a7021 thru 0x7a7e7b (0xe5b bytes) 000053[0] 1761318969.524035 write 0x4563e5 thru 0x458026 (0x1c42 bytes) 000054[1] 1761318969.541368 write 0x620b4a thru 0x62251b (0x19d2 bytes) 000059[1] 1761318969.638896 write 0x6b361f thru 0x6b4aba (0x149c bytes) 000061[1] 1761318969.680822 mapwrite 0x19fd4f thru 0x1a1b24 (0x1dd6 bytes) 000062[0] 1761318969.737482 mapread 0x3feb35 thru 0x3ffa49 (0xf15 bytes) 000064[1] 1761318969.768173 write 0x00609f thru 0x0077bd (0x171f bytes) 000067[1] 1761318969.800568 read 0x79f923 thru 0x7a04ba (0xb98 bytes) 000068[1] 1761318969.836245 mapread 0x885a32 thru 0x885d52 (0x321 bytes) 000069[1] 1761318969.850614 mapread 0x6aab77 thru 0x6ab71d (0xba7 bytes) 000073[0] 1761318969.887375 write 0x4abb3d thru 0x4ad1ba (0x167e bytes) 000074[1] 1761318969.917184 write 0x2c5d02 thru 0x2c7435 (0x1734 bytes) 000075[0] 1761318969.937083 read 0x3f44bf thru 0x3f5ce1 (0x1823 bytes) 000076[0] 1761318969.953744 mapread 0x910b4b thru 0x9119e2 (0xe98 bytes) 000079[0] 1761318969.977218 read 0x1cc210 thru 0x1cd602 (0x13f3 bytes) 000080[0] 1761318969.988723 mapwrite 0x80d3b1 thru 0x80d8ad (0x4fd bytes) 000082[1] 1761318970.152148 trunc from 0x9becc0 to 0x4a859b (0x516724 bytes) 000083[0] 1761318970.203335 mapwrite 0x8dcd86 thru 0x8de1a7 (0x1422 bytes) 000085[0] 1761318970.329890 trunc from 0x8de1a8 to 0x009fb5 (0x8d41f2 bytes) 000089[0] 1761318970.406949 mapread 0x000304 thru 0x0011a1 (0xe9e bytes) 000090[1] 1761318970.436017 trunc from 0x009fb6 to 0x145fb1 (0x13bffc bytes) 000091[0] 1761318970.480069 read 0x028a5a thru 0x029fde (0x1585 bytes) 000093[0] 1761318970.529722 trunc from 0x145fb2 to 0x31e58f (0x1d85de bytes) 000094[1] 1761318970.584711 read 0x208ef6 thru 0x209725 (0x830 bytes) 000095[1] 1761318970.622468 trunc from 0x31e590 to 0x3510c1 (0x32b32 bytes) 000099[0] 1761318970.701775 mapwrite 0x264e1b thru 0x266c72 (0x1e58 bytes) 000100[0] 1761318970.793980 mapwrite 0x9e3d88 thru 0x9e4d40 (0xfb9 bytes) 000102[1] 1761318970.961167 mapread 0x1ad1e1 thru 0x1ad9b5 (0x7d5 bytes) 000103[1] 1761318971.017772 read 0x1679c1 thru 0x168c98 (0x12d8 bytes) 000104[1] 1761318971.043370 read 0x384d36 thru 0x384f40 (0x20b bytes) 000106[1] 1761318971.044144 read 0x7a2ac2 thru 0x7a39e5 (0xf24 bytes) 000108[0] 1761318971.094381 mapwrite 0x1d7cf5 thru 0x1d95d4 (0x18e0 bytes) 000110[1] 1761318971.172779 mapwrite 0x8a7b5f thru 0x8a8335 (0x7d7 bytes) 000111[1] 1761318971.255881 trunc from 0x9e4d41 to 0x22a369 (0x7ba9d7 bytes) 000113[1] 1761318971.302601 mapwrite 0x9c1a7c thru 0x9c2a6a (0xfef bytes) 000114[1] 1761318971.372435 mapwrite 0x95da78 thru 0x95de88 (0x411 bytes) 000115[1] 1761318971.408274 read 0x08603e thru 0x08696c (0x92f bytes) 000116[0] 1761318971.517129 write 0x41f242 thru 0x420d17 (0x1ad6 bytes) 000121[0] 1761318971.624760 read 0x640eaa thru 0x64139b (0x4f2 bytes) 000123[1] 1761318971.675651 trunc from 0x9c2a6b to 0x923a60 (0x9f00a bytes) 000124[0] 1761318971.722556 trunc from 0x923a61 to 0x793364 (0x1906fc bytes) 000128[0] 1761318971.758434 write 0x31f4b2 thru 0x3213ee (0x1f3d bytes) 000134[0] 1761318971.828338 write 0x0488e8 thru 0x048b97 (0x2b0 bytes) 000135[1] 1761318971.832027 trunc from 0x793365 to 0x5d3af6 (0x1bf86e bytes) 000138[1] 1761318971.906876 mapwrite 0x12b189 thru 0x12ba5a (0x8d2 bytes) 000139[1] 1761318971.965310 write 0x995acb thru 0x9961f2 (0x728 bytes) 000140[1] 1761318971.978461 trunc from 0x9961f3 to 0x7e0f71 (0x1b5281 bytes) 000142[0] 1761318972.014189 write 0x7677ee thru 0x7678b7 (0xca bytes) 000143[1] 1761318972.053977 trunc from 0x7e0f72 to 0x519a50 (0x2c7521 bytes) 000145[0] 1761318972.115587 write 0x72dc72 thru 0x72dcb0 (0x3f bytes) 000148[0] 1761318972.174087 mapread 0x0293d4 thru 0x02a3b5 (0xfe2 bytes) 000149[1] 1761318972.202858 trunc from 0x72dcb1 to 0x3dbe7c (0x351e34 bytes) 000150[0] 1761318972.248033 mapwrite 0x90977f thru 0x90a06f (0x8f1 bytes) 000151[0] 1761318972.321664 mapread 0x77b4f8 thru 0x77bbfc (0x705 bytes) 000152[0] 1761318972.346911 trunc from 0x90a070 to 0x043b3d (0x8c6532 bytes) 000155[0] 1761318972.376153 read 0x03c2e9 thru 0x03d09e (0xdb6 bytes) 000157[1] 1761318972.394201 trunc from 0x043b3e to 0x30863c (0x2c4aff bytes) 000158[1] 1761318972.435631 mapwrite 0x254604 thru 0x25510e (0xb0b bytes) 000160[0] 1761318972.519097 mapread 0x0b4b70 thru 0x0b5994 (0xe25 bytes) 000161[0] 1761318972.555187 mapread 0x178c2f thru 0x17ab6f (0x1f41 bytes) 000164[0] 1761318972.585721 mapread 0x1b9fa7 thru 0x1bbb41 (0x1b9b bytes) 000165[0] 1761318972.597355 write 0x75de28 thru 0x75f838 (0x1a11 bytes) 000167[0] 1761318972.634395 write 0x5e8d42 thru 0x5e9d88 (0x1047 bytes) 000168[1] 1761318972.648139 write 0x604a1b thru 0x6065f9 (0x1bdf bytes) 000169[1] 1761318972.671443 mapread 0x3dcc7f thru 0x3de0c1 (0x1443 bytes) 000172[1] 1761318972.714035 read 0x1ec9c6 thru 0x1ee5dc (0x1c17 bytes) 000173[1] 1761318972.794617 read 0x47ae3c thru 0x47c224 (0x13e9 bytes) 000174[0] 1761318972.814263 mapwrite 0x42d006 thru 0x42edbd (0x1db8 bytes) 000175[0] 1761318972.885267 read 0x6f5447 thru 0x6f676e (0x1328 bytes) 000176[1] 1761318972.895178 read 0x5a3cb9 thru 0x5a5851 (0x1b99 bytes) 000180[1] 1761318972.928083 mapread 0x290a3f thru 0x2925e3 (0x1ba5 bytes) 000182[1] 1761318972.962860 read 0x0c6a59 thru 0x0c714d (0x6f5 bytes) 000184[1] 1761318973.008074 write 0x4c14e9 thru 0x4c2ffe (0x1b16 bytes) 000185[1] 1761318973.047659 read 0x550a4f thru 0x55116e (0x720 bytes) 000186[0] 1761318973.048141 trunc from 0x75f839 to 0x59f44a (0x1c03ee bytes) 000187[1] 1761318973.093543 write 0x8ed61e thru 0x8ee280 (0xc63 bytes) 000192[0] 1761318973.183814 mapwrite 0x98948d thru 0x98ae69 (0x19dd bytes) 000193[0] 1761318973.292398 write 0x67a9c9 thru 0x67be6f (0x14a7 bytes) 000195[0] 1761318973.327029 trunc from 0x98ae6a to 0x606245 (0x384c24 bytes) 000197[0] 1761318973.375597 trunc from 0x606246 to 0x7d34df (0x1cd29a bytes) truncating to largest ever: 0x9f4027 000199[0] 1761318973.408903 trunc from 0x7d34e0 to 0x9f4026 (0x220b47 bytes) 000203[1] 1761318973.437969 read 0x43479a thru 0x4363d2 (0x1c39 bytes) 000204[0] 1761318973.468036 write 0x09c9b5 thru 0x09d8d2 (0xf1e bytes) 000206[0] 1761318973.483995 trunc from 0x9f4027 to 0x729daa (0x2ca27c bytes) 000208[0] 1761318973.499542 mapread 0x204184 thru 0x204e81 (0xcfe bytes) 000215[1] 1761318973.548993 mapread 0x6e32f0 thru 0x6e345c (0x16d bytes) 000217[1] 1761318973.579972 read 0x4671d5 thru 0x467acd (0x8f9 bytes) 000220[0] 1761318973.617927 read 0x0414f1 thru 0x0426f8 (0x1208 bytes) 000221[0] 1761318973.693942 read 0x0a31ec thru 0x0a4197 (0xfac bytes) 000224[0] 1761318973.709631 read 0x300f9c thru 0x301300 (0x365 bytes) 000226[1] 1761318973.725221 write 0x2d544f thru 0x2d56cc (0x27e bytes) 000227[1] 1761318973.770014 read 0x147b78 thru 0x149400 (0x1889 bytes) 000229[0] 1761318973.842542 mapwrite 0x5c89af thru 0x5ca2e8 (0x193a bytes) 000230[1] 1761318973.909537 write 0x6a5ba4 thru 0x6a6a05 (0xe62 bytes) 000234[1] 1761318973.953790 mapread 0x427e36 thru 0x429de8 (0x1fb3 bytes) 000235[1] 1761318973.970238 trunc from 0x729dab to 0x87d2f3 (0x153549 bytes) 000236[0] 1761318974.002052 write 0x0aabba thru 0x0acad2 (0x1f19 bytes) 000238[1] 1761318974.066477 mapwrite 0x02a896 thru 0x02c2a3 (0x1a0e bytes) 000242[1] 1761318974.182884 trunc from 0x87d2f4 to 0x866ab2 (0x16841 bytes) 000245[1] 1761318974.224318 write 0x8f2860 thru 0x8f371b (0xebc bytes) 000246[1] 1761318974.225381 mapread 0x1f2147 thru 0x1f2511 (0x3cb bytes) 000248[0] 1761318974.269096 write 0x88d852 thru 0x88ec78 (0x1427 bytes) 000252[0] 1761318974.353786 read 0x3c15ee thru 0x3c28a5 (0x12b8 bytes) 000253[0] 1761318974.412239 read 0x6735b7 thru 0x67404f (0xa99 bytes) 000254[1] 1761318974.426448 mapwrite 0x5982d7 thru 0x5984c9 (0x1f3 bytes) 000256[0] 1761318974.494814 write 0x63a557 thru 0x63bf14 (0x19be bytes) 000257[1] 1761318974.509813 mapwrite 0x20c64f thru 0x20d9f5 (0x13a7 bytes) 000261[1] 1761318974.620112 write 0x095014 thru 0x096a4e (0x1a3b bytes) 000264[1] 1761318974.661765 mapread 0x4cde5e thru 0x4cf9ce (0x1b71 bytes) 000265[1] 1761318974.688167 trunc from 0x8f371c to 0x302476 (0x5f12a5 bytes) 000266[0] 1761318974.706231 read 0x084d43 thru 0x08530f (0x5cd bytes) 000268[0] 1761318974.754587 write 0x59b488 thru 0x59b585 (0xfe bytes) 000270[0] 1761318974.787487 write 0x6598c7 thru 0x65b2a1 (0x19db bytes) 000273[1] 1761318974.830236 write 0x85519a thru 0x8570c6 (0x1f2d bytes) 000274[1] 1761318974.847925 write 0x228419 thru 0x228538 (0x120 bytes) 000276[1] 1761318974.867635 mapwrite 0x8425ec thru 0x84445e (0x1e73 bytes) 000277[0] 1761318974.924957 mapread 0x426b2e thru 0x4271aa (0x67d bytes) 000282[0] 1761318975.010695 write 0x7da98e thru 0x7dc5f6 (0x1c69 bytes) 000283[0] 1761318975.011432 trunc from 0x8570c7 to 0x38e0dc (0x4c8fea bytes) 000286[0] 1761318975.061716 trunc from 0x38e0dd to 0x557d58 (0x1c9c7c bytes) 000288[1] 1761318975.099185 trunc from 0x557d59 to 0x558593 (0x83b bytes) 000289[0] 1761318975.123571 write 0x2de2a9 thru 0x2df563 (0x12bb bytes) 000292[0] 1761318975.169678 mapwrite 0x732e29 thru 0x73474f (0x1927 bytes) 000295[0] 1761318975.244245 trunc from 0x734750 to 0x82f3d0 (0xfac81 bytes) 000296[0] 1761318975.254465 mapwrite 0x94b0d7 thru 0x94cbe6 (0x1b10 bytes) 000298[1] 1761318975.325902 trunc from 0x94cbe7 to 0x7c7d4e (0x184e98 bytes) 000300[0] 1761318975.370986 write 0x077dce thru 0x079b9f (0x1dd2 bytes) 000301[1] 1761318975.392537 trunc from 0x7c7d4f to 0x7a2f78 (0x24dd6 bytes) 000305[0] 1761318975.451401 mapread 0x747fe9 thru 0x7498fe (0x1916 bytes) 000307[0] 1761318975.496919 read 0x6dd835 thru 0x6de0e8 (0x8b4 bytes) 000308[0] 1761318975.517331 mapwrite 0x7e3c85 thru 0x7e517c (0x14f8 bytes) 000311[1] 1761318975.588307 mapwrite 0x2e68aa thru 0x2e8202 (0x1959 bytes) 000312[1] 1761318975.644715 write 0x6229cb thru 0x622a2f (0x65 bytes) 000313[0] 1761318975.656175 mapread 0x4a965d thru 0x4a9aed (0x491 bytes) 000315[0] 1761318975.673577 trunc from 0x7e517d to 0x1035e8 (0x6e1b94 bytes) 000316[0] 1761318975.701866 trunc from 0x1035e9 to 0x1aa887 (0xa729f bytes) 000317[0] 1761318975.725034 mapread 0x18a1f9 thru 0x18b4f5 (0x12fd bytes) 000318[0] 1761318975.735484 trunc from 0x1aa888 to 0x65e850 (0x4b3fc9 bytes) 000319[1] 1761318975.749384 mapwrite 0x568968 thru 0x56a8f5 (0x1f8e bytes) 000320[0] 1761318975.821917 write 0x78b2ea thru 0x78cab5 (0x17cc bytes) 000322[0] 1761318975.865625 mapwrite 0x765e7a thru 0x766306 (0x48d bytes) 000323[0] 1761318975.907928 write 0x746942 thru 0x74770b (0xdca bytes) 000324[1] 1761318975.922883 mapwrite 0x364455 thru 0x364862 (0x40e bytes) 000325[0] 1761318975.977380 mapwrite 0x9f92bd thru 0x9f95e7 (0x32b bytes) 000330[1] 1761318976.082970 write 0x5dec2a thru 0x5dfc2f (0x1006 bytes) 000331[1] 1761318976.103065 trunc from 0x9f95e8 to 0x3fd763 (0x5fbe84 bytes) 000333[1] 1761318976.121890 trunc from 0x3fd764 to 0x5537a2 (0x15603f bytes) 000335[1] 1761318976.159091 read 0x2c6614 thru 0x2c7fc5 (0x19b2 bytes) 000337[0] 1761318976.209278 mapread 0x45c56f thru 0x45c755 (0x1e7 bytes) 000339[1] 1761318976.244713 read 0x1ef323 thru 0x1f0ebb (0x1b99 bytes) 000341[0] 1761318976.259434 read 0x428ed5 thru 0x42995f (0xa8b bytes) 000342[1] 1761318976.278007 write 0x7b65f3 thru 0x7b7c54 (0x1662 bytes) 000345[0] 1761318976.300775 mapread 0x6f8e29 thru 0x6fa208 (0x13e0 bytes) 000348[1] 1761318976.370473 mapread 0x2f34cf thru 0x2f3c86 (0x7b8 bytes) 000354[0] 1761318976.399335 read 0x1f8a43 thru 0x1f919d (0x75b bytes) 000357[1] 1761318976.467994 trunc from 0x7b7c55 to 0x108dad (0x6aeea7 bytes) 000359[1] 1761318976.522475 read 0x047c58 thru 0x0486b1 (0xa5a bytes) 000361[0] 1761318976.552907 read 0x0d204c thru 0x0d357b (0x1530 bytes) 000362[0] 1761318976.578377 read 0x02cedd thru 0x02dc39 (0xd5d bytes) 000363[1] 1761318976.599452 write 0x909920 thru 0x909cfe (0x3df bytes) 000364[0] 1761318976.605742 mapread 0x6a0d09 thru 0x6a14f6 (0x7ee bytes) 000365[1] 1761318976.633490 mapwrite 0x40b647 thru 0x40ccf7 (0x16b1 bytes) 000366[1] 1761318976.728912 write 0x98dc21 thru 0x98f81b (0x1bfb bytes) 000367[0] 1761318976.741476 mapread 0x172d02 thru 0x173802 (0xb01 bytes) 000368[0] 1761318976.796146 mapwrite 0x52752c thru 0x528b63 (0x1638 bytes) 000369[1] 1761318976.868852 write 0x741434 thru 0x742dd1 (0x199e bytes) 000370[0] 1761318976.892624 trunc from 0x98f81c to 0x246b14 (0x748d07 bytes) 000371[1] 1761318976.934232 mapread 0x0d3332 thru 0x0d3a68 (0x737 bytes) 000372[1] 1761318976.959617 mapread 0x0b8224 thru 0x0b93fa (0x11d7 bytes) 000373[1] 1761318976.971757 mapwrite 0x4a4743 thru 0x4a52ac (0xb6a bytes) 000375[1] 1761318977.044200 read 0x46c8e2 thru 0x46d6ff (0xe1e bytes) 000376[1] 1761318977.052586 trunc from 0x4a52ad to 0x723d3a (0x27ea8e bytes) 000380[0] 1761318977.092935 mapread 0x6f8e97 thru 0x6f9fdf (0x1149 bytes) 000381[0] 1761318977.111406 mapread 0x140094 thru 0x14171c (0x1689 bytes) 000382[0] 1761318977.125676 trunc from 0x723d3b to 0x8cd0fc (0x1a93c2 bytes) 000385[0] 1761318977.170492 write 0x841392 thru 0x84277d (0x13ec bytes) 000390[1] 1761318977.198183 write 0x145ece thru 0x147e37 (0x1f6a bytes) 000393[1] 1761318977.251413 trunc from 0x8cd0fd to 0x77cdc9 (0x150333 bytes) 000394[0] 1761318977.277318 mapwrite 0x963fbe thru 0x965b35 (0x1b78 bytes) 000396[1] 1761318977.392060 trunc from 0x965b36 to 0x529bac (0x43bf89 bytes) 000399[1] 1761318977.435063 read 0x05a206 thru 0x05b84a (0x1645 bytes) 000400[1] 1761318977.478378 write 0x5c10f2 thru 0x5c2b98 (0x1aa7 bytes) 000405[0] 1761318977.501154 write 0x921d58 thru 0x9231e8 (0x1491 bytes) 000406[1] 1761318977.527388 mapread 0x071b32 thru 0x072c21 (0x10f0 bytes) 000407[1] 1761318977.540813 mapwrite 0x556831 thru 0x557a5a (0x122a bytes) 000408[0] 1761318977.633439 read 0x03d49e thru 0x03ed0d (0x1870 bytes) 000410[1] 1761318977.688873 write 0x6a9868 thru 0x6aa6a2 (0xe3b bytes) 000415[0] 1761318977.714022 mapwrite 0x09a8af thru 0x09ad04 (0x456 bytes) 000416[1] 1761318977.780636 mapwrite 0x9768ec thru 0x9779d7 (0x10ec bytes) 000417[1] 1761318977.884134 mapwrite 0x9e8f7d thru 0x9e8fac (0x30 bytes) 000419[1] 1761318977.959389 write 0x4efb76 thru 0x4f0e0f (0x129a bytes) 000421[0] 1761318978.004445 write 0x90f1ee thru 0x90ff7f (0xd92 bytes) 000422[0] 1761318978.049977 write 0x655384 thru 0x656450 (0x10cd bytes) 000423[1] 1761318978.143187 trunc from 0x9e8fad to 0x85b22e (0x18dd7e bytes) 000432[1] 1761318978.315716 read 0x3db7a9 thru 0x3dd097 (0x18ef bytes) 000434[0] 1761318978.381126 trunc from 0x85b22f to 0x19c7b4 (0x6bea7a bytes) 000438[1] 1761318978.449193 trunc from 0x19c7b5 to 0x95d93e (0x7c118a bytes) 000439[0] 1761318978.487587 read 0x8f81c3 thru 0x8f9214 (0x1052 bytes) 000442[0] 1761318978.529435 trunc from 0x95d93f to 0x1c68c3 (0x79707b bytes) 000445[0] 1761318978.589964 mapwrite 0x482508 thru 0x482e8d (0x986 bytes) 000448[0] 1761318978.668073 mapwrite 0x66e2b5 thru 0x66f125 (0xe71 bytes) 000449[1] 1761318978.736755 mapwrite 0x88ac6f thru 0x88ae37 (0x1c9 bytes) 000451[1] 1761318978.818272 write 0x605f3f thru 0x606790 (0x852 bytes) 000453[1] 1761318978.843448 trunc from 0x88ae38 to 0x627c1f (0x263218 bytes) 000454[0] 1761318978.867079 mapwrite 0x5d8210 thru 0x5d8f6f (0xd60 bytes) 000456[1] 1761318978.953859 trunc from 0x627c20 to 0x2b55b0 (0x37266f bytes) 000457[1] 1761318978.989888 read 0x0c6769 thru 0x0c78a3 (0x113b bytes) 000459[0] 1761318979.020771 trunc from 0x2b55b1 to 0x9d6c4b (0x72169b bytes) 000460[0] 1761318979.083146 mapwrite 0x812761 thru 0x813e23 (0x16c3 bytes) 000461[1] 1761318979.151258 trunc from 0x9d6c4c to 0x2d20d8 (0x704b73 bytes) 000465[0] 1761318979.193117 read 0x157946 thru 0x1582c3 (0x97e bytes) 000466[1] 1761318979.225946 trunc from 0x2d20d9 to 0x869894 (0x5977bc bytes) 000467[1] 1761318979.257927 trunc from 0x869895 to 0x3d82d5 (0x4915bf bytes) 000468[1] 1761318979.272785 write 0x5e8fa1 thru 0x5ea6d9 (0x1739 bytes) 000470[0] 1761318979.295749 read 0x5b5393 thru 0x5b59df (0x64d bytes) 000472[1] 1761318979.357563 mapwrite 0x342d98 thru 0x343c0d (0xe76 bytes) 000474[0] 1761318979.494658 trunc from 0x5ea6da to 0x11f1d6 (0x4cb503 bytes) 000475[1] 1761318979.545592 read 0x0a69c2 thru 0x0a8835 (0x1e74 bytes) 000477[0] 1761318979.591466 mapwrite 0x5da069 thru 0x5db058 (0xff0 bytes) 000478[1] 1761318979.677565 mapread 0x35c7d7 thru 0x35dc59 (0x1483 bytes) 000481[0] 1761318979.775195 write 0x684d2c thru 0x685a43 (0xd18 bytes) 000488[0] 1761318979.816663 mapwrite 0x2bf0e0 thru 0x2c03a0 (0x12c1 bytes) 000489[1] 1761318979.873332 read 0x379757 thru 0x3799a0 (0x24a bytes) 000494[0] 1761318979.926541 read 0x501b9d thru 0x501f3b (0x39f bytes) 000497[0] 1761318979.976169 mapwrite 0x8580b6 thru 0x8590db (0x1026 bytes) 000498[0] 1761318980.072855 trunc from 0x8590dc to 0x95ffc8 (0x106eed bytes) 000499[0] 1761318980.102621 trunc from 0x95ffc9 to 0x93e06d (0x21f5b bytes) All operations completed A-OK! PASS 16b (16s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 11:16:22 (1761318982) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (2s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 11:16:24 (1761318984) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.204061 s, 10.3 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.136354 s, 15.4 MB/s ldlm.namespaces.MGC192.168.203.143@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffffa0ec47e05000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffffa0ec56019000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffffa0ec47e05000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffffa0ec56019000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffffa0ec47e05000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffffa0ec56019000.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.9637 s, 17.6 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.54176 s, 68.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.21835 s, 86.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 3.13909 s, 33.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.22331 s, 16.8 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.24769 s, 46.7 MB/s PASS 16d (28s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 11:16:52 (1761319012) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.270435 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0748158 s, 14.0 MB/s PASS 16e (5s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 11:16:57 (1761319017) TIMEOUT PASS 16f (25s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 11:17:22 (1761319042) TIMEOUT PASS 16g (25s) == sanityn test 16i: read after truncate file ============ 11:17:47 (1761319067) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (4s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 11:17:51 (1761319071) fail_loc=0x8000030a PASS 17 (7s) == sanityn test 18: mmap sanity check =========================================================================================== 11:17:58 (1761319078) mmap test1: basic mmap operation (PASS, 0.067163s) mmap test2: MAP_PRIVATE not write back (PASS, 0.042165s) mmap test3: concurrent mmap ops on two nodes (PASS, 17.808s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.2435s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.064051s) mmap test6: check mmap write/read content on two nodes (PASS, 0.092604s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0808s) PASS 18 (26s) == sanityn test 19: test concurrent uncached read races ========================================================================= 11:18:24 (1761319104) oleg343-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg343-client: oleg343-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (2s) == sanityn test 20: test extra readahead page left in cache ============================================================== 11:18:26 (1761319106) PASS 20 (4s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 11:18:30 (1761319110) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (4s) == sanityn test 23: others should see updated atime while another read============================================================== 11:18:34 (1761319114) now is 1761319115 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7123 new atime is 1761319176 PASS 23 (65s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 11:19:39 (1761319179) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 12288 7526400 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 12288 7526400 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 374 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 375 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.2K 369 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 374 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 375 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 369 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 7.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 5.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 12.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118006 374 117632 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118071 375 117696 1% /mnt/lustre2[OST:1] filesystem_summary: 235697 369 235328 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 12288 7526400 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 374 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 375 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 369 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118006 374 117632 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118071 375 117696 1% /mnt/lustre[OST:1] filesystem_summary: 235697 369 235328 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118006 374 117632 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118071 375 117696 1% /mnt/lustre2[OST:1] filesystem_summary: 235697 369 235328 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 12288 7526400 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 12288 7526400 1% /mnt/lustre2 PASS 24a (4s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 11:19:43 (1761319183) PASS 24b (3s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 11:19:46 (1761319186) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (4s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 11:19:50 (1761319190) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (2s) == sanityn test 26a: allow mtime to get older ============ 11:19:52 (1761319192) utime: good mknod times 1761319192 <= 1761319193 <= 1761319193 for /mnt/lustre/f26a utime: good mknod times 1761319192 <= 1761319193 <= 1761319193 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (5s) == sanityn test 26b: sync mtime between ost and mds ====== 11:19:57 (1761319197) PASS 26b (6s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 11:20:05 (1761319205) 0 PASS 30 (5s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 11:20:10 (1761319210) fail_loc=0x314 PASS 31a (5s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 11:20:15 (1761319215) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg343-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.203.143@tcp:/lustre 7542784 13312 7523328 1% /mnt/lustre2 PASS 31b (15s) == sanityn test 31r: open-rename(replace) race =========== 11:20:30 (1761319230) -rw-r--r-- 1 root root 0 Oct 24 11:20 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (7s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 11:20:38 (1761319238) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (2s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 11:20:40 (1761319240) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (2s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 11:20:42 (1761319242) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (2s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 11:20:45 (1761319245) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (3s) == sanityn test 34: no lock timeout under IO ============= 11:20:47 (1761319247) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg343-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0ec47e05000.ost_server_uuid,osc.lustre-OST0000-osc-ffffa0ec56019000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffffa0ec47e05000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffffa0ec56019000.ost_server_uuid in FULL state after 0 sec oleg343-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffffa0ec47e05000.ost_server_uuid,osc.lustre-OST0001-osc-ffffa0ec56019000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffffa0ec47e05000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffffa0ec56019000.ost_server_uuid in IDLE state after 0 sec oleg343-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0ec47e05000.ost_server_uuid,osc.lustre-OST0000-osc-ffffa0ec56019000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffffa0ec47e05000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffffa0ec56019000.ost_server_uuid in IDLE state after 0 sec oleg343-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffffa0ec47e05000.ost_server_uuid,osc.lustre-OST0001-osc-ffffa0ec56019000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffffa0ec47e05000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffffa0ec56019000.ost_server_uuid in IDLE state after 0 sec oleg343-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0ec47e05000.ost_server_uuid,osc.lustre-OST0000-osc-ffffa0ec56019000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffffa0ec47e05000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffffa0ec56019000.ost_server_uuid in IDLE state after 0 sec oleg343-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffffa0ec47e05000.ost_server_uuid,osc.lustre-OST0001-osc-ffffa0ec56019000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffffa0ec47e05000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffffa0ec56019000.ost_server_uuid in IDLE state after 0 sec PASS 34 (90s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 11:22:17 (1761319337) fail_loc=0x80000317 Race attempt 0 Wait for 44652 44776 for 60 sec... - open/close 1315 (time 1761319350.14 total 10.00 last 131.45) - open/close 2609 (time 1761319360.15 total 20.01 last 129.35) total: 4000 open/close in 27.63 seconds: 144.76 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.203.143@tcp:/lustre 7.2G 14M 7.2G 1% /mnt/lustre 192.168.203.143@tcp:/lustre 7.2G 14M 7.2G 1% /mnt/lustre2 PASS 35 (66s) == sanityn test 36: handle ESTALE/open-unlink correctly == 11:23:23 (1761319403) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg343-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.501041 s, 105 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7123 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.425986 s, 123 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7123 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.581754 s, 90.1 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7123 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7524352) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.4402 s, 119 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7123 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7522304) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.421025 s, 125 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7123 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.469066 s, 112 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7123 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.44771 s, 117 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7123 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.412913 s, 127 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7123 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.432105 s, 121 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7123 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 2.02178 s, 25.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7123 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7524352) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.50377 s, 34.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7123 sleep 5 for ZFS MDS Waiting for MDT destroys to complete *** cycle(10) *** before(7522304) after_dd(7473152) after(7524352) PASS 36 (253s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 11:27:36 (1761319656) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7123 - create 1483 (time 1761319669.60 total 10.01 last 148.16) - create 3202 (time 1761319679.60 total 20.01 last 171.87) - create 4870 (time 1761319689.60 total 30.01 last 166.80) - create 6621 (time 1761319699.61 total 40.02 last 175.00) - create 8394 (time 1761319709.62 total 50.03 last 177.08) total: 10000 create in 59.17 seconds: 169.00 ops/second PASS 37 (67s) == sanityn test 39a: test from 11063 ============================================================================================ 11:28:43 (1761319723) repeat after cancel_lru_locks PASS 39a (6s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 11:28:49 (1761319729) repeat after cancel_lru_locks PASS 39b (6s) == sanityn test 39c: check truncate mtime update ================================================================================ 11:28:55 (1761319735) repeat after cancel_lru_locks PASS 39c (6s) == sanityn test 39d: sync write should update mtime ====== 11:29:01 (1761319741) fail_loc=0x411 fail_loc=0 PASS 39d (5s) == sanityn test 40a: pdirops: create vs others ======================================================================== 11:29:06 (1761319746) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (13s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 11:29:19 (1761319759) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (14s) == sanityn test 40c: pdirops: link and others ======================================================================== 11:29:33 (1761319773) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (15s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 11:29:49 (1761319789) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (13s) == sanityn test 40e: pdirops: rename and others ======================================================================== 11:30:02 (1761319802) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (14s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 11:30:16 (1761319816) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (10s) == sanityn test 41b: pdirops: create vs create ======================================================================== 11:30:26 (1761319826) open: File exists Conflict PASS 41b (10s) == sanityn test 41c: pdirops: create vs link ======================================================================== 11:30:36 (1761319836) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (12s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 11:30:48 (1761319848) Conflict PASS 41d (11s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 11:30:59 (1761319859) Conflict PASS 41e (9s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 11:31:09 (1761319869) Conflict PASS 41f (10s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 11:31:19 (1761319879) Conflict PASS 41g (9s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 11:31:28 (1761319888) Conflict PASS 41h (11s) == sanityn test 41i: reint_open: create vs create ======== 11:31:39 (1761319899) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (922s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 11:47:01 (1761320821) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (8s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 11:47:09 (1761320829) open: File exists Conflict PASS 42b (8s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 11:47:17 (1761320837) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (8s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 11:47:25 (1761320845) Conflict PASS 42d (8s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 11:47:34 (1761320854) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (9s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 11:47:42 (1761320862) Conflict PASS 42f (8s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 11:47:50 (1761320870) Conflict PASS 42g (8s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 11:47:58 (1761320878) Conflict PASS 42h (8s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 11:48:06 (1761320886) PASS 43a (28s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 11:48:34 (1761320914) Conflict PASS 43b (7s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 11:48:41 (1761320921) Conflict PASS 43c (7s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 11:48:48 (1761320928) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (7s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 11:48:55 (1761320935) Conflict PASS 43e (8s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 11:49:03 (1761320943) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (8s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 11:49:11 (1761320951) stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (7s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 11:49:18 (1761320958) Conflict PASS 43h (7s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 11:49:25 (1761320965) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (2s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 11:49:27 (1761320967) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (66s) == sanityn test 43k: unlink vs create ==================== 11:50:33 (1761321033) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (586s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 12:00:19 (1761321619) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Fri Oct 24 12:00:25 EDT 2025 PASS 44a (8s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 12:00:27 (1761321627) open: File exists Conflict PASS 44b (6s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 12:00:33 (1761321633) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (7s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 12:00:40 (1761321640) Conflict PASS 44d (7s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 12:00:47 (1761321647) Conflict PASS 44e (7s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 12:00:54 (1761321654) Conflict PASS 44f (7s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 12:01:01 (1761321661) Conflict PASS 44g (6s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 12:01:07 (1761321667) Conflict PASS 44h (6s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 12:01:13 (1761321673) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (2s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 12:01:15 (1761321675) PASS 45a (44s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 12:01:59 (1761321719) Conflict PASS 45b (6s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 12:02:05 (1761321725) Conflict PASS 45c (6s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 12:02:11 (1761321731) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (6s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 12:02:17 (1761321737) Conflict PASS 45e (6s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 12:02:23 (1761321743) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (7s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 12:02:30 (1761321750) stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (6s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 12:02:36 (1761321756) Conflict PASS 45h (8s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 12:02:44 (1761321764) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (2s) == sanityn test 45j: read vs rename ====================== 12:02:46 (1761321766) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (657s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 12:13:43 (1761322423) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (7s) == sanityn test 46b: pdirops: link vs create ======================================================================== 12:13:50 (1761322430) open: File exists Conflict PASS 46b (7s) == sanityn test 46c: pdirops: link vs link ======================================================================== 12:13:57 (1761322437) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (7s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 12:14:04 (1761322444) Conflict PASS 46d (7s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 12:14:12 (1761322452) Conflict PASS 46e (8s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 12:14:19 (1761322459) Conflict PASS 46f (7s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 12:14:26 (1761322466) Conflict PASS 46g (8s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 12:14:34 (1761322474) Conflict PASS 46h (7s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 12:14:42 (1761322482) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 12:14:43 (1761322483) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (2s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 12:14:45 (1761322485) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (2s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 12:14:47 (1761322487) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 12:14:48 (1761322488) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (2s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 12:14:50 (1761322490) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 12:14:51 (1761322491) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (2s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 12:14:53 (1761322493) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 12:14:55 (1761322495) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.0399554 s, 256 kB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (8s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 12:15:03 (1761322503) 0+1 records in 0+1 records out 158 bytes copied, 0.01377 s, 11.5 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (242351) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 12:15:08 (1761322508) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00451961 s, 227 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.03037 s, 34.5 MB/s 3145728 PASS 51b (16s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 12:15:23 (1761322523) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0122511 s, 83.6 kB/s PASS 51c (7s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 12:15:30 (1761322530) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179692 s, 58.4 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (4s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 12:15:34 (1761322534) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2433 0x981 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2433 0x981 0 read lease(1) has applied. PASS 51e (5s) == sanityn test 54: rename locking ======================= 12:15:39 (1761322539) llite.lustre-ffffa0ec47e05000.xattr_cache=1 llite.lustre-ffffa0ec56019000.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504962 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 12:15:40.000000000 -0400 Modify: 2025-10-24 12:15:40.000000000 -0400 Change: 2025-10-24 12:15:40.000000000 -0400 Birth: 2025-10-24 12:15:39.000000000 -0400 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504965 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 12:15:46.000000000 -0400 Modify: 2025-10-24 12:15:46.000000000 -0400 Change: 2025-10-24 12:15:46.000000000 -0400 Birth: 2025-10-24 12:15:45.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 12:15:51.000000000 -0400 Modify: 2025-10-24 12:15:51.000000000 -0400 Change: 2025-10-24 12:15:51.000000000 -0400 Birth: 2025-10-24 12:15:50.000000000 -0400 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504967 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 12:15:51.000000000 -0400 Modify: 2025-10-24 12:15:52.000000000 -0400 Change: 2025-10-24 12:15:52.000000000 -0400 Birth: 2025-10-24 12:15:50.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504970 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 12:15:57.000000000 -0400 Modify: 2025-10-24 12:15:57.000000000 -0400 Change: 2025-10-24 12:15:57.000000000 -0400 Birth: 2025-10-24 12:15:56.000000000 -0400 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 12:15:57.000000000 -0400 Modify: 2025-10-24 12:15:57.000000000 -0400 Change: 2025-10-24 12:15:57.000000000 -0400 Birth: 2025-10-24 12:15:56.000000000 -0400 PASS 54 (26s) == sanityn test 55a: rename vs unlink target dir ========= 12:16:05 (1761322565) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (9s) == sanityn test 55b: rename vs unlink source dir ========= 12:16:14 (1761322574) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (9s) == sanityn test 55c: rename vs unlink orphan target dir == 12:16:23 (1761322583) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (13s) == sanityn test 55d: rename file vs link ================= 12:16:36 (1761322596) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (11s) == sanityn test 60: Verify data_version behaviour ======== 12:16:47 (1761322607) PASS 60 (2s) == sanityn test 70a: cd directory && rm directory ======== 12:16:49 (1761322609) PASS 70a (3s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 12:16:52 (1761322612) PASS 70b (3s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 12:16:55 (1761322615) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00413897 s, 15.8 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (2s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 12:16:57 (1761322617) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0056786 s, 11.5 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 12:16:58 (1761322618) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 12:16:59 (1761322619) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00594718 s, 11.0 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (2s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 12:17:01 (1761322621) llite.lustre-ffffa0ec47e05000.xattr_cache=1 llite.lustre-ffffa0ec56019000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 12:17:03 (1761322623) llite.lustre-ffffa0ec47e05000.xattr_cache=1 llite.lustre-ffffa0ec56019000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (3s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 12:17:06 (1761322626) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0050523 s, 203 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00493188 s, 208 kB/s 252389: get lock2 252390: get lock1 252389: done 252389: get lock1 252390: done 252390: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 252390: cannot get lock2: Resource deadlock avoided 252390: exit rc=0 252389: done 252389: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 12:17:12 (1761322632) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0253391 s, 82.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121017 s, 86.6 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 252963 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (8s) == sanityn test 76: Verify MDT open_files listing ======== 12:17:20 (1761322640) opening files: ........ closing files: ........ PASS 76 (27s) == sanityn test 77a: check FIFO NRS policy =============== 12:17:48 (1761322668) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.160018 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356322 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0453956 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0576824 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0555789 s, 18.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0ec56019000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0952561 s, 11.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0955858 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117361 s, 8.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0949725 s, 11.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0914463 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0868269 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823615 s, 12.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0921348 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104568 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124607 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141031 s, 7.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0ec56019000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffffa0ec56019000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139125 s, 7.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256212 s, 40.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014623 s, 71.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225113 s, 46.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165251 s, 63.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699723 s, 150 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249249 s, 42.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314468 s, 33.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105771 s, 99.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803789 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302256 s, 34.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313008 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317731 s, 33.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0346728 s, 30.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275473 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253144 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113414 s, 92.5 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg343-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg343-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (3s) == sanityn test 77b: check CRR-N NRS policy ============== 12:17:51 (1761322671) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.14378 s, 117 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0615843 s, 17.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0712577 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0709428 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0797955 s, 13.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0731289 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0845275 s, 12.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0851156 s, 12.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10222 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103219 s, 10.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0720527 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0878303 s, 11.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0876727 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100012 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107043 s, 9.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0999346 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109522 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911052 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285934 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020303 s, 51.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140265 s, 74.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0361112 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02476 s, 42.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154767 s, 67.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163701 s, 64.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183494 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021836 s, 48.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.039621 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246754 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025852 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268727 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279671 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296991 s, 35.3 MB/s ldlm.namespaces.lustre-OST0001-osc-ffffa0ec56019000.lock_unused_count=1 removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg343-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg343-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.141119 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0354989 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0679453 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0642868 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0837565 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0688477 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0839114 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0762481 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107666 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0956158 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107153 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106718 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10173 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0967953 s, 10.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0985759 s, 10.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112055 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105767 s, 9.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0ec56019000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffffa0ec56019000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229505 s, 45.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267339 s, 39.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302198 s, 34.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287226 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210603 s, 49.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220457 s, 47.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273357 s, 38.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253246 s, 41.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285264 s, 36.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257306 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234602 s, 44.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235709 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303705 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250295 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281312 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276468 s, 37.9 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg343-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg343-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (5s) == sanityn test 77c: check ORR NRS policy ================ 12:17:56 (1761322676) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.147149 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0354749 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0470659 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0766319 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0808849 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0777409 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0777879 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0904603 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0659089 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104136 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0912024 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0690631 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0576858 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0817867 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0968856 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0734734 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108078 s, 9.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0ec56019000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffffa0ec56019000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201016 s, 52.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296753 s, 35.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257913 s, 40.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249886 s, 42.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247388 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277256 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205181 s, 51.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285935 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232019 s, 45.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229439 s, 45.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306377 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288841 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298278 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326768 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293275 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327178 s, 32.0 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg343-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg343-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.166891 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0500709 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.049191 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0616196 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0710751 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0759184 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0543346 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0638061 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0978798 s, 10.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11338 s, 9.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12844 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122961 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109209 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12661 s, 8.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133327 s, 7.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150795 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156052 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0433716 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0408323 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016251 s, 64.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205605 s, 51.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288725 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0360331 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024862 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025323 s, 41.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0596667 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0393604 s, 26.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.037627 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031241 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312159 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381862 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.03412 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0462352 s, 22.7 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg343-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg343-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.183825 s, 91.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0639615 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0767147 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0855796 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0373615 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0374126 s, 28.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823993 s, 12.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12817 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114189 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129671 s, 8.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149819 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140976 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133299 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137383 s, 7.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158363 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13616 s, 7.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffffa0ec56019000.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168838 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253457 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0411162 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0392893 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321428 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336752 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0380345 s, 27.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0385767 s, 27.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0377914 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0408105 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0411638 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0431778 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0422681 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0455896 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0470677 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0487899 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0553374 s, 18.9 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg343-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg343-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (7s) == sanityn test 77d: check TRR nrs policy ================ 12:18:03 (1761322683) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.175404 s, 95.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0441033 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0415531 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0571483 s, 18.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860935 s, 12.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823711 s, 12.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0741785 s, 14.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0799923 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0716185 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10728 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0957643 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108106 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116274 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100449 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101968 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108071 s, 9.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0ec56019000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffffa0ec56019000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110981 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159545 s, 65.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220293 s, 47.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294688 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0340223 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176112 s, 59.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316026 s, 33.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019832 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236258 s, 44.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244564 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273878 s, 38.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274972 s, 38.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261659 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248753 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314202 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.039944 s, 26.3 MB/s ldlm.namespaces.lustre-OST0001-osc-ffffa0ec56019000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338234 s, 31.0 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg343-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg343-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.186225 s, 90.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0591909 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0882126 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127015 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124707 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139627 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114728 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131863 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128689 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123463 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137106 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11114 s, 9.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121114 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12972 s, 8.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132707 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138981 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105285 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132442 s, 79.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254365 s, 41.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245242 s, 42.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204875 s, 51.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213828 s, 49.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225257 s, 46.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023626 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255431 s, 41.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246978 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234082 s, 44.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269826 s, 38.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274938 s, 38.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259506 s, 40.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249593 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285914 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290766 s, 36.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffffa0ec56019000.lock_unused_count=1 removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg343-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg343-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.154437 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0403615 s, 26.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0547464 s, 19.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0599386 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0590564 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0711084 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0610121 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0625576 s, 16.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0958773 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0743678 s, 14.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104637 s, 10.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0821259 s, 12.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0873256 s, 12.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.093086 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103786 s, 10.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111226 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107137 s, 9.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300786 s, 34.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235637 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211115 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255081 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221507 s, 47.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306512 s, 34.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309775 s, 33.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329471 s, 31.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183066 s, 57.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285523 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214031 s, 49.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224452 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229214 s, 45.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffffa0ec56019000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312846 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263123 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0432778 s, 24.2 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg343-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg343-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (7s) == sanityn test 77e: check TBF NID nrs policy ============ 12:18:10 (1761322690) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.203.43@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.136072 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328467 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197469 s, 53.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0559387 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0718228 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0523054 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0764672 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0554069 s, 18.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.071781 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0699562 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.072149 s, 14.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0810522 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0947279 s, 11.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0908373 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0737804 s, 14.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0627799 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0947737 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191284 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171446 s, 61.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195651 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214771 s, 48.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220406 s, 47.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226775 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224766 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341499 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265698 s, 39.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0383346 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261997 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316726 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0325943 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332908 s, 31.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287322 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0379803 s, 27.6 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg343-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg343-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.139341 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285064 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372176 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.056947 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0678732 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0539714 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0739487 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0885856 s, 11.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0740116 s, 14.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0781628 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0844261 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0845769 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0711171 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.063297 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0773983 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0769306 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.075079 s, 14.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170174 s, 61.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182507 s, 57.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014564 s, 72.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157451 s, 66.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250651 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148951 s, 70.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025115 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183901 s, 57.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682531 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248829 s, 42.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063672 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253059 s, 41.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231392 s, 45.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.004762 s, 220 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025039 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029942 s, 35.0 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg343-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg343-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.174544 s, 96.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242888 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280067 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0347362 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0421715 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.051159 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0664243 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0936384 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0964972 s, 10.9 MB/s ldlm.namespaces.lustre-OST0001-osc-ffffa0ec56019000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0755098 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0755196 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0919153 s, 11.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1073 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0869858 s, 12.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102458 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10274 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107052 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142027 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138151 s, 75.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181522 s, 57.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106541 s, 98.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163004 s, 64.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155216 s, 67.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130802 s, 80.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671359 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020107 s, 52.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00412506 s, 254 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662391 s, 158 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113901 s, 92.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00910405 s, 115 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119111 s, 88.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0046384 s, 226 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010369 s, 101 MB/s ldlm.namespaces.lustre-OST0001-osc-ffffa0ec56019000.lock_unused_count=1 removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg343-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg343-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.153059 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244156 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330342 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0346425 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0834706 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0931132 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0995531 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104263 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0872144 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0817064 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0765676 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100458 s, 10.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffffa0ec56019000.lock_unused_count=1 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0809693 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105838 s, 9.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111782 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0946176 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104563 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115096 s, 91.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01095 s, 95.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018253 s, 57.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136496 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013718 s, 76.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234825 s, 44.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156828 s, 66.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158951 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163989 s, 63.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00294302 s, 356 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183618 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284433 s, 36.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00453972 s, 231 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00404908 s, 259 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00309246 s, 339 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0028994 s, 362 MB/s ldlm.namespaces.lustre-OST0001-osc-ffffa0ec56019000.lock_unused_count=1 removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg343-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg343-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (9s) == sanityn test 77f: check TBF JobID nrs policy ========== 12:18:19 (1761322699) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg343-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.197967 s, 84.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0577208 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0591378 s, 17.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0489801 s, 21.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0635475 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0479743 s, 21.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0588469 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0674069 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0743108 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0522312 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0812293 s, 12.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0544762 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0775746 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0692011 s, 15.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0662934 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.067084 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0760408 s, 13.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139834 s, 75.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164047 s, 63.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180561 s, 58.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157002 s, 66.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185687 s, 56.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140001 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201492 s, 52.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281454 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239321 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214703 s, 48.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296378 s, 35.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232806 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213002 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299417 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260863 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0340484 s, 30.8 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg343-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg343-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg343-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.143914 s, 117 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028442 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0507887 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330803 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0601161 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.067429 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0758915 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0588045 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0525227 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0851982 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0977847 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140036 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124678 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142457 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159344 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153971 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156517 s, 6.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0ec56019000.lock_unused_count=4 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278878 s, 37.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180363 s, 58.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166417 s, 63.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190212 s, 55.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185733 s, 56.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297051 s, 35.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200802 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021867 s, 48.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221272 s, 47.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222886 s, 47.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228402 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024034 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245404 s, 42.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243115 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249648 s, 42.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263981 s, 39.7 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg343-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg343-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg343-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.193525 s, 86.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0590786 s, 17.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0849093 s, 12.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0681287 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0762643 s, 13.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100659 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0906595 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0963082 s, 10.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860167 s, 12.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100929 s, 10.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0966923 s, 10.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0977831 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0971462 s, 10.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11387 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103166 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11457 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140735 s, 7.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119259 s, 87.9 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261291 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166645 s, 62.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277845 s, 37.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02176 s, 48.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190832 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212553 s, 49.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223768 s, 46.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239476 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243044 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283003 s, 37.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259969 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260083 s, 40.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275916 s, 38.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275858 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285373 s, 36.7 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg343-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg343-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg343-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.199573 s, 84.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264211 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0674497 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0916627 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0942735 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0910172 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0965734 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0931242 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0774142 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.076699 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0825058 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0747258 s, 14.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0ec56019000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105512 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112012 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0835939 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0634411 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103725 s, 10.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0366438 s, 28.6 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0353551 s, 29.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323422 s, 32.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0439543 s, 23.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0430635 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0436295 s, 24.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0428107 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304611 s, 34.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0444853 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0408769 s, 25.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.037047 s, 28.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0384286 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0403475 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0425925 s, 24.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0389416 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0448697 s, 23.4 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg343-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg343-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (9s) == sanityn test 77g: Change TBF type directly ============ 12:18:28 (1761322708) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (5s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 12:18:33 (1761322713) oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg343-client: oleg343-server: ssh exited with exit code 2 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg343-client: oleg343-server: ssh exited with exit code 12 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg343-client: oleg343-server: ssh exited with exit code 12 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg343-client: oleg343-server: ssh exited with exit code 12 PASS 77h (4s) == sanityn test 77i: Change rank of TBF rule ============= 12:18:37 (1761322717) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (9s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 12:18:46 (1761322726) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.1461 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0489288 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0651794 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.052035 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0710163 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0642995 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0485339 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0786395 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109945 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.096825 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119964 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.09384 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105733 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11282 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134035 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0970714 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105603 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170865 s, 61.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025506 s, 41.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196353 s, 53.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203994 s, 51.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205069 s, 51.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021423 s, 48.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234731 s, 44.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230998 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258928 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263231 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268467 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280821 s, 37.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236863 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301166 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308205 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287387 s, 36.5 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg343-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg343-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.02407 s, 20.9 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4821 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (54s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 12:19:40 (1761322780) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg343-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.673204 s, 24.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17386 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174406 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195851 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18646 s, 5.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183896 s, 5.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206278 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197765 s, 5.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.337182 s, 3.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.336493 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.351048 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.323676 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.354336 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.371016 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.349102 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.37311 s, 2.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128676 s, 8.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130442 s, 8.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129828 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138061 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139287 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133378 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134592 s, 7.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149875 s, 7.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138751 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139418 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141128 s, 7.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140877 s, 7.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142406 s, 7.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142942 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143544 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145604 s, 7.2 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg343-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg343-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8439 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.0316 s, 5.2 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg343-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.831783 s, 20.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107558 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162614 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171678 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176914 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251043 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23018 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.226577 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.214567 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.246067 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.320725 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.32787 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.322689 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.418453 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.40947 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.42605 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.468451 s, 2.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267389 s, 39.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868605 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140854 s, 74.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00227851 s, 460 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0898593 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572832 s, 183 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00416066 s, 252 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993136 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00212721 s, 493 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140518 s, 7.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179746 s, 58.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1032 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0838053 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0461936 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0472761 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0760649 s, 13.8 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg343-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg343-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8879 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9094 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (143s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 12:22:03 (1761322923) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg343-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.264786 s, 63.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269826 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341305 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0455917 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0553529 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0612322 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0773013 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0951385 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120243 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125471 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134305 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122678 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107317 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112299 s, 9.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120392 s, 8.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149546 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140606 s, 7.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00354361 s, 296 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254377 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0656068 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0821274 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0758079 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0992249 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0947798 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0536591 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018829 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0865411 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0616487 s, 17.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0371751 s, 28.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309622 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0431745 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.060027 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0426793 s, 24.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg343-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg343-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.59895 s, 18.7 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.8537 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.43@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.43@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.215705 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0401537 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0556748 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10603 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127911 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141798 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13679 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16568 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13876 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151856 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154245 s, 6.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173854 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171615 s, 6.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140571 s, 7.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181986 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165657 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215374 s, 4.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0441781 s, 23.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0480271 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0498991 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0505146 s, 20.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0563264 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0386695 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0417503 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0358591 s, 29.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0472451 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.045652 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.041381 s, 25.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0433671 s, 24.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0441699 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0491009 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0444506 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0541484 s, 19.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg343-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg343-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.4431 s, 19.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.83197 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.43@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg343-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.227089 s, 73.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0357764 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0469336 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0541044 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100985 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0990321 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102731 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111172 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144851 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143839 s, 7.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147027 s, 7.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138444 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123749 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136274 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121291 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125652 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146783 s, 7.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151367 s, 69.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192914 s, 54.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276011 s, 38.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269798 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309914 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293775 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031711 s, 33.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239577 s, 43.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263585 s, 39.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266173 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282263 s, 37.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277156 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287055 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320386 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328691 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0361353 s, 29.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg343-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg343-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.70356 s, 18.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.97288 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg343-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.365581 s, 45.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0357515 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0536677 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0963215 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111347 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14398 s, 7.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145542 s, 7.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110163 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1479 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126844 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162721 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17395 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198066 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.342983 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.341261 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.420395 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.446368 s, 2.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994173 s, 105 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00415054 s, 253 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146991 s, 71.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292627 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203168 s, 51.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156413 s, 67.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178927 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185409 s, 56.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00407059 s, 258 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736131 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655575 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00453526 s, 231 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00415047 s, 253 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00423483 s, 248 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0386443 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00374992 s, 280 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg343-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg343-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.90781 s, 10.6 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.79214 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.15725 s, 20.3 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.82033 s, 10.7 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg343-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.471157 s, 35.6 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317676 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0426805 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0577705 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0767447 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0750629 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0780986 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0668843 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.059947 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.335104 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.347624 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.369009 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.376192 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.387708 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.378654 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.395434 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.376545 s, 2.8 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00437481 s, 240 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120318 s, 87.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202189 s, 51.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145993 s, 71.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156835 s, 66.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158005 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015953 s, 65.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233274 s, 45.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229603 s, 45.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213018 s, 49.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239423 s, 43.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278503 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00371974 s, 282 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261214 s, 40.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0385278 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0478554 s, 21.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg343-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg343-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6154 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4699 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg343-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.222271 s, 75.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301856 s, 34.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0417679 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0432574 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0652941 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0693347 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0693088 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0608515 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0733412 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.080754 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0963858 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0764287 s, 13.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0995578 s, 10.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0745667 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0804335 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0854725 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112449 s, 9.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100689 s, 104 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573763 s, 183 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281195 s, 37.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225415 s, 46.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273028 s, 38.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865042 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00511038 s, 205 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0333733 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160072 s, 65.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027393 s, 38.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00418713 s, 250 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0038168 s, 275 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0375263 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111553 s, 94.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013934 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336048 s, 31.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg343-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg343-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.0807 s, 20.6 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75967 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg343-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.279598 s, 60.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0409569 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0881405 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0727505 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.073088 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0889164 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0980206 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0941758 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.096438 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120958 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106844 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176302 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164343 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173145 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160403 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317136 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.325736 s, 3.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187799 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257665 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107447 s, 97.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00380411 s, 276 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356342 s, 29.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189717 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00445252 s, 236 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282247 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00480377 s, 218 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00342285 s, 306 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198676 s, 52.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187204 s, 56.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200481 s, 52.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00398777 s, 263 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00457205 s, 229 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597907 s, 175 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg343-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg343-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.8936 s, 10.6 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.77615 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.78729 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75839 s, 10.7 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (373s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 12:28:17 (1761323297) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (5s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 12:28:21 (1761323301) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg343-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.19852 s, 328 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg343-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.0345 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (51s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 12:29:12 (1761323352) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.181218 s, 92.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0441954 s, 23.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0539956 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0583439 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100803 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0924406 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103648 s, 10.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffffa0ec56019000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135767 s, 7.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101989 s, 10.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150462 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116349 s, 9.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141254 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119923 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161185 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13609 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149916 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173646 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00531204 s, 197 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0409346 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249281 s, 42.1 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffffa0ec56019000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281169 s, 37.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315917 s, 33.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.038442 s, 27.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0396747 s, 26.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0405901 s, 25.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0407239 s, 25.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332612 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286919 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0391908 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287416 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0558939 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0405603 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0026303 s, 399 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg343-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg343-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.24447 s, 20.0 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.95874 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.238139 s, 70.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268756 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0466657 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0879475 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10284 s, 10.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115322 s, 9.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104323 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12368 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12069 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119035 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.099958 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0973372 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113298 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104467 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113756 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.235626 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220859 s, 4.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251258 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182198 s, 57.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245201 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208633 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251478 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213355 s, 49.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0478004 s, 21.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294942 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322712 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315103 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303653 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0334629 s, 31.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332865 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356487 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0358822 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0385676 s, 27.2 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg343-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg343-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.09388 s, 20.6 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.96351 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (67s) == sanityn test 77o: Changing rank should not panic ====== 12:30:19 (1761323419) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (4s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 12:30:24 (1761323424) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (48s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 12:31:11 (1761323471) mds.MDS.mdt.nrs_policies=tbf oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg343-client: oleg343-server: ssh exited with exit code 36 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg343-client: oleg343-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (14s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 12:31:25 (1761323485) oleg343-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device pdsh@oleg343-client: oleg343-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (3s) == sanityn test 79: xattr: intent error ================== 12:31:29 (1761323489) File: /mnt/lustre/d79.sanityn Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505430 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 12:31:29.000000000 -0400 Modify: 2025-10-24 12:31:29.000000000 -0400 Change: 2025-10-24 12:31:29.000000000 -0400 Birth: 2025-10-24 12:31:28.000000000 -0400 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (13s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 12:31:42 (1761323502) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 12:31:43 (1761323503) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (2s) == sanityn test 81a: rename and stat under striped directory ========================================================== 12:31:45 (1761323505) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 12:31:46 (1761323506) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 12:31:48 (1761323508) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 12:31:49 (1761323509) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7123 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7123 PASS 82 (3s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 12:31:52 (1761323512) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (2s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 12:31:54 (1761323514) fail_loc=0x8000060b PASS 84 (8s) == sanityn test 90: open/create and unlink striped directory ========================================================== 12:32:02 (1761323522) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 12:32:03 (1761323523) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (2s) == sanityn test 92: create remote directory under orphan directory ========================================================== 12:32:05 (1761323525) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 12:32:06 (1761323526) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2628 0xa44 0 0 2642 0xa52 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2643 0xa53 0 1 2629 0xa45 0 rc1=0 and rc2=0 PASS 93 (11s) == sanityn test 94: signal vs CP callback race =========== 12:32:17 (1761323537) osc.lustre-OST0000-osc-ffffa0ec47e05000.idle_timeout=0 osc.lustre-OST0000-osc-ffffa0ec56019000.idle_timeout=0 osc.lustre-OST0001-osc-ffffa0ec47e05000.idle_timeout=0 osc.lustre-OST0001-osc-ffffa0ec56019000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0879284 s, 93.2 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 3.09727 s, 1.3 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 283291 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0254968 s, 161 kB/s osc.lustre-OST0000-osc-ffffa0ec47e05000.idle_timeout=debug osc.lustre-OST0000-osc-ffffa0ec56019000.idle_timeout=debug osc.lustre-OST0001-osc-ffffa0ec47e05000.idle_timeout=debug osc.lustre-OST0001-osc-ffffa0ec56019000.idle_timeout=debug PASS 94 (11s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 12:32:28 (1761323548) llite.lustre-ffffa0ec47e05000.fast_read=0 llite.lustre-ffffa0ec56019000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.00256317 s, 4.8 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffffa0ec47e05000.fast_read=1 llite.lustre-ffffa0ec56019000.fast_read=1 PASS 95b (9s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 12:32:37 (1761323557) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 12:32:38 (1761323558) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00271885 s, 1.5 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (3s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 12:32:41 (1761323561) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0282982 s, 74.1 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (3s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 12:32:44 (1761323564) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0358234 s, 58.5 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 12:32:46 (1761323566) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.00280968 s, 4.3 MB/s PASS 100e (3s) == sanityn test 101a: Discard DoM data on unlink ========= 12:32:49 (1761323569) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000617422 s, 6.6 MB/s PASS 101a (3s) == sanityn test 101b: Discard DoM data on rename ========= 12:32:52 (1761323572) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000695721 s, 5.9 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 12:32:54 (1761323574) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00060989 s, 6.7 MB/s PASS 101c (4s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 12:32:59 (1761323579) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1761323580 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1761323583 Finishing test test23 at 1761323583 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 12:33:07 (1761323587) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (2s) == sanityn test 105: Glimpse and lock cancel race ======== 12:33:08 (1761323588) fail_loc=0x416 PASS 105 (48s) == sanityn test 106a: Verify the btime via statx() ======= 12:33:56 (1761323636) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 12:33:58 (1761323638) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00397551 s, 258 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505458 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 12:33:58.000000000 -0400 Modify: 2025-10-24 12:33:58.000000000 -0400 Change: 2025-10-24 12:33:58.000000000 -0400 Birth: 2025-10-24 12:33:57.000000000 -0400 /mnt/lustre/f106b.sanityn 144115205272505458 -rw-r--r-- 1 0 0 1761323637 1761323638 1761323638 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505458 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 12:33:58.000000000 -0400 Modify: 2025-10-24 12:33:58.000000000 -0400 Change: 2025-10-24 12:33:58.000000000 -0400 Birth: 2025-10-24 12:33:57.000000000 -0400 1761323638 1024 11 PASS 106b (2s) == sanityn test 106c: Verify statx attributes mask ======= 12:34:00 (1761323640) PASS 106c (3s) == sanityn test 107a: Basic grouplock conflict =========== 12:34:03 (1761323643) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.0925315 s, 113 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2651 0xa5b 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7123 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7123 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 12:34:07 (1761323647) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.0862265 s, 122 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2634 0xa4a 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7123 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) == sanityn test 108a: lseek: parallel updates ============ 12:34:16 (1761323656) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.00194424 s, 5.1 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 4.33326 s, 1.9 MB/s PASS 108a (7s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 12:34:22 (1761323662) 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 192.168.203.143@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.143@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg343-client.virtnet /mnt/lustre (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg343-server' oleg343-server: oleg343-server.virtnet: executing load_modules_local oleg343-server: Loading modules from /home/green/git/lustre-release/lustre oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg343-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg343-server: detected 4 online CPUs by sysfs oleg343-server: Force libcfs to create 2 CPU partitions Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre Starting client: oleg343-client.virtnet: -o user_xattr,flock oleg343-server@tcp:/lustre /mnt/lustre2 PASS 109 (544s) == sanityn test 110: do not grant another lock on resend ========================================================== 12:43:26 (1761324206) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 12:43:27.000000000 -0400 Modify: 2025-10-24 12:43:27.000000000 -0400 Change: 2025-10-24 12:43:27.000000000 -0400 Birth: 2025-10-24 12:43:26.000000000 -0400 fail_loc=0 fail_val=0 Connected clients: oleg343-client.virtnet oleg343-client.virtnet PASS 110 (36s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 12:44:02 (1761324242) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 12:44:03 (1761324243) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (2s) == sanityn test 113: check servers of specified fs ======= 12:44:05 (1761324245) lustre-OST0000-osc-ffffa0ec47bb7800 active. lustre-OST0001-osc-ffffa0ec47bb7800 active. lustre-OST0000-osc-ffffa0ec47bb7800 active. lustre-OST0001-osc-ffffa0ec47bb7800 active. MGC192.168.203.143@tcp active. PASS 113 (2s) cleanup: ====================================================== == sanityn test complete, duration 5651 sec ============== 12:44:08 (1761324248) Stopping clients: oleg343-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg343-client.virtnet /mnt/lustre2 opts: