-----============= acceptance-small: sanityn ============----- Thu Jun 20 19:46:15 EDT 2024 excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 Started clients oleg237-client.virtnet: 192.168.202.137@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt) oleg237-client.virtnet: executing check_config_client /mnt/lustre oleg237-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg237-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800a9905800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b3a56800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800a9905800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b3a56800.idle_timeout=debug disable quota as required oleg237-server: oleg237-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129577 s, 80.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6872] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 19:46:24 (1718927184) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (2s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 19:46:26 (1718927186) -rw-r--r-- 1 root root 0 Jun 20 19:46 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (1s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 19:46:27 (1718927187) -rw-r--r-- 1 root root 0 Jun 20 19:46 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (2s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 19:46:29 (1718927189) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (1s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 19:46:30 (1718927190) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (2s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 19:46:32 (1718927192) total 1 -rwxrwxrwx 1 root root 0 Jun 20 19:46 f2a -rwxrwxrwx 1 root root 0 Jun 20 19:46 f2b total 1 -rwxrwxrwx 1 root root 0 Jun 20 19:46 f2a -rwxrwxrwx 1 root root 0 Jun 20 19:46 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00402726 s, 127 kB/s PASS 2e (1s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 19:46:33 (1718927193) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 19:46:34 (1718927194) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.093704 s, 22.4 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (2s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 19:46:36 (1718927196) PASS 3 (1s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 19:46:37 (1718927197) PASS 4 (3s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 19:46:40 (1718927200) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (1s) == sanityn test 6: remove of open file on other node ============================================================================ 19:46:41 (1718927201) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (1s) == sanityn test 7: remove of open directory on other node ======================================================================= 19:46:42 (1718927202) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (2s) == sanityn test 8: remove of open special file on other node ==================================================================== 19:46:44 (1718927204) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (1s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 19:46:45 (1718927205) PASS 9a (1s) == sanityn test 9b: append to striped sparse file ======== 19:46:46 (1718927206) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116696 s, 89.9 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0099793 s, 0.3 kB/s Data read (expecting 'foo'): foo PASS 9b (2s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 19:46:48 (1718927208) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00114667 s, 0.9 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00158971 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00127943 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00123547 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00120552 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00128085 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00122608 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00123415 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00120665 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00139011 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00124637 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00141529 s, 0.7 kB/s PASS 10a (1s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 19:46:49 (1718927209) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00268228 s, 1.1 MB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00646509 s, 634 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000183886 s, 16.7 MB/s PASS 10b (1s) == sanityn test 11: execution of file opened for write should return error ============================================================== 19:46:50 (1718927210) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6872 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (2s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 19:46:52 (1718927212) warning: '-runtime' deprecated, use '-t runtime' instead - link 3350 (time 1718927222.84 total 10.00 last 334.93) - link 6598 (time 1718927232.84 total 20.00 last 324.72) - link 9851 (time 1718927242.84 total 30.01 last 325.20) - link 13101 (time 1718927252.84 total 40.01 last 324.97) - link 16302 (time 1718927262.85 total 50.01 last 320.04) - link 19134 (time 1718927272.85 total 60.01 last 283.20) - link 20000 (time 1718927275.95 total 63.11 last 279.11) - link 22842 (time 1718927285.95 total 73.11 last 284.16) - link 25716 (time 1718927295.95 total 83.12 last 287.38) - link 28188 (time 1718927305.95 total 93.12 last 247.16) - link 30000 (time 1718927312.28 total 99.44 last 286.55) total: 30167 link in 100.00 seconds: 301.66 ops/second using seed 2036841795 running for 100 seconds - stat 10000 (time 1718927216 ; total 3 ; last 3) - stat 20000 (time 1718927219 ; total 6 ; last 3) - stat 30000 (time 1718927222 ; total 9 ; last 3) - stat 40000 (time 1718927224 ; total 11 ; last 2) - stat 50000 (time 1718927227 ; total 14 ; last 3) - stat 60000 (time 1718927230 ; total 17 ; last 3) - stat 70000 (time 1718927232 ; total 19 ; last 2) - stat 80000 (time 1718927235 ; total 22 ; last 3) - stat 90000 (time 1718927237 ; total 24 ; last 2) - stat 100000 (time 1718927240 ; total 27 ; last 3) - stat 110000 (time 1718927243 ; total 30 ; last 3) - stat 120000 (time 1718927245 ; total 32 ; last 2) - stat 130000 (time 1718927248 ; total 35 ; last 3) - stat 140000 (time 1718927251 ; total 38 ; last 3) - stat 150000 (time 1718927253 ; total 40 ; last 2) - stat 160000 (time 1718927256 ; total 43 ; last 3) - stat 170000 (time 1718927258 ; total 45 ; last 2) - stat 180000 (time 1718927261 ; total 48 ; last 3) - stat 190000 (time 1718927264 ; total 51 ; last 3) - stat 200000 (time 1718927270 ; total 57 ; last 6) - stat 210000 (time 1718927276 ; total 63 ; last 6) - stat 220000 (time 1718927282 ; total 69 ; last 6) - stat 230000 (time 1718927289 ; total 76 ; last 7) - stat 240000 (time 1718927295 ; total 82 ; last 6) - stat 250000 (time 1718927301 ; total 88 ; last 6) - stat 260000 (time 1718927307 ; total 94 ; last 6) total: 269271 stats in 100 seconds: 2692.709961 stats/second - unlinked 0 (time 1718927263 ; total 0 ; last 0) - unlinked 10000 (time 1718927315 ; total 52 ; last 52) - unlinked 20000 (time 1718927343 ; total 80 ; last 28) - unlinked 30000 (time 1718927368 ; total 105 ; last 25) unlink(/mnt/lustre2/lockdir/lockfile30167) error: No such file or directory total: 30166 unlinks in 105 seconds: 287.295227 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (20463) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (20466) - No such process PASS 12 (158s) == sanityn test 13: test directory page revocation ======= 19:49:30 (1718927370) f13 f13 PASS 13 (1s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 19:49:31 (1718927371) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6872 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4261: 21854 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 19:49:33 (1718927373) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4261: 22482 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (1s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 19:49:34 (1718927374) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 23108 Terminated $DIR1/$tdir/sleep 60 PASS 14b (2s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 19:49:36 (1718927376) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 23736 Terminated $DIR1/$tdir/sleep 60 PASS 14c (1s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 19:49:37 (1718927377) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 24365 Terminated $DIR1/$tdir/sleep 60 PASS 14d (2s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 19:49:39 (1718927379) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7517184 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7517184kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (10s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 19:49:49 (1718927389) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0818891 s, 25.6 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0744222 s, 28.2 MB/s Chance of close/open is 1 in 50 Seed set to 7390 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x6bb3bd 000004[0] 1718927391.033878 trunc from 00000000 to 0x6bb3bc (0x6bb3bd bytes) 000006[0] 1718927391.046296 trunc from 0x6bb3bd to 0x0f571c (0x5c5ca0 bytes) 000007[0] 1718927391.058780 read 0x0e5a4c thru 0x0e9359 (0x390e bytes) 000009[1] 1718927391.066621 read 0x0a0687 thru 0x0aa8bc (0xa236 bytes) 000010[0] 1718927391.077470 read 0x09423b thru 0x0a2150 (0xdf16 bytes) 000011[0] 1718927391.093691 mapwrite 0x82166b thru 0x825eda (0x4870 bytes) 000013[1] 1718927391.207795 write 0x7e1a2f thru 0x7ea874 (0x8e46 bytes) 000014[1] 1718927391.223500 mapwrite 0x9c1695 thru 0x9c2e2f (0x179b bytes) 000016[1] 1718927391.263546 write 0x39ad4d thru 0x3a83bf (0xd673 bytes) 000017[0] 1718927391.268461 read 0x83af52 thru 0x83f78d (0x483c bytes) truncating to largest ever: 0x92ecae 000018[0] 1718927391.284897 trunc from 0x9c2e30 to 0x92ecad (0x94182 bytes) 000019[0] 1718927391.302320 trunc from 0x92ecae to 0x5da9be (0x3542ef bytes) 000020[1] 1718927391.312996 write 0x93f5cf thru 0x942c5c (0x368e bytes) 000021[0] 1718927391.323072 mapread 0x21c03b thru 0x228765 (0xc72b bytes) 000022[1] 1718927391.344475 write 0x786f62 thru 0x7959f4 (0xea93 bytes) 000023[1] 1718927391.351552 mapread 0x889954 thru 0x88a7e3 (0xe90 bytes) 000024[0] 1718927391.360783 read 0x385c8d thru 0x38f845 (0x9bb9 bytes) 000025[1] 1718927391.384704 write 0x5febdc thru 0x60b9b7 (0xcddc bytes) 000026[0] 1718927391.397650 mapwrite 0x2c128a thru 0x2cedb2 (0xdb29 bytes) 000027[1] 1718927391.438671 write 0x82e99c thru 0x835f4e (0x75b3 bytes) 000029[1] 1718927391.447484 mapwrite 0x1818cc thru 0x1828d1 (0x1006 bytes) 000030[1] 1718927391.482711 trunc from 0x942c5d to 0x880654 (0xc2608 bytes) 000031[0] 1718927391.491053 mapread 0x46641c thru 0x47437a (0xdf5f bytes) 000036[1] 1718927391.549321 write 0x3b57a1 thru 0x3b8715 (0x2f75 bytes) 000038[0] 1718927391.553017 mapread 0x771693 thru 0x77a614 (0x8f82 bytes) 000039[0] 1718927391.579384 mapwrite 0x3127aa thru 0x3149d3 (0x222a bytes) 000041[1] 1718927391.610152 mapread 0x0681ed thru 0x071702 (0x9516 bytes) 000042[0] 1718927391.626903 mapread 0x81be27 thru 0x81bf4b (0x125 bytes) 000044[1] 1718927391.634071 mapwrite 0x60689f thru 0x60d2ca (0x6a2c bytes) 000047[1] 1718927391.692666 mapread 0x6fe2e4 thru 0x707746 (0x9463 bytes) 000049[1] 1718927391.709575 mapwrite 0x22177a thru 0x2227e9 (0x1070 bytes) 000050[1] 1718927391.730542 read 0x69ee24 thru 0x6a99ca (0xaba7 bytes) 000051[1] 1718927391.741656 mapread 0x4822e9 thru 0x4889e8 (0x6700 bytes) 000053[0] 1718927391.760058 write 0x87c167 thru 0x88a107 (0xdfa1 bytes) 000055[0] 1718927391.766342 mapwrite 0x1a3579 thru 0x1a8e68 (0x58f0 bytes) 000058[0] 1718927391.811508 trunc from 0x88a108 to 0x4930fc (0x3f700b bytes) 000059[0] 1718927391.818030 write 0x7c4358 thru 0x7d3f86 (0xfc2f bytes) 000060[0] 1718927391.819559 write 0x63822e thru 0x6430f5 (0xaec8 bytes) 000064[0] 1718927391.836761 mapread 0x74782a thru 0x748ec2 (0x1699 bytes) 000065[1] 1718927391.840562 read 0x0fbf11 thru 0x0fe8ee (0x29de bytes) 000066[0] 1718927391.850646 mapwrite 0x8f05ac thru 0x8f6e05 (0x685a bytes) 000068[0] 1718927391.895242 mapread 0x3d4a2f thru 0x3e4691 (0xfc63 bytes) 000071[0] 1718927391.944970 mapwrite 0x51d410 thru 0x524745 (0x7336 bytes) 000075[1] 1718927392.056263 write 0x028dde thru 0x02d4b5 (0x46d8 bytes) 000076[1] 1718927392.067175 read 0x375573 thru 0x37796e (0x23fc bytes) 000077[1] 1718927392.094922 read 0x725679 thru 0x72c778 (0x7100 bytes) 000081[1] 1718927392.108686 read 0x4c7b35 thru 0x4c8baf (0x107b bytes) 000082[1] 1718927392.111804 trunc from 0x8f6e06 to 0x5580c2 (0x39ed43 bytes) 000083[1] 1718927392.122475 read 0x0e14f3 thru 0x0ec1f2 (0xad00 bytes) 000087[0] 1718927392.147772 write 0x663252 thru 0x670efa (0xdca9 bytes) 000089[1] 1718927392.161677 trunc from 0x670efb to 0x2d556d (0x39b98d bytes) 000090[1] 1718927392.172599 mapread 0x211b66 thru 0x2196e4 (0x7b7f bytes) 000092[0] 1718927392.197660 write 0x3e895d thru 0x3f6e0e (0xe4b2 bytes) 000093[1] 1718927392.205788 mapread 0x2b6399 thru 0x2be99a (0x8602 bytes) 000094[1] 1718927392.222109 read 0x3d5ebe thru 0x3e451d (0xe660 bytes) 000097[1] 1718927392.240774 mapread 0x00ae9b thru 0x0151a1 (0xa307 bytes) 000098[1] 1718927392.258413 trunc from 0x3f6e0f to 0x1d41f9 (0x222c15 bytes) 000100[1] 1718927392.265645 mapwrite 0x22adc0 thru 0x2315e9 (0x682a bytes) 000101[1] 1718927392.293865 mapread 0x1653d5 thru 0x16f425 (0xa051 bytes) 000102[0] 1718927392.306431 mapwrite 0x8b08b5 thru 0x8ba6bf (0x9e0b bytes) 000103[0] 1718927392.345890 read 0x7e43b0 thru 0x7eb47e (0x70cf bytes) 000105[1] 1718927392.355452 read 0x1f8e63 thru 0x2076c7 (0xe865 bytes) 000106[0] 1718927392.388730 mapread 0x4eb62f thru 0x4f361f (0x7ff1 bytes) 000107[1] 1718927392.406632 trunc from 0x8ba6c0 to 0x791633 (0x12908c bytes) 000108[1] 1718927392.434197 write 0x2d0856 thru 0x2df123 (0xe8ce bytes) 000111[0] 1718927392.440906 read 0x4aac29 thru 0x4b37da (0x8bb2 bytes) 000113[0] 1718927392.455693 trunc from 0x791634 to 0x5563f2 (0x23b241 bytes) 000114[1] 1718927392.473830 read 0x3e3e7f thru 0x3ef34c (0xb4ce bytes) 000115[1] 1718927392.484537 mapwrite 0x515368 thru 0x523b2a (0xe7c3 bytes) 000116[0] 1718927392.545302 read 0x0f9f1e thru 0x0fdfc6 (0x40a9 bytes) 000117[1] 1718927392.552691 read 0x1c808c thru 0x1d651e (0xe493 bytes) 000118[0] 1718927392.560460 write 0x52aac4 thru 0x536fab (0xc4e8 bytes) 000121[1] 1718927392.596197 write 0x4ce7d5 thru 0x4dd60a (0xee36 bytes) 000125[0] 1718927392.627144 read 0x36ba22 thru 0x37a6b3 (0xec92 bytes) 000126[1] 1718927392.630979 mapread 0x432354 thru 0x438bf9 (0x68a6 bytes) 000127[1] 1718927392.644612 trunc from 0x5563f3 to 0x5ee99a (0x985a8 bytes) 000130[1] 1718927392.662294 mapread 0x5183c1 thru 0x526423 (0xe063 bytes) 000132[0] 1718927392.677239 mapread 0x28f78a thru 0x29114d (0x19c4 bytes) 000136[0] 1718927392.703967 trunc from 0x5ee99b to 0x04e260 (0x5a073a bytes) 000137[1] 1718927392.714409 write 0x786784 thru 0x795efd (0xf77a bytes) 000138[0] 1718927392.722182 trunc from 0x795efe to 0x8e8f04 (0x153007 bytes) 000140[1] 1718927392.731789 mapwrite 0x83814e thru 0x842f24 (0xadd7 bytes) 000142[1] 1718927392.775862 read 0x866ed4 thru 0x86c449 (0x5576 bytes) 000143[1] 1718927392.786340 trunc from 0x8e8f05 to 0x7c2c12 (0x1262f2 bytes) 000144[0] 1718927392.798625 write 0x15af3b thru 0x15e5c1 (0x3687 bytes) 000145[1] 1718927392.807445 mapwrite 0x087d03 thru 0x090ba1 (0x8e9f bytes) 000146[1] 1718927392.855944 trunc from 0x7c2c13 to 0x2ddb1a (0x4e50f8 bytes) 000149[0] 1718927392.878201 mapread 0x0e2a47 thru 0x0efe91 (0xd44b bytes) 000151[1] 1718927392.901277 read 0x1185ac thru 0x12172b (0x9180 bytes) 000152[0] 1718927392.910994 mapwrite 0x38e51e thru 0x39d023 (0xeb06 bytes) 000154[1] 1718927392.967531 mapread 0x0e7217 thru 0x0eed14 (0x7afe bytes) 000155[1] 1718927392.990738 read 0x2ad8cf thru 0x2b90d2 (0xb804 bytes) 000156[1] 1718927392.991044 write 0x4f5256 thru 0x5012b1 (0xc05c bytes) 000158[0] 1718927393.002416 read 0x44fe27 thru 0x457ce2 (0x7ebc bytes) 000162[1] 1718927393.025032 read 0x0a7fa2 thru 0x0b76dc (0xf73b bytes) 000163[1] 1718927393.043990 trunc from 0x5012b2 to 0x77aadb (0x27982a bytes) 000165[0] 1718927393.058642 trunc from 0x77aadc to 0x6a582b (0xd52b0 bytes) 000166[1] 1718927393.071676 read 0x5231e4 thru 0x52e996 (0xb7b3 bytes) 000167[1] 1718927393.083409 mapread 0x0a76b7 thru 0x0a9e70 (0x27ba bytes) 000171[0] 1718927393.099296 mapread 0x28133b thru 0x2843f1 (0x30b7 bytes) 000173[0] 1718927393.112925 write 0x288250 thru 0x28f556 (0x7307 bytes) 000174[1] 1718927393.119148 write 0x3b1955 thru 0x3b44b0 (0x2b5c bytes) 000175[1] 1718927393.128350 mapread 0x44c1af thru 0x44e47a (0x22cc bytes) 000176[1] 1718927393.137947 mapwrite 0x7ea19b thru 0x7f3943 (0x97a9 bytes) 000178[1] 1718927393.179591 read 0x04ae04 thru 0x050cc9 (0x5ec6 bytes) 000179[0] 1718927393.212280 write 0x27a83c thru 0x28721d (0xc9e2 bytes) 000182[0] 1718927393.232795 mapwrite 0x03039b thru 0x03ebbc (0xe822 bytes) 000184[0] 1718927393.265764 trunc from 0x7f3944 to 0x8bcaa1 (0xc915e bytes) 000185[1] 1718927393.278688 write 0x5f6328 thru 0x6025a1 (0xc27a bytes) 000187[1] 1718927393.299245 write 0x048093 thru 0x0537da (0xb748 bytes) 000189[0] 1718927393.306987 mapwrite 0x724a3e thru 0x72c69b (0x7c5e bytes) 000190[0] 1718927393.343727 mapread 0x43f845 thru 0x446b7d (0x7339 bytes) 000191[1] 1718927393.366824 write 0x4d3aca thru 0x4de82c (0xad63 bytes) 000195[0] 1718927393.385494 mapwrite 0x85787e thru 0x8663e4 (0xeb67 bytes) 000196[0] 1718927393.434447 trunc from 0x8bcaa2 to 0x2256fb (0x6973a6 bytes) 000197[1] 1718927393.442597 mapread 0x0ee5c4 thru 0x0f8e95 (0xa8d2 bytes) 000199[1] 1718927393.477164 write 0x3e4626 thru 0x3ed148 (0x8b23 bytes) 000200[1] 1718927393.485028 trunc from 0x3ed149 to 0x0351c6 (0x3b7f82 bytes) 000202[1] 1718927393.502620 mapread 0x0070a0 thru 0x00cb56 (0x5ab7 bytes) 000203[0] 1718927393.510659 mapwrite 0x8f398d thru 0x8f5fc3 (0x2637 bytes) 000204[0] 1718927393.538140 read 0x3e145d thru 0x3e7ebd (0x6a61 bytes) 000205[0] 1718927393.558456 mapwrite 0x7cfd9c thru 0x7d3bae (0x3e13 bytes) 000207[0] 1718927393.574949 write 0x3116b5 thru 0x317afd (0x6449 bytes) 000209[1] 1718927393.584372 mapwrite 0x41f759 thru 0x42d740 (0xdfe8 bytes) 000210[0] 1718927393.629616 write 0x705381 thru 0x705a12 (0x692 bytes) 000211[1] 1718927393.633261 read 0x1c489c thru 0x1d192c (0xd091 bytes) 000212[0] 1718927393.671517 mapwrite 0x0220f8 thru 0x02bcd1 (0x9bda bytes) 000213[0] 1718927393.708175 mapwrite 0x727d44 thru 0x72a2ab (0x2568 bytes) 000215[1] 1718927393.732341 read 0x5c905b thru 0x5cade7 (0x1d8d bytes) 000216[1] 1718927393.738782 mapread 0x3f476f thru 0x3fd7b3 (0x9045 bytes) 000219[0] 1718927393.753972 read 0x798586 thru 0x7a0026 (0x7aa1 bytes) 000222[0] 1718927393.770149 read 0x725c06 thru 0x733f7a (0xe375 bytes) 000223[1] 1718927393.779969 trunc from 0x8f5fc4 to 0x501f13 (0x3f40b0 bytes) 000224[1] 1718927393.793563 mapwrite 0x687cb9 thru 0x6935cb (0xb913 bytes) 000225[0] 1718927393.822981 write 0x4f542c thru 0x502377 (0xcf4c bytes) 000230[0] 1718927393.844977 trunc from 0x6935cc to 0x08db2b (0x605aa0 bytes) 000235[1] 1718927393.857863 trunc from 0x08db2c to 0x5a0818 (0x512ced bytes) 000236[1] 1718927393.867761 mapread 0x2129ee thru 0x21f1eb (0xc7fe bytes) 000237[1] 1718927393.887343 read 0x1a588b thru 0x1ac6b0 (0x6e26 bytes) 000239[1] 1718927393.905091 mapwrite 0x772c35 thru 0x773bbd (0xf89 bytes) 000240[1] 1718927393.935515 write 0x5d6dcc thru 0x5e1b01 (0xad36 bytes) 000241[0] 1718927393.936087 mapread 0x528fab thru 0x536def (0xde45 bytes) 000242[0] 1718927393.961547 mapread 0x69f54b thru 0x6a4ab0 (0x5566 bytes) 000246[1] 1718927393.977506 trunc from 0x773bbe to 0x466067 (0x30db56 bytes) 000250[1] 1718927393.997800 mapwrite 0x6b2c41 thru 0x6ba5b3 (0x7973 bytes) 000251[0] 1718927394.034467 mapread 0x4e7129 thru 0x4ebd87 (0x4c5f bytes) 000252[0] 1718927394.048188 mapread 0x1ee570 thru 0x1f92fc (0xad8d bytes) 000256[0] 1718927394.071518 write 0x58154b thru 0x58ca92 (0xb548 bytes) 000258[1] 1718927394.077314 mapread 0x3a68b3 thru 0x3a97b0 (0x2efe bytes) 000261[0] 1718927394.092075 trunc from 0x6ba5b4 to 0x129ed8 (0x5906db bytes) 000264[1] 1718927394.111868 write 0x73225d thru 0x73cb5a (0xa8fe bytes) 000265[0] 1718927394.118613 write 0x6a1e4d thru 0x6a644e (0x4602 bytes) 000266[1] 1718927394.122039 read 0x39ce6d thru 0x3a0209 (0x339d bytes) 000267[0] 1718927394.127996 trunc from 0x73cb5b to 0x51c223 (0x220937 bytes) 000269[0] 1718927394.142226 mapread 0x396778 thru 0x39df21 (0x77aa bytes) 000270[1] 1718927394.159527 read 0x26a864 thru 0x26c358 (0x1af5 bytes) 000272[1] 1718927394.168732 write 0x900e0f thru 0x904df7 (0x3fe9 bytes) 000273[1] 1718927394.172664 mapwrite 0x5631b2 thru 0x572f9e (0xfded bytes) 000275[1] 1718927394.202968 mapread 0x02c734 thru 0x03a435 (0xdd02 bytes) 000277[0] 1718927394.219926 mapwrite 0x7920d7 thru 0x79f0bf (0xcfe9 bytes) 000278[0] 1718927394.258526 read 0x269ba6 thru 0x270d51 (0x71ac bytes) 000280[1] 1718927394.277835 mapwrite 0x3f5172 thru 0x40102f (0xbebe bytes) 000285[1] 1718927394.335708 read 0x12c0b9 thru 0x138700 (0xc648 bytes) 000286[1] 1718927394.353760 trunc from 0x904df8 to 0x7c9a95 (0x13b362 bytes) 000288[1] 1718927394.363370 read 0x165d51 thru 0x175374 (0xf624 bytes) 000289[0] 1718927394.366674 write 0x226648 thru 0x230c9b (0xa654 bytes) 000290[0] 1718927394.378234 read 0x6892ef thru 0x693f5b (0xac6d bytes) 000292[0] 1718927394.390548 trunc from 0x7c9a96 to 0x90b61d (0x141b88 bytes) 000294[0] 1718927394.406361 read 0x06d62a thru 0x073145 (0x5b1c bytes) 000296[1] 1718927394.443685 read 0x0724ef thru 0x081501 (0xf013 bytes) 000298[1] 1718927394.465464 write 0x8997c2 thru 0x8a7147 (0xd986 bytes) 000300[1] 1718927394.483783 write 0x42b455 thru 0x42f676 (0x4222 bytes) 000301[1] 1718927394.491942 mapwrite 0x99f869 thru 0x9a6f82 (0x771a bytes) 000304[1] 1718927394.553828 write 0x412daf thru 0x4133b3 (0x605 bytes) 000306[1] 1718927394.554379 mapread 0x4e0ac3 thru 0x4e8517 (0x7a55 bytes) 000307[1] 1718927394.554882 trunc from 0x9a6f83 to 0x4a2e95 (0x5040ed bytes) 000308[1] 1718927394.569002 trunc from 0x4a2e96 to 0x2154aa (0x28d9eb bytes) 000310[1] 1718927394.583146 trunc from 0x2154ab to 0x1c129f (0x5420b bytes) 000313[0] 1718927394.593534 mapwrite 0x70c744 thru 0x70fae9 (0x33a6 bytes) 000314[0] 1718927394.628982 mapread 0x615077 thru 0x6205c3 (0xb54d bytes) truncating to largest ever: 0x9ea51e 000318[1] 1718927394.657499 trunc from 0x70faea to 0x9ea51d (0x2daa34 bytes) 000319[0] 1718927394.668850 write 0x9f4579 thru 0x9fffff (0xba87 bytes) 000320[0] 1718927394.676248 write 0x5a7b8e thru 0x5b37e4 (0xbc57 bytes) 000321[1] 1718927394.690813 write 0x5cc12e thru 0x5da6f8 (0xe5cb bytes) 000322[1] 1718927394.705132 read 0x327919 thru 0x33765a (0xfd42 bytes) 000325[0] 1718927394.731832 trunc from 0xa00000 to 0x387e50 (0x6781af bytes) 000326[1] 1718927394.750517 write 0x5c0c48 thru 0x5c7567 (0x6920 bytes) 000327[1] 1718927394.758052 trunc from 0x5c7568 to 0x3e13bd (0x1e61aa bytes) 000328[0] 1718927394.767369 write 0x2c8a49 thru 0x2c9aa9 (0x1061 bytes) 000331[0] 1718927394.783931 mapread 0x07ed82 thru 0x07f531 (0x7b0 bytes) 000332[0] 1718927394.791978 trunc from 0x3e13be to 0x6c4587 (0x2e31ca bytes) 000333[0] 1718927394.801809 read 0x051e24 thru 0x05df4c (0xc129 bytes) 000334[0] 1718927394.824755 write 0x0b4e5c thru 0x0be598 (0x973d bytes) 000335[1] 1718927394.830506 mapwrite 0x120227 thru 0x12f729 (0xf503 bytes) 000336[1] 1718927394.881967 mapwrite 0x155040 thru 0x157686 (0x2647 bytes) 000340[0] 1718927394.916587 mapwrite 0x2adb12 thru 0x2aebb0 (0x109f bytes) 000341[0] 1718927394.936855 read 0x0c43e6 thru 0x0cc1b1 (0x7dcc bytes) 000342[0] 1718927394.940328 mapwrite 0x061b77 thru 0x062ee9 (0x1373 bytes) 000343[0] 1718927394.962732 mapwrite 0x999592 thru 0x9a02ee (0x6d5d bytes) 000345[1] 1718927395.006107 read 0x7a09ff thru 0x7b0445 (0xfa47 bytes) 000346[1] 1718927395.014522 read 0x17d85b thru 0x18bead (0xe653 bytes) 000347[0] 1718927395.019154 mapread 0x23aa6b thru 0x23e686 (0x3c1c bytes) 000349[1] 1718927395.019688 mapwrite 0x7d1162 thru 0x7d6eb5 (0x5d54 bytes) 000351[0] 1718927395.068111 trunc from 0x9a02ef to 0x7ae541 (0x1f1dad bytes) 000354[1] 1718927395.086705 mapread 0x6923da thru 0x6924b2 (0xd9 bytes) 000355[0] 1718927395.100520 write 0x38ac99 thru 0x3998d4 (0xec3c bytes) 000356[1] 1718927395.107851 mapwrite 0x89ac69 thru 0x89dd07 (0x309f bytes) 000359[1] 1718927395.149038 mapwrite 0x11347f thru 0x11a7a0 (0x7322 bytes) 000360[0] 1718927395.182900 write 0x7be8b1 thru 0x7bf307 (0xa57 bytes) 000364[1] 1718927395.203892 trunc from 0x89dd08 to 0x848e71 (0x54e96 bytes) 000368[0] 1718927395.228092 mapread 0x6dbd2f thru 0x6e1e9b (0x616d bytes) 000369[0] 1718927395.242380 trunc from 0x848e72 to 0x03e8e3 (0x80a58e bytes) 000370[1] 1718927395.251353 mapwrite 0x5dc3ba thru 0x5df20e (0x2e55 bytes) 000372[1] 1718927395.289052 mapread 0x227308 thru 0x22745b (0x154 bytes) 000376[0] 1718927395.298251 write 0x8a68dd thru 0x8b0a83 (0xa1a7 bytes) 000379[0] 1718927395.320104 mapread 0x65aa49 thru 0x65c19a (0x1752 bytes) 000381[0] 1718927395.334150 mapread 0x2a4237 thru 0x2af9ec (0xb7b6 bytes) 000383[0] 1718927395.361352 mapread 0x647e9c thru 0x657c65 (0xfdca bytes) 000385[0] 1718927395.385784 trunc from 0x8b0a84 to 0x769fae (0x146ad5 bytes) 000386[0] 1718927395.392700 read 0x257f67 thru 0x25f9ce (0x7a68 bytes) 000389[1] 1718927395.419265 mapread 0x2bfa66 thru 0x2ce5a3 (0xeb3e bytes) 000392[1] 1718927395.452524 write 0x5c5411 thru 0x5d2f8e (0xdb7e bytes) 000393[1] 1718927395.463874 mapread 0x43967d thru 0x445e7c (0xc800 bytes) 000395[1] 1718927395.478057 mapwrite 0x25e56f thru 0x2619b7 (0x3449 bytes) 000396[0] 1718927395.503411 write 0x54cfb2 thru 0x55a6f5 (0xd744 bytes) 000397[0] 1718927395.512814 trunc from 0x769faf to 0x9ba00f (0x250061 bytes) 000398[0] 1718927395.524270 write 0x74f55a thru 0x7571e9 (0x7c90 bytes) 000400[0] 1718927395.540637 trunc from 0x9ba010 to 0x0bf170 (0x8fae9f bytes) 000401[0] 1718927395.548022 trunc from 0x0bf171 to 0x31ef06 (0x25fd96 bytes) 000403[1] 1718927395.551698 trunc from 0x31ef07 to 0x95f74e (0x640848 bytes) 000404[0] 1718927395.560318 write 0x3e63e5 thru 0x3eb046 (0x4c62 bytes) 000406[1] 1718927395.572583 mapread 0x1ee1cf thru 0x1f0490 (0x22c2 bytes) 000407[0] 1718927395.584042 trunc from 0x95f74f to 0x0c92e3 (0x89646b bytes) 000408[0] 1718927395.597160 read 0x0542e4 thru 0x061818 (0xd535 bytes) 000409[1] 1718927395.605585 mapread 0x070f54 thru 0x07bee6 (0xaf93 bytes) 000410[1] 1718927395.627070 mapwrite 0x6522c9 thru 0x655b35 (0x386d bytes) 000412[0] 1718927395.661599 mapwrite 0x83cecf thru 0x84119f (0x42d1 bytes) 000413[0] 1718927395.709821 write 0x5d1135 thru 0x5d9c88 (0x8b54 bytes) 000415[0] 1718927395.719341 read 0x0ddc2b thru 0x0e5fec (0x83c2 bytes) 000417[1] 1718927395.719972 mapread 0x46fc2b thru 0x478abb (0x8e91 bytes) 000418[1] 1718927395.746080 mapwrite 0x027cd4 thru 0x027d27 (0x54 bytes) 000420[1] 1718927395.769779 mapread 0x201e4d thru 0x202e2e (0xfe2 bytes) 000424[0] 1718927395.787821 mapwrite 0x766de8 thru 0x76df46 (0x715f bytes) 000426[1] 1718927395.814260 mapwrite 0x49d51e thru 0x49ff63 (0x2a46 bytes) 000428[0] 1718927395.845163 trunc from 0x8411a0 to 0x8f65ca (0xb542b bytes) 000430[1] 1718927395.864610 write 0x7fd301 thru 0x807647 (0xa347 bytes) 000432[0] 1718927395.888062 trunc from 0x8f65cb to 0x4f6cb8 (0x3ff912 bytes) 000433[1] 1718927395.903900 trunc from 0x4f6cb9 to 0x8f6511 (0x3ff859 bytes) 000434[1] 1718927395.915192 mapread 0x3e0e65 thru 0x3ef84e (0xe9ea bytes) 000436[1] 1718927395.941247 write 0x4832bc thru 0x484dd6 (0x1b1b bytes) 000437[1] 1718927395.943714 read 0x56997c thru 0x57835d (0xe9e2 bytes) 000441[0] 1718927395.963193 trunc from 0x8f6512 to 0x26a524 (0x68bfed bytes) 000442[0] 1718927395.979757 write 0x63764e thru 0x6440e7 (0xca9a bytes) 000445[1] 1718927395.993144 mapread 0x55091e thru 0x55c009 (0xb6ec bytes) 000449[1] 1718927396.049879 mapwrite 0x8071d0 thru 0x80e174 (0x6fa5 bytes) 000450[0] 1718927396.089105 mapwrite 0x8d2157 thru 0x8da1db (0x8085 bytes) 000452[1] 1718927396.129888 mapread 0x6b759b thru 0x6b8870 (0x12d6 bytes) 000453[0] 1718927396.139421 mapread 0x535a0e thru 0x53ae8a (0x547d bytes) 000454[0] 1718927396.148351 write 0x241ad7 thru 0x24c4ed (0xaa17 bytes) 000455[0] 1718927396.153453 mapwrite 0x601af5 thru 0x60c2a4 (0xa7b0 bytes) 000456[1] 1718927396.197485 mapread 0x498c7c thru 0x49a3a1 (0x1726 bytes) 000457[1] 1718927396.204911 trunc from 0x8da1dc to 0x36093b (0x5798a0 bytes) 000459[1] 1718927396.221147 write 0x06f9c5 thru 0x07dc6f (0xe2ab bytes) 000463[1] 1718927396.234284 mapread 0x03dd5f thru 0x042949 (0x4beb bytes) 000464[0] 1718927396.243096 mapread 0x0442b0 thru 0x051c3c (0xd98d bytes) 000465[1] 1718927396.265040 mapread 0x352912 thru 0x35c725 (0x9e14 bytes) 000466[0] 1718927396.280647 read 0x32ff82 thru 0x338b17 (0x8b96 bytes) 000469[1] 1718927396.289183 mapread 0x2b7d98 thru 0x2c2ea9 (0xb112 bytes) 000471[1] 1718927396.302085 write 0x787bfa thru 0x789141 (0x1548 bytes) 000473[1] 1718927396.309272 read 0x4c5345 thru 0x4d1de2 (0xca9e bytes) 000475[1] 1718927396.328031 mapread 0x5d9f94 thru 0x5e99f0 (0xfa5d bytes) 000476[0] 1718927396.332266 mapwrite 0x887ab9 thru 0x8949ca (0xcf12 bytes) 000477[1] 1718927396.384442 read 0x70d098 thru 0x715335 (0x829e bytes) 000481[1] 1718927396.402666 read 0x42f38e thru 0x437c69 (0x88dc bytes) 000482[1] 1718927396.423827 read 0x3ae376 thru 0x3b3ec6 (0x5b51 bytes) 000483[0] 1718927396.429144 mapread 0x2aa03b thru 0x2b01db (0x61a1 bytes) 000484[1] 1718927396.437386 mapwrite 0x09aaab thru 0x09c539 (0x1a8f bytes) 000486[0] 1718927396.463962 mapread 0x877f4f thru 0x87aeab (0x2f5d bytes) 000489[1] 1718927396.472644 read 0x0938c7 thru 0x0949a3 (0x10dd bytes) 000490[0] 1718927396.490114 mapread 0x3cbccc thru 0x3cded8 (0x220d bytes) 000492[0] 1718927396.494653 write 0x2389b5 thru 0x23f63d (0x6c89 bytes) 000493[1] 1718927396.503083 read 0x0f8752 thru 0x1052a7 (0xcb56 bytes) 000496[0] 1718927396.522024 mapread 0x84d471 thru 0x850ede (0x3a6e bytes) 000497[0] 1718927396.526675 mapwrite 0x73665b thru 0x741cc2 (0xb668 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 7396 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read 000002[0] 1718927396.776534 mapwrite 0x659000 thru 0x65ea24 (0x5a25 bytes) 000004[1] 1718927396.814722 read 0x445000 thru 0x44fcdf (0xace0 bytes) 000005[1] 1718927396.818753 write 0x24f000 thru 0x255121 (0x6122 bytes) 000006[1] 1718927396.825510 mapread 0x273000 thru 0x280fd9 (0xdfda bytes) truncating to largest ever: 0x78d08 000007[0] 1718927396.832094 trunc from 0x65ea25 to 0x078d07 (0x5e5d1d bytes) 000011[0] 1718927396.857811 mapread 0x06f000 thru 0x078124 (0x9125 bytes) 000015[0] 1718927396.878649 read_OD 0x032000 thru 0x03bfff (0xa000 bytes) 000017[0] 1718927396.883269 mapwrite 0x79f000 thru 0x7a9191 (0xa192 bytes) 000021[0] 1718927396.913896 mapread 0x2f9000 thru 0x307a78 (0xea79 bytes) truncating to largest ever: 0x5bdb9f 000022[1] 1718927396.920282 trunc from 0x7a9192 to 0x5bdb9e (0x1eb5f3 bytes) 000023[0] 1718927396.926378 write_OD 0x4ac000 thru 0x4b2fff (0x7000 bytes) 000025[0] 1718927396.943329 read_OD 0x4bf000 thru 0x4c7fff (0x9000 bytes) 000027[0] 1718927396.945626 mapread 0x3d7000 thru 0x3e3873 (0xc874 bytes) 000028[0] 1718927396.966713 read_OD 0x30e000 thru 0x312fff (0x5000 bytes) 000032[1] 1718927396.969260 mapread 0x14c000 thru 0x157e26 (0xbe27 bytes) 000033[0] 1718927396.975813 mapread 0x410000 thru 0x41ee17 (0xee18 bytes) 000037[1] 1718927396.986751 mapread 0x07b000 thru 0x084bd2 (0x9bd3 bytes) truncating to largest ever: 0x8d4765 000038[1] 1718927396.989942 trunc from 0x5bdb9f to 0x8d4764 (0x316bc6 bytes) 000039[0] 1718927397.001292 trunc from 0x8d4765 to 0x51a5e5 (0x3ba17f bytes) 000041[1] 1718927397.013941 mapwrite 0x120000 thru 0x12f318 (0xf319 bytes) 000042[1] 1718927397.037072 write 0x5b8000 thru 0x5c47ea (0xc7eb bytes) 000043[0] 1718927397.042970 trunc from 0x5c47eb to 0x02a571 (0x59a279 bytes) 000045[0] 1718927397.052111 mapread 0x007000 thru 0x0129e8 (0xb9e9 bytes) 000048[1] 1718927397.069202 trunc from 0x02a572 to 0x789a7d (0x75f50c bytes) 000049[1] 1718927397.077342 read 0x4f8000 thru 0x4fc276 (0x4277 bytes) 000050[0] 1718927397.087505 read_OD 0x07d000 thru 0x087fff (0xb000 bytes) 000053[1] 1718927397.097934 trunc from 0x789a7e to 0x7e3234 (0x597b7 bytes) 000054[0] 1718927397.109187 mapread 0x5bd000 thru 0x5c19e5 (0x49e6 bytes) 000056[1] 1718927397.143325 write 0x833000 thru 0x833f68 (0xf69 bytes) 000057[1] 1718927397.153848 mapwrite 0x6bc000 thru 0x6cb11d (0xf11e bytes) 000058[1] 1718927397.190353 read 0x0a1000 thru 0x0aeeaf (0xdeb0 bytes) 000059[1] 1718927397.214994 mapread 0x383000 thru 0x384261 (0x1262 bytes) 000060[1] 1718927397.215474 mapwrite 0x063000 thru 0x07214e (0xf14f bytes) 000062[0] 1718927397.254846 mapwrite 0x143000 thru 0x150f99 (0xdf9a bytes) 000063[1] 1718927397.298422 trunc from 0x833f69 to 0x05a0c6 (0x7d9ea2 bytes) 000064[1] 1718927397.305088 read 0x006000 thru 0x011fc2 (0xbfc3 bytes) 000065[1] 1718927397.308954 write 0x3a6000 thru 0x3a97a0 (0x37a1 bytes) 000069[0] 1718927397.315486 mapread 0x249000 thru 0x249127 (0x128 bytes) 000070[1] 1718927397.324182 mapwrite 0x0f2000 thru 0x0ff064 (0xd065 bytes) 000071[0] 1718927397.357146 mapwrite 0x134000 thru 0x1359cc (0x19cd bytes) 000074[1] 1718927397.388458 trunc from 0x3a97a1 to 0x1c0861 (0x1e8f3f bytes) 000075[1] 1718927397.395313 read 0x1b0000 thru 0x1bb937 (0xb938 bytes) 000077[0] 1718927397.397961 mapread 0x10c000 thru 0x10ca2d (0xa2e bytes) 000079[1] 1718927397.404025 mapwrite 0x1c8000 thru 0x1d7023 (0xf024 bytes) 000080[1] 1718927397.428197 mapwrite 0x90f000 thru 0x91a30f (0xb310 bytes) 000083[1] 1718927397.479260 trunc from 0x91a310 to 0x3c0337 (0x559fd8 bytes) 000090[0] 1718927397.512639 read_OD 0x293000 thru 0x294fff (0x2000 bytes) 000091[0] 1718927397.519276 read_OD 0x249000 thru 0x24afff (0x2000 bytes) 000093[0] 1718927397.521789 trunc from 0x3c0338 to 0x2cd07a (0xf32bd bytes) 000097[0] 1718927397.540946 mapwrite 0x221000 thru 0x22cf3a (0xbf3b bytes) 000099[0] 1718927397.580949 read_OD 0x063000 thru 0x06afff (0x8000 bytes) 000103[1] 1718927397.586748 write 0x7f5000 thru 0x7fe20c (0x920d bytes) 000104[0] 1718927397.590655 mapread 0x0aa000 thru 0x0af216 (0x5217 bytes) 000106[0] 1718927397.621126 write_OD 0x134000 thru 0x13efff (0xb000 bytes) 000112[1] 1718927397.645023 mapread 0x10b000 thru 0x116e5e (0xbe5f bytes) 000113[1] 1718927397.658134 mapread 0x0ef000 thru 0x0f034d (0x134e bytes) 000117[1] 1718927397.660855 read 0x252000 thru 0x25a916 (0x8917 bytes) 000118[0] 1718927397.679478 mapread 0x4b1000 thru 0x4b26d0 (0x16d1 bytes) 000121[1] 1718927397.683427 mapread 0x7ec000 thru 0x7ed596 (0x1597 bytes) 000122[1] 1718927397.683725 read 0x0e7000 thru 0x0f4954 (0xd955 bytes) 000123[1] 1718927397.692140 mapread 0x1c8000 thru 0x1d3778 (0xb779 bytes) 000124[0] 1718927397.692646 write_OD 0x050000 thru 0x053fff (0x4000 bytes) 000125[0] 1718927397.715424 write_OD 0x189000 thru 0x18cfff (0x4000 bytes) 000127[1] 1718927397.730266 mapwrite 0x92c000 thru 0x9309b7 (0x49b8 bytes) 000128[1] 1718927397.763298 write 0x09d000 thru 0x0a62ff (0x9300 bytes) 000130[0] 1718927397.771315 write_OD 0x3ad000 thru 0x3b4fff (0x8000 bytes) 000132[0] 1718927397.797685 trunc from 0x9309b8 to 0x18495f (0x7ac058 bytes) 000133[1] 1718927397.803584 mapwrite 0x1b0000 thru 0x1b2021 (0x2022 bytes) 000134[0] 1718927397.837271 write 0x06a000 thru 0x077180 (0xd181 bytes) 000136[0] 1718927397.841476 mapwrite 0x131000 thru 0x140ff5 (0xfff6 bytes) 000138[1] 1718927397.872324 read 0x066000 thru 0x067de5 (0x1de6 bytes) 000139[0] 1718927397.883132 read 0x188000 thru 0x18c7af (0x47b0 bytes) 000141[1] 1718927397.886396 mapwrite 0x982000 thru 0x98e83b (0xc83c bytes) 000143[0] 1718927397.940648 write 0x089000 thru 0x08d74b (0x474c bytes) 000146[0] 1718927397.952561 trunc from 0x98e83c to 0x15b8c4 (0x832f77 bytes) 000148[1] 1718927397.960739 read 0x0ff000 thru 0x1000c5 (0x10c6 bytes) 000152[1] 1718927397.969649 read 0x06b000 thru 0x071f72 (0x6f73 bytes) 000154[1] 1718927397.975247 trunc from 0x15b8c5 to 0x6b2283 (0x5569bf bytes) 000155[0] 1718927397.987309 mapwrite 0x542000 thru 0x54b80f (0x9810 bytes) 000157[1] 1718927398.018505 write 0x095000 thru 0x0956f4 (0x6f5 bytes) 000158[0] 1718927398.041847 mapwrite 0x86a000 thru 0x8712a5 (0x72a6 bytes) 000159[1] 1718927398.074390 mapwrite 0x421000 thru 0x424450 (0x3451 bytes) 000160[0] 1718927398.106881 mapwrite 0x066000 thru 0x06f54c (0x954d bytes) 000163[0] 1718927398.147841 mapread 0x21a000 thru 0x21c9cd (0x29ce bytes) 000166[1] 1718927398.153623 mapwrite 0x14f000 thru 0x15e101 (0xf102 bytes) 000167[1] 1718927398.194654 read 0x228000 thru 0x2357fb (0xd7fc bytes) 000171[0] 1718927398.226684 mapread 0x42a000 thru 0x430c8d (0x6c8e bytes) 000172[1] 1718927398.243004 mapwrite 0x37f000 thru 0x385803 (0x6804 bytes) 000173[0] 1718927398.268559 write 0x8c6000 thru 0x8cd108 (0x7109 bytes) 000180[0] 1718927398.294626 write 0x109000 thru 0x10adbc (0x1dbd bytes) 000182[1] 1718927398.296952 mapread 0x369000 thru 0x3702e0 (0x72e1 bytes) 000183[1] 1718927398.320314 mapwrite 0x593000 thru 0x59965b (0x665c bytes) 000185[1] 1718927398.353600 mapread 0x3a0000 thru 0x3abbe2 (0xbbe3 bytes) 000187[0] 1718927398.367586 write 0x4c1000 thru 0x4c5f67 (0x4f68 bytes) 000188[1] 1718927398.374592 read 0x147000 thru 0x14a5ee (0x35ef bytes) 000189[1] 1718927398.401609 trunc from 0x8cd109 to 0x2b5e4e (0x6172ba bytes) 000190[1] 1718927398.415484 trunc from 0x2b5e4f to 0x69f15c (0x3e930e bytes) 000195[0] 1718927398.432764 mapread 0x584000 thru 0x58a537 (0x6538 bytes) 000196[0] 1718927398.440595 write 0x053000 thru 0x05dd67 (0xad68 bytes) 000197[0] 1718927398.447973 write 0x16d000 thru 0x176cbe (0x9cbf bytes) 000198[0] 1718927398.452719 read 0x327000 thru 0x32f83e (0x883f bytes) 000199[1] 1718927398.468364 trunc from 0x69f15d to 0x1e6573 (0x4b8be9 bytes) 000200[0] 1718927398.484718 write 0x01d000 thru 0x0257c8 (0x87c9 bytes) 000202[0] 1718927398.492099 write 0x9cd000 thru 0x9dbe42 (0xee43 bytes) 000204[0] 1718927398.498575 mapread 0x470000 thru 0x476672 (0x6673 bytes) 000205[0] 1718927398.512674 mapwrite 0x767000 thru 0x76ea4f (0x7a50 bytes) 000208[0] 1718927398.536637 mapread 0x79c000 thru 0x7ab0b7 (0xf0b8 bytes) 000210[1] 1718927398.537141 mapread 0x765000 thru 0x76d2f6 (0x82f7 bytes) 000211[0] 1718927398.565095 mapwrite 0x3bf000 thru 0x3c17b8 (0x27b9 bytes) 000212[0] 1718927398.592374 trunc from 0x9dbe43 to 0x2b5216 (0x726c2c bytes) 000214[1] 1718927398.597024 write 0x298000 thru 0x29ad1b (0x2d1c bytes) 000216[0] 1718927398.606852 write 0x653000 thru 0x653584 (0x585 bytes) 000218[1] 1718927398.615939 write 0x024000 thru 0x02c2cb (0x82cc bytes) 000222[1] 1718927398.630330 read 0x1a9000 thru 0x1b8994 (0xf995 bytes) 000224[0] 1718927398.651189 read 0x26e000 thru 0x27092c (0x292d bytes) 000226[1] 1718927398.674135 write 0x130000 thru 0x13454e (0x454f bytes) 000229[0] 1718927398.688302 read 0x16c000 thru 0x17b3f9 (0xf3fa bytes) 000233[0] 1718927398.713643 write 0x744000 thru 0x74e2a2 (0xa2a3 bytes) 000235[0] 1718927398.725039 mapwrite 0x217000 thru 0x21e0c5 (0x70c6 bytes) 000237[1] 1718927398.758907 write 0x904000 thru 0x90a2de (0x62df bytes) truncating to largest ever: 0x9cf172 000238[0] 1718927398.766415 trunc from 0x90a2df to 0x9cf171 (0xc4e93 bytes) 000240[0] 1718927398.774966 read 0x68a000 thru 0x68db73 (0x3b74 bytes) 000241[0] 1718927398.788158 mapread 0x3f5000 thru 0x4009b7 (0xb9b8 bytes) 000243[1] 1718927398.809025 read 0x050000 thru 0x05b795 (0xb796 bytes) 000244[1] 1718927398.843190 mapwrite 0x50a000 thru 0x512bfe (0x8bff bytes) 000245[0] 1718927398.881079 trunc from 0x9cf172 to 0x540c38 (0x48e539 bytes) 000246[1] 1718927398.889052 write 0x7b0000 thru 0x7be90f (0xe910 bytes) 000247[0] 1718927398.895313 mapread 0x58e000 thru 0x59d6b3 (0xf6b4 bytes) 000248[0] 1718927398.922346 read 0x3cd000 thru 0x3daa55 (0xda56 bytes) 000251[1] 1718927398.939699 mapread 0x473000 thru 0x481f5b (0xef5c bytes) 000252[0] 1718927398.955438 mapread 0x2d9000 thru 0x2e82f6 (0xf2f7 bytes) 000253[1] 1718927398.975315 trunc from 0x7be910 to 0x4e638a (0x2d8585 bytes) 000254[1] 1718927398.991497 trunc from 0x4e638b to 0x1b8e80 (0x32d50a bytes) 000255[1] 1718927398.996179 read 0x19d000 thru 0x19f857 (0x2858 bytes) 000258[0] 1718927399.008346 mapwrite 0x8f4000 thru 0x8f955c (0x555d bytes) 000263[0] 1718927399.042731 read 0x5d7000 thru 0x5d815e (0x115f bytes) 000264[1] 1718927399.052348 mapread 0x11c000 thru 0x11ce13 (0xe14 bytes) 000265[0] 1718927399.085580 read 0x5a7000 thru 0x5a9f0c (0x2f0d bytes) 000270[1] 1718927399.100646 mapread 0x197000 thru 0x1a6e59 (0xfe5a bytes) 000271[1] 1718927399.119183 mapwrite 0x10e000 thru 0x116709 (0x870a bytes) 000273[0] 1718927399.157197 mapwrite 0x07a000 thru 0x07fba9 (0x5baa bytes) 000274[0] 1718927399.193144 write 0x79d000 thru 0x7a0b47 (0x3b48 bytes) 000277[0] 1718927399.201690 read 0x3cb000 thru 0x3d7e0e (0xce0f bytes) 000278[0] 1718927399.221051 mapread 0x202000 thru 0x206b94 (0x4b95 bytes) 000279[0] 1718927399.231107 mapread 0x08f000 thru 0x0959bf (0x69c0 bytes) 000281[0] 1718927399.239803 write 0x056000 thru 0x05db1a (0x7b1b bytes) 000284[0] 1718927399.250678 mapread 0x781000 thru 0x78d887 (0xc888 bytes) 000287[0] 1718927399.254208 read 0x5ec000 thru 0x5f9ecc (0xdecd bytes) 000288[1] 1718927399.257321 trunc from 0x8f955d to 0x63c28c (0x2bd2d0 bytes) 000290[0] 1718927399.277092 write 0x54a000 thru 0x550cc4 (0x6cc5 bytes) 000291[1] 1718927399.281006 mapwrite 0x798000 thru 0x7a6505 (0xe506 bytes) 000292[1] 1718927399.327374 write 0x059000 thru 0x062460 (0x9461 bytes) 000293[1] 1718927399.333752 trunc from 0x7a6506 to 0x53fc72 (0x266893 bytes) 000295[0] 1718927399.343447 read 0x505000 thru 0x5112fa (0xc2fb bytes) 000296[0] 1718927399.352088 write 0x2ef000 thru 0x2f0a31 (0x1a32 bytes) 000297[1] 1718927399.359352 write 0x4b4000 thru 0x4c23ae (0xe3af bytes) 000300[0] 1718927399.374372 write 0x5b3000 thru 0x5beaa1 (0xbaa2 bytes) 000301[0] 1718927399.382111 mapread 0x134000 thru 0x13e616 (0xa617 bytes) 000305[1] 1718927399.403478 mapread 0x242000 thru 0x24af57 (0x8f58 bytes) 000309[0] 1718927399.423077 mapread 0x248000 thru 0x24a7f6 (0x27f7 bytes) 000311[0] 1718927399.426706 trunc from 0x5beaa2 to 0x91297d (0x353edc bytes) 000312[0] 1718927399.437321 mapread 0x0dd000 thru 0x0e29c5 (0x59c6 bytes) 000315[1] 1718927399.466891 trunc from 0x91297e to 0x4d6392 (0x43c5eb bytes) 000316[1] 1718927399.479890 mapwrite 0x3d5000 thru 0x3daf40 (0x5f41 bytes) 000319[1] 1718927399.510718 mapread 0x448000 thru 0x44c313 (0x4314 bytes) 000323[0] 1718927399.526294 write 0x99e000 thru 0x99fc7e (0x1c7f bytes) 000324[1] 1718927399.533064 write 0x34a000 thru 0x34c512 (0x2513 bytes) 000327[0] 1718927399.548449 write 0x26d000 thru 0x276e1b (0x9e1c bytes) 000328[0] 1718927399.561261 write 0x212000 thru 0x219fcc (0x7fcd bytes) 000329[0] 1718927399.563095 mapwrite 0x38b000 thru 0x38b572 (0x573 bytes) 000330[1] 1718927399.579551 write 0x81f000 thru 0x825d80 (0x6d81 bytes) 000331[1] 1718927399.584567 trunc from 0x99fc7f to 0x8fb8ea (0xa4394 bytes) 000333[0] 1718927399.589887 read 0x6d5000 thru 0x6db959 (0x695a bytes) 000334[1] 1718927399.607199 trunc from 0x8fb8eb to 0x4d250a (0x4293e0 bytes) 000336[0] 1718927399.619569 write 0x99a000 thru 0x99af54 (0xf55 bytes) 000339[0] 1718927399.626918 mapread 0x261000 thru 0x2669c1 (0x59c2 bytes) 000340[0] 1718927399.639803 write 0x3ed000 thru 0x3f301d (0x601e bytes) 000342[0] 1718927399.647688 read 0x1d6000 thru 0x1d8603 (0x2604 bytes) 000345[1] 1718927399.672280 mapread 0x8e7000 thru 0x8f0e18 (0x9e19 bytes) 000347[1] 1718927399.697731 mapread 0x931000 thru 0x940bb6 (0xfbb7 bytes) 000348[1] 1718927399.717631 mapread 0x309000 thru 0x30ca32 (0x3a33 bytes) 000351[0] 1718927399.722373 mapwrite 0x004000 thru 0x012131 (0xe132 bytes) 000359[0] 1718927399.775092 write 0x933000 thru 0x93d412 (0xa413 bytes) 000360[1] 1718927399.776903 write 0x848000 thru 0x84ebfd (0x6bfe bytes) 000361[1] 1718927399.786258 read 0x590000 thru 0x5969c1 (0x69c2 bytes) 000362[1] 1718927399.803792 mapwrite 0x002000 thru 0x006529 (0x452a bytes) 000363[0] 1718927399.826986 mapread 0x71d000 thru 0x72c7c1 (0xf7c2 bytes) 000366[1] 1718927399.853776 trunc from 0x99af55 to 0x803e1e (0x197136 bytes) 000367[1] 1718927399.863584 write 0x3b6000 thru 0x3beb55 (0x8b56 bytes) 000369[1] 1718927399.869597 mapwrite 0x908000 thru 0x917782 (0xf783 bytes) 000370[1] 1718927399.917630 write 0x291000 thru 0x29715d (0x615e bytes) 000374[0] 1718927399.928255 mapwrite 0x8a3000 thru 0x8b13e4 (0xe3e5 bytes) 000378[0] 1718927399.970992 trunc from 0x917783 to 0x556ec6 (0x3c08bc bytes) 000379[1] 1718927399.976775 write 0x41b000 thru 0x428e5b (0xde5c bytes) 000380[1] 1718927399.983918 write 0x98d000 thru 0x99453a (0x753b bytes) 000381[1] 1718927399.990027 mapwrite 0x288000 thru 0x28f215 (0x7216 bytes) 000384[0] 1718927400.027792 mapwrite 0x05b000 thru 0x066e4f (0xbe50 bytes) 000386[0] 1718927400.058595 read 0x15d000 thru 0x163b70 (0x6b71 bytes) 000389[0] 1718927400.086662 read 0x6a1000 thru 0x6a47f4 (0x37f5 bytes) 000392[1] 1718927400.090592 trunc from 0x99453b to 0x13c6d1 (0x857e69 bytes) 000393[0] 1718927400.109243 mapread 0x0e5000 thru 0x0f156a (0xc56b bytes) 000396[0] 1718927400.133178 trunc from 0x13c6d2 to 0x98b937 (0x84f266 bytes) 000397[1] 1718927400.142772 write 0x838000 thru 0x841878 (0x9879 bytes) 000398[0] 1718927400.150404 trunc from 0x98b938 to 0x6e7642 (0x2a42f5 bytes) 000400[1] 1718927400.164933 mapread 0x2e7000 thru 0x2ec51e (0x551f bytes) 000403[0] 1718927400.177360 write 0x0a2000 thru 0x0a4a2f (0x2a30 bytes) 000410[1] 1718927400.191036 trunc from 0x6e7643 to 0x2d37db (0x413e67 bytes) 000411[1] 1718927400.198171 mapread 0x19a000 thru 0x19bc3e (0x1c3f bytes) 000414[1] 1718927400.204931 mapwrite 0x878000 thru 0x88600e (0xe00f bytes) 000415[1] 1718927400.246056 mapread 0x254000 thru 0x2625fd (0xe5fe bytes) 000416[0] 1718927400.269350 read 0x098000 thru 0x0a3c89 (0xbc8a bytes) 000417[0] 1718927400.297191 trunc from 0x88600f to 0x7eef8f (0x9707f bytes) 000419[1] 1718927400.308985 trunc from 0x7eef90 to 0x84684e (0x578bf bytes) 000420[0] 1718927400.314103 mapread 0x2dc000 thru 0x2ea31f (0xe320 bytes) 000421[1] 1718927400.335292 trunc from 0x84684f to 0x6cbf25 (0x17a929 bytes) 000422[0] 1718927400.340462 mapwrite 0x4fd000 thru 0x4fe2a1 (0x12a2 bytes) 000423[0] 1718927400.362326 write 0x0e1000 thru 0x0e208d (0x108e bytes) 000424[1] 1718927400.365064 mapread 0x4fe000 thru 0x506b2d (0x8b2e bytes) 000425[1] 1718927400.381344 mapwrite 0x0f7000 thru 0x0f844e (0x144f bytes) 000426[1] 1718927400.399870 mapwrite 0x1d0000 thru 0x1d8229 (0x822a bytes) 000427[1] 1718927400.427852 mapwrite 0x14d000 thru 0x156d23 (0x9d24 bytes) 000428[0] 1718927400.457740 write 0x959000 thru 0x961dea (0x8deb bytes) 000432[0] 1718927400.467922 mapwrite 0x9ab000 thru 0x9abaef (0xaf0 bytes) 000433[1] 1718927400.497179 read 0x1aa000 thru 0x1b9267 (0xf268 bytes) 000434[1] 1718927400.531041 mapread 0x949000 thru 0x94fb79 (0x6b7a bytes) 000437[0] 1718927400.531626 write 0x8fa000 thru 0x900966 (0x6967 bytes) 000441[0] 1718927400.555861 mapwrite 0x8a0000 thru 0x8a659a (0x659b bytes) 000444[1] 1718927400.583380 mapwrite 0x5a4000 thru 0x5aa1a9 (0x61aa bytes) 000445[0] 1718927400.613204 write 0x88f000 thru 0x898b4b (0x9b4c bytes) 000448[0] 1718927400.620688 trunc from 0x9abaf0 to 0x37c3b4 (0x62f73b bytes) 000452[1] 1718927400.630940 write 0x3ef000 thru 0x3f9b23 (0xab24 bytes) 000453[1] 1718927400.637200 mapwrite 0x13d000 thru 0x149205 (0xc206 bytes) 000454[1] 1718927400.675381 mapread 0x364000 thru 0x36e786 (0xa787 bytes) 000457[1] 1718927400.700791 read 0x298000 thru 0x29bb42 (0x3b43 bytes) 000459[1] 1718927400.712675 mapread 0x23a000 thru 0x23c3cc (0x23cd bytes) 000460[1] 1718927400.716350 mapwrite 0x650000 thru 0x650332 (0x333 bytes) 000461[1] 1718927400.748173 mapwrite 0x603000 thru 0x60f748 (0xc749 bytes) 000463[1] 1718927400.765827 write 0x209000 thru 0x214b79 (0xbb7a bytes) 000464[1] 1718927400.771129 trunc from 0x650333 to 0x20f9a8 (0x44098a bytes) 000466[1] 1718927400.779544 write 0x019000 thru 0x01c9d6 (0x39d7 bytes) 000467[0] 1718927400.786796 trunc from 0x20f9a9 to 0x5862b3 (0x37690b bytes) 000468[0] 1718927400.798532 write 0x558000 thru 0x55aeef (0x2ef0 bytes) 000470[1] 1718927400.806184 read 0x340000 thru 0x3485a4 (0x85a5 bytes) 000471[1] 1718927400.821163 mapwrite 0x35d000 thru 0x36504d (0x804e bytes) 000472[0] 1718927400.866824 mapwrite 0x343000 thru 0x346da8 (0x3da9 bytes) 000474[0] 1718927400.888501 write 0x168000 thru 0x16e045 (0x6046 bytes) 000475[1] 1718927400.894026 mapwrite 0x7e3000 thru 0x7e4ec2 (0x1ec3 bytes) 000477[0] 1718927400.919890 read 0x3aa000 thru 0x3af479 (0x547a bytes) 000479[1] 1718927400.945938 mapread 0x111000 thru 0x1190e1 (0x80e2 bytes) 000484[0] 1718927400.953009 mapread 0x44d000 thru 0x4578ac (0xa8ad bytes) 000486[0] 1718927400.953408 mapwrite 0x896000 thru 0x89ccc5 (0x6cc6 bytes) 000487[1] 1718927400.987543 mapwrite 0x23f000 thru 0x241afc (0x2afd bytes) 000488[0] 1718927401.011467 read 0x08e000 thru 0x093095 (0x5096 bytes) 000490[0] 1718927401.037695 write 0x9a8000 thru 0x9a9450 (0x1451 bytes) 000495[0] 1718927401.060326 mapread 0x315000 thru 0x32073b (0xb73c bytes) 000496[1] 1718927401.078400 trunc from 0x9a9451 to 0x777189 (0x2322c7 bytes) 000499[1] 1718927401.095565 mapwrite 0x9b1000 thru 0x9b4629 (0x362a bytes) All operations completed A-OK! PASS 16a (13s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 19:50:02 (1718927402) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0863841 s, 24.3 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0689814 s, 30.4 MB/s Chance of close/open is 1 in 50 Seed set to 7402 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[1] 1718927403.088031 mapwrite 0x48faa5 thru 0x490637 (0xb93 bytes) 000002[1] 1718927403.132224 mapread 0x2e9965 thru 0x2e9c00 (0x29c bytes) 000003[1] 1718927403.137051 mapread 0x111257 thru 0x1119bc (0x766 bytes) truncating to largest ever: 0x5be73f 000005[1] 1718927403.144996 trunc from 0x490638 to 0x5be73e (0x12e107 bytes) 000007[1] 1718927403.150837 mapread 0x1fdab5 thru 0x1ff4ad (0x19f9 bytes) 000011[1] 1718927403.164239 trunc from 0x5be73f to 0x28c644 (0x3320fa bytes) truncating to largest ever: 0x7c238f 000012[0] 1718927403.172066 trunc from 0x28c645 to 0x7c238e (0x535d4a bytes) 000014[1] 1718927403.181721 write 0x9e167d thru 0x9e196f (0x2f3 bytes) 000015[1] 1718927403.187977 write 0x386394 thru 0x387fa8 (0x1c15 bytes) 000016[1] 1718927403.195717 write 0x448610 thru 0x4488ab (0x29c bytes) 000017[1] 1718927403.208107 read 0x8afe26 thru 0x8b0ebf (0x109a bytes) 000019[0] 1718927403.213355 read 0x65d0a4 thru 0x65d271 (0x1ce bytes) 000020[1] 1718927403.222984 trunc from 0x9e1970 to 0x0d895c (0x909013 bytes) 000025[1] 1718927403.239444 mapwrite 0x5a3807 thru 0x5a53cf (0x1bc9 bytes) 000029[0] 1718927403.289881 read 0x286b1f thru 0x28733c (0x81e bytes) 000030[1] 1718927403.300492 mapwrite 0x2c350a thru 0x2c3e5f (0x956 bytes) 000031[0] 1718927403.325278 trunc from 0x5a53d0 to 0x151faa (0x453425 bytes) 000032[1] 1718927403.334993 write 0x45fdd4 thru 0x46119e (0x13cb bytes) 000033[0] 1718927403.341790 mapwrite 0x6e08b3 thru 0x6e2539 (0x1c87 bytes) 000034[0] 1718927403.389759 trunc from 0x6e253a to 0x462489 (0x2800b0 bytes) 000036[1] 1718927403.401672 read 0x15c732 thru 0x15c9aa (0x279 bytes) 000037[0] 1718927403.412867 trunc from 0x46248a to 0x5dbf00 (0x179a77 bytes) 000040[1] 1718927403.437257 mapwrite 0x4593d3 thru 0x45a5f1 (0x121f bytes) 000042[0] 1718927403.477913 mapread 0x026720 thru 0x026aaa (0x38b bytes) 000043[1] 1718927403.487373 read 0x50becf thru 0x50cb3e (0xc70 bytes) 000044[0] 1718927403.498665 read 0x2be362 thru 0x2bf171 (0xe10 bytes) 000047[1] 1718927403.513531 write 0x6e2184 thru 0x6e26e7 (0x564 bytes) 000048[1] 1718927403.517789 mapread 0x598069 thru 0x598765 (0x6fd bytes) 000050[0] 1718927403.522836 write 0x1acd15 thru 0x1ad06f (0x35b bytes) 000051[1] 1718927403.533734 trunc from 0x6e26e8 to 0x6d527f (0xd468 bytes) 000052[1] 1718927403.542004 mapwrite 0x470a52 thru 0x472501 (0x1ab0 bytes) 000053[0] 1718927403.570170 read 0x16c488 thru 0x16c836 (0x3af bytes) 000058[1] 1718927403.600337 mapread 0x62224b thru 0x623e89 (0x1c3f bytes) 000060[1] 1718927403.606967 write 0x9d4ae8 thru 0x9d610b (0x1624 bytes) 000062[1] 1718927403.610562 trunc from 0x9d610c to 0x799694 (0x23ca77 bytes) 000064[0] 1718927403.614737 mapread 0x4403c6 thru 0x440b8f (0x7ca bytes) truncating to largest ever: 0x9cd00b 000065[0] 1718927403.633052 trunc from 0x799695 to 0x9cd00a (0x233976 bytes) 000069[1] 1718927403.654897 trunc from 0x9cd00b to 0x14f1e8 (0x87de22 bytes) 000070[0] 1718927403.664017 mapread 0x08fa54 thru 0x0915bf (0x1b6c bytes) 000071[0] 1718927403.677618 trunc from 0x14f1e9 to 0x371b08 (0x222920 bytes) 000072[0] 1718927403.689535 mapread 0x141304 thru 0x142773 (0x1470 bytes) 000073[1] 1718927403.696988 mapwrite 0x4e293b thru 0x4e2c78 (0x33e bytes) 000074[0] 1718927403.720998 mapwrite 0x655c1a thru 0x655f38 (0x31f bytes) 000075[1] 1718927403.761018 mapread 0x3c8b54 thru 0x3c936c (0x819 bytes) 000078[0] 1718927403.770853 mapread 0x032008 thru 0x03364b (0x1644 bytes) 000079[1] 1718927403.773300 write 0x028244 thru 0x029fa6 (0x1d63 bytes) 000080[1] 1718927403.778792 write 0x4b9dd8 thru 0x4ba343 (0x56c bytes) 000081[1] 1718927403.784648 mapwrite 0x28fdb5 thru 0x290899 (0xae5 bytes) 000082[1] 1718927403.806918 read 0x5381f2 thru 0x539afd (0x190c bytes) 000083[0] 1718927403.808640 read 0x06f93b thru 0x0700a4 (0x76a bytes) 000084[1] 1718927403.836976 mapread 0x03eda6 thru 0x040250 (0x14ab bytes) 000085[0] 1718927403.843162 mapread 0x13e165 thru 0x13fdaa (0x1c46 bytes) 000087[1] 1718927403.843742 mapwrite 0x4cd71a thru 0x4cf1d4 (0x1abb bytes) 000090[0] 1718927403.881420 trunc from 0x655f39 to 0x00db00 (0x648438 bytes) 000092[1] 1718927403.898620 mapread 0x00257d thru 0x003ca7 (0x172b bytes) 000095[1] 1718927403.914404 mapread 0x00319d thru 0x003a40 (0x8a4 bytes) 000096[1] 1718927403.917377 read 0x00a405 thru 0x00b0d9 (0xcd5 bytes) 000098[1] 1718927403.919265 mapread 0x002512 thru 0x00370f (0x11fe bytes) 000099[0] 1718927403.924103 mapread 0x009ccb thru 0x00b181 (0x14b7 bytes) 000102[0] 1718927403.928094 trunc from 0x00db01 to 0x183766 (0x175c66 bytes) 000103[0] 1718927403.934262 trunc from 0x183767 to 0x40537d (0x281c17 bytes) 000104[0] 1718927403.940745 mapread 0x286ee4 thru 0x287f54 (0x1071 bytes) 000105[0] 1718927403.946189 read 0x0c38f0 thru 0x0c3b4c (0x25d bytes) 000106[1] 1718927403.962615 write 0x54ca85 thru 0x54d6db (0xc57 bytes) 000108[0] 1718927403.974461 mapwrite 0x371df5 thru 0x3725cd (0x7d9 bytes) 000109[1] 1718927404.001628 trunc from 0x54d6dc to 0x04ffc0 (0x4fd71b bytes) 000111[0] 1718927404.014359 mapwrite 0x20ff6c thru 0x2119c9 (0x1a5e bytes) 000113[1] 1718927404.045179 write 0x9f1861 thru 0x9f1893 (0x33 bytes) 000115[1] 1718927404.056192 mapread 0x0ce58d thru 0x0cf615 (0x1089 bytes) 000117[0] 1718927404.088895 trunc from 0x9f1894 to 0x797792 (0x25a101 bytes) 000121[1] 1718927404.113563 mapwrite 0x348f06 thru 0x34a1fa (0x12f5 bytes) 000122[0] 1718927404.138731 read 0x2bb425 thru 0x2bb481 (0x5d bytes) 000124[0] 1718927404.149377 mapwrite 0x8a65fc thru 0x8a749b (0xea0 bytes) 000125[1] 1718927404.192925 write 0x1402c2 thru 0x141737 (0x1476 bytes) 000127[0] 1718927404.208981 mapread 0x7e6964 thru 0x7e8085 (0x1722 bytes) 000130[0] 1718927404.222143 read 0x1fd07c thru 0x1fdc21 (0xba6 bytes) 000135[1] 1718927404.248841 mapread 0x42a3c7 thru 0x42a7bc (0x3f6 bytes) 000137[1] 1718927404.260065 write 0x6a11ab thru 0x6a2985 (0x17db bytes) 000143[1] 1718927404.275526 write 0x8811f3 thru 0x881b00 (0x90e bytes) 000145[0] 1718927404.277409 mapread 0x72dc3b thru 0x72f462 (0x1828 bytes) 000146[1] 1718927404.279591 mapread 0x58409d thru 0x58433c (0x2a0 bytes) 000147[0] 1718927404.284175 mapread 0x0d4f7f thru 0x0d5cb6 (0xd38 bytes) 000151[1] 1718927404.294249 mapwrite 0x98cebe thru 0x98e904 (0x1a47 bytes) 000152[0] 1718927404.331136 trunc from 0x98e905 to 0x97cf51 (0x119b3 bytes) 000153[0] 1718927404.339024 read 0x6a6897 thru 0x6a7106 (0x870 bytes) 000154[1] 1718927404.353506 write 0x471fb5 thru 0x4732b0 (0x12fc bytes) 000155[1] 1718927404.365512 mapread 0x50da09 thru 0x50f637 (0x1c2f bytes) 000156[1] 1718927404.375716 mapread 0x103eee thru 0x105260 (0x1373 bytes) 000159[0] 1718927404.386747 write 0x8995a9 thru 0x89b04b (0x1aa3 bytes) 000163[1] 1718927404.396824 mapread 0x592fc1 thru 0x593d7e (0xdbe bytes) 000165[1] 1718927404.404900 write 0x3c25c5 thru 0x3c2f02 (0x93e bytes) 000168[0] 1718927404.417322 read 0x8360fd thru 0x8376a1 (0x15a5 bytes) 000171[1] 1718927404.437403 write 0x695383 thru 0x696c8b (0x1909 bytes) 000174[0] 1718927404.447652 trunc from 0x97cf52 to 0x0e1f39 (0x89b018 bytes) 000175[1] 1718927404.462061 mapread 0x0504f8 thru 0x050d24 (0x82d bytes) 000176[0] 1718927404.469748 trunc from 0x0e1f3a to 0x13c14c (0x5a213 bytes) 000177[0] 1718927404.476267 trunc from 0x13c14d to 0x3aabfb (0x26eaaf bytes) 000180[1] 1718927404.488573 trunc from 0x3aabfc to 0x8213ba (0x4767bf bytes) 000182[1] 1718927404.498512 write 0x2271b3 thru 0x227368 (0x1b6 bytes) 000183[0] 1718927404.500085 write 0x803206 thru 0x804307 (0x1102 bytes) 000186[0] 1718927404.517442 mapread 0x16c2ce thru 0x16c46c (0x19f bytes) 000188[0] 1718927404.525709 mapread 0x3f2fbd thru 0x3f3146 (0x18a bytes) 000189[1] 1718927404.531598 write 0x063043 thru 0x063daf (0xd6d bytes) 000190[0] 1718927404.539733 mapwrite 0x434055 thru 0x43409c (0x48 bytes) 000191[1] 1718927404.566850 mapwrite 0x2e1157 thru 0x2e145f (0x309 bytes) 000192[0] 1718927404.591261 mapread 0x279c55 thru 0x27b45c (0x1808 bytes) 000193[0] 1718927404.600727 write 0x52f709 thru 0x531379 (0x1c71 bytes) 000194[0] 1718927404.608779 write 0x01d6ab thru 0x01f03e (0x1994 bytes) 000199[1] 1718927404.619507 mapread 0x0bc626 thru 0x0bcdf6 (0x7d1 bytes) 000200[0] 1718927404.628545 mapwrite 0x54595d thru 0x545f2c (0x5d0 bytes) 000202[0] 1718927404.647139 write 0x7a3fcc thru 0x7a4293 (0x2c8 bytes) 000204[1] 1718927404.655714 write 0x2fdad7 thru 0x2fe9d2 (0xefc bytes) 000206[0] 1718927404.665685 read 0x5a3068 thru 0x5a445a (0x13f3 bytes) 000209[1] 1718927404.679935 write 0x6d0f84 thru 0x6d2a23 (0x1aa0 bytes) 000210[1] 1718927404.686035 trunc from 0x8213bb to 0x92831a (0x106f60 bytes) 000212[1] 1718927404.700277 mapread 0x251ef9 thru 0x252a45 (0xb4d bytes) 000214[0] 1718927404.703991 read 0x1e684f thru 0x1e812b (0x18dd bytes) 000215[0] 1718927404.710761 mapwrite 0x536ad8 thru 0x538108 (0x1631 bytes) 000219[1] 1718927404.744370 write 0x57e0c6 thru 0x57fb67 (0x1aa2 bytes) 000220[1] 1718927404.754059 trunc from 0x92831b to 0x6fc2e4 (0x22c036 bytes) 000221[0] 1718927404.761675 mapwrite 0x71dd93 thru 0x71f1df (0x144d bytes) 000229[1] 1718927404.831927 read 0x114f71 thru 0x116479 (0x1509 bytes) 000230[0] 1718927404.847281 read 0x60b303 thru 0x60b7b8 (0x4b6 bytes) 000231[1] 1718927404.848493 write 0x9b6dff thru 0x9b8466 (0x1668 bytes) 000232[1] 1718927404.855328 read 0x18b720 thru 0x18ba6e (0x34f bytes) 000234[1] 1718927404.867240 mapwrite 0x8a56e2 thru 0x8a6e37 (0x1756 bytes) 000235[0] 1718927404.913600 read 0x889e8f thru 0x88ad9f (0xf11 bytes) 000236[0] 1718927404.924959 write 0x8fbccb thru 0x8fdbe6 (0x1f1c bytes) 000237[0] 1718927404.930313 trunc from 0x9b8467 to 0x49c3a6 (0x51c0c0 bytes) 000238[1] 1718927404.944662 read 0x2fba46 thru 0x2fbe02 (0x3bd bytes) 000239[0] 1718927404.951879 read 0x4175ac thru 0x418d22 (0x1777 bytes) 000240[0] 1718927404.958089 write 0x733040 thru 0x7348d1 (0x1892 bytes) 000241[1] 1718927404.958958 mapwrite 0x53dbe1 thru 0x53ddaa (0x1ca bytes) 000242[0] 1718927404.995059 read 0x412a4a thru 0x413bba (0x1171 bytes) 000247[0] 1718927405.013062 read 0x017270 thru 0x01800f (0xda0 bytes) 000251[0] 1718927405.035081 mapread 0x673385 thru 0x674abe (0x173a bytes) 000252[0] 1718927405.047045 mapwrite 0x42db06 thru 0x42edb7 (0x12b2 bytes) 000253[0] 1718927405.085904 write 0x25471f thru 0x255574 (0xe56 bytes) 000257[0] 1718927405.104148 write 0x6299ce thru 0x62b743 (0x1d76 bytes) 000258[1] 1718927405.108203 trunc from 0x7348d2 to 0x64e3cf (0xe6502 bytes) 000259[0] 1718927405.116858 trunc from 0x64e3d0 to 0x159b2c (0x4f48a3 bytes) 000261[1] 1718927405.126753 write 0x8fbc77 thru 0x8fccf2 (0x107c bytes) 000262[0] 1718927405.134286 trunc from 0x8fccf3 to 0x9c44f4 (0xc7802 bytes) 000263[1] 1718927405.141069 write 0x3775a9 thru 0x37914a (0x1ba2 bytes) 000264[1] 1718927405.155740 trunc from 0x9c44f5 to 0x8012fb (0x1c31f9 bytes) 000266[1] 1718927405.167270 mapread 0x7f256d thru 0x7f35fe (0x1092 bytes) 000272[0] 1718927405.180247 mapwrite 0x081937 thru 0x08288e (0xf58 bytes) 000273[0] 1718927405.203069 mapread 0x43647d thru 0x4366b8 (0x23c bytes) 000274[1] 1718927405.210808 read 0x4277b9 thru 0x428593 (0xddb bytes) 000276[0] 1718927405.232738 mapread 0x2d02a4 thru 0x2d2287 (0x1fe4 bytes) 000279[1] 1718927405.238485 trunc from 0x8012fc to 0x72a959 (0xd69a2 bytes) 000280[0] 1718927405.251848 mapwrite 0x0a081a thru 0x0a1577 (0xd5e bytes) 000282[0] 1718927405.278960 mapread 0x05ea8c thru 0x0609c8 (0x1f3d bytes) 000283[0] 1718927405.288594 write 0x903e23 thru 0x905913 (0x1af1 bytes) 000285[1] 1718927405.297744 read 0x36219b thru 0x362f32 (0xd98 bytes) 000286[1] 1718927405.308832 mapwrite 0x85d5b8 thru 0x85d73a (0x183 bytes) 000288[0] 1718927405.348333 mapwrite 0x1f9e52 thru 0x1faad8 (0xc87 bytes) 000289[1] 1718927405.372785 mapread 0x1b37a8 thru 0x1b42e8 (0xb41 bytes) 000291[1] 1718927405.382196 read 0x5e6406 thru 0x5e68cb (0x4c6 bytes) 000295[1] 1718927405.402594 trunc from 0x905914 to 0x2ba095 (0x64b87e bytes) 000296[1] 1718927405.414338 trunc from 0x2ba096 to 0x49e14b (0x1e40b6 bytes) 000297[1] 1718927405.418117 trunc from 0x49e14c to 0x79305a (0x2f4f0f bytes) 000301[1] 1718927405.425973 read 0x3b85e1 thru 0x3b9ca3 (0x16c3 bytes) 000302[1] 1718927405.439205 mapread 0x28c1fb thru 0x28ca36 (0x83c bytes) 000303[0] 1718927405.441538 mapread 0x0f5296 thru 0x0f7020 (0x1d8b bytes) 000304[1] 1718927405.453264 trunc from 0x79305b to 0x2046ef (0x58e96b bytes) 000305[1] 1718927405.461144 mapwrite 0x12e49e thru 0x12f7c2 (0x1325 bytes) 000306[1] 1718927405.484923 write 0x97badd thru 0x97ca63 (0xf87 bytes) 000307[0] 1718927405.487035 trunc from 0x97ca64 to 0x123f5a (0x858b09 bytes) 000309[0] 1718927405.498409 write 0x4151fd thru 0x4167a3 (0x15a7 bytes) 000310[0] 1718927405.504339 write 0x7d2556 thru 0x7d3f64 (0x1a0f bytes) 000311[1] 1718927405.505710 write 0x28f41d thru 0x28f492 (0x76 bytes) 000317[1] 1718927405.532274 mapread 0x5f1299 thru 0x5f229e (0x1006 bytes) 000318[1] 1718927405.543520 read 0x26217a thru 0x26358c (0x1413 bytes) 000319[1] 1718927405.573128 write 0x61e425 thru 0x61e674 (0x250 bytes) 000321[1] 1718927405.582999 write 0x8920f4 thru 0x893272 (0x117f bytes) 000322[1] 1718927405.588709 write 0x61aad2 thru 0x61b863 (0xd92 bytes) 000324[1] 1718927405.594481 read 0x5cb429 thru 0x5ccb64 (0x173c bytes) 000325[0] 1718927405.594827 read 0x74b064 thru 0x74c564 (0x1501 bytes) 000326[1] 1718927405.601281 mapwrite 0x6aeb14 thru 0x6b0a15 (0x1f02 bytes) 000333[0] 1718927405.641798 read 0x1fe333 thru 0x1fe8ee (0x5bc bytes) 000334[1] 1718927405.644038 mapwrite 0x5b7f45 thru 0x5b8e93 (0xf4f bytes) 000335[1] 1718927405.678425 trunc from 0x893273 to 0x8f90c2 (0x65e50 bytes) 000338[1] 1718927405.688840 read 0x19d595 thru 0x19e038 (0xaa4 bytes) 000339[0] 1718927405.709548 write 0x0284bc thru 0x028a6e (0x5b3 bytes) 000345[1] 1718927405.749850 mapwrite 0x700431 thru 0x70161d (0x11ed bytes) 000346[1] 1718927405.772397 mapread 0x5ab7ef thru 0x5ac4c0 (0xcd2 bytes) 000347[1] 1718927405.778881 write 0x2d07e0 thru 0x2d2068 (0x1889 bytes) 000348[0] 1718927405.785821 trunc from 0x8f90c3 to 0x68bb69 (0x26d559 bytes) 000350[0] 1718927405.806515 read 0x28c09f thru 0x28cbcd (0xb2f bytes) 000353[0] 1718927405.831645 mapwrite 0x17cb32 thru 0x17cd5c (0x22b bytes) 000354[1] 1718927405.852508 mapread 0x30f45b thru 0x30fcdf (0x885 bytes) 000355[1] 1718927405.863653 read 0x379ca5 thru 0x37b912 (0x1c6e bytes) 000359[1] 1718927405.871717 mapwrite 0x286f01 thru 0x28702b (0x12b bytes) 000360[0] 1718927405.922586 trunc from 0x68bb6a to 0x1421dd (0x54998c bytes) 000362[1] 1718927405.940005 mapwrite 0x89e94e thru 0x89fa86 (0x1139 bytes) 000363[0] 1718927405.989763 mapread 0x55f9cf thru 0x55fb5d (0x18f bytes) 000364[0] 1718927405.999587 write 0x568df8 thru 0x5694ce (0x6d7 bytes) 000365[1] 1718927406.008453 mapread 0x24b530 thru 0x24b5e4 (0xb5 bytes) 000367[0] 1718927406.019178 trunc from 0x89fa87 to 0x71c8eb (0x18319b bytes) 000368[0] 1718927406.026634 mapread 0x0ee82f thru 0x0ef981 (0x1153 bytes) 000370[1] 1718927406.043380 mapread 0x673dc2 thru 0x674c9d (0xedc bytes) 000371[1] 1718927406.051493 mapread 0x0ebaee thru 0x0ec69b (0xbae bytes) 000372[0] 1718927406.059780 write 0x79fe4d thru 0x7a1dbe (0x1f72 bytes) 000374[1] 1718927406.063981 read 0x3f946e thru 0x3fa31a (0xead bytes) 000375[0] 1718927406.084109 read 0x593e2d thru 0x594d49 (0xf1d bytes) 000376[1] 1718927406.096122 mapwrite 0x42297d thru 0x422a71 (0xf5 bytes) 000378[0] 1718927406.133082 read 0x1e55b5 thru 0x1e5ea1 (0x8ed bytes) 000379[0] 1718927406.136639 read 0x2d95c0 thru 0x2db0fa (0x1b3b bytes) 000380[0] 1718927406.157734 trunc from 0x7a1dbf to 0x046e7e (0x75af40 bytes) 000381[1] 1718927406.170865 write 0x5e37c0 thru 0x5e4ae4 (0x1325 bytes) 000384[1] 1718927406.187437 mapread 0x2be891 thru 0x2bfaf6 (0x1266 bytes) 000387[0] 1718927406.204683 trunc from 0x5e4ae5 to 0x5fbff0 (0x1750c bytes) 000388[0] 1718927406.217368 read 0x3268b6 thru 0x3274a1 (0xbec bytes) 000389[1] 1718927406.233149 read 0x5eea00 thru 0x5efe32 (0x1433 bytes) 000390[1] 1718927406.241845 mapwrite 0x8c0e77 thru 0x8c17c8 (0x952 bytes) 000391[0] 1718927406.282758 write 0x4a505f thru 0x4a524e (0x1f0 bytes) 000395[1] 1718927406.303208 write 0x641a54 thru 0x643a24 (0x1fd1 bytes) 000399[0] 1718927406.326051 read 0x8a25b8 thru 0x8a301b (0xa64 bytes) 000400[0] 1718927406.336122 read 0x4aeb16 thru 0x4af562 (0xa4d bytes) 000403[1] 1718927406.349175 trunc from 0x8c17c9 to 0x1e8a4b (0x6d8d7d bytes) 000404[0] 1718927406.360142 trunc from 0x1e8a4c to 0x396ec0 (0x1ae475 bytes) 000406[1] 1718927406.372563 write 0x43c585 thru 0x43d8c5 (0x1341 bytes) 000413[1] 1718927406.399589 trunc from 0x43d8c6 to 0x312be8 (0x12acdd bytes) 000414[0] 1718927406.408627 write 0x27044e thru 0x2707ce (0x381 bytes) 000415[1] 1718927406.415699 write 0x8c4a7e thru 0x8c684a (0x1dcd bytes) 000416[1] 1718927406.423856 mapread 0x2c7bbf thru 0x2c9823 (0x1c65 bytes) 000419[1] 1718927406.434251 write 0x792c2a thru 0x79404f (0x1426 bytes) 000420[0] 1718927406.436503 mapread 0x8a4ce0 thru 0x8a67ee (0x1b0f bytes) 000423[0] 1718927406.453925 mapwrite 0x42ecc3 thru 0x42f0ae (0x3ec bytes) 000427[0] 1718927406.486531 trunc from 0x8c684b to 0x6f5feb (0x1d085f bytes) truncating to largest ever: 0x9d143a 000428[1] 1718927406.494639 trunc from 0x6f5fec to 0x9d1439 (0x2db44e bytes) 000429[1] 1718927406.507042 write 0x833b2a thru 0x834b61 (0x1038 bytes) 000431[1] 1718927406.514316 mapwrite 0x1facbf thru 0x1fc027 (0x1369 bytes) 000433[0] 1718927406.535334 mapwrite 0x01feae thru 0x021e04 (0x1f57 bytes) 000435[1] 1718927406.565417 read 0x2027f4 thru 0x202de1 (0x5ee bytes) 000436[0] 1718927406.596064 mapread 0x724943 thru 0x7262a2 (0x1960 bytes) 000437[1] 1718927406.625375 mapwrite 0x3a23e5 thru 0x3a2dda (0x9f6 bytes) 000438[0] 1718927406.646139 mapwrite 0x7c2b28 thru 0x7c3670 (0xb49 bytes) 000439[0] 1718927406.672358 read 0x2f18e7 thru 0x2f3752 (0x1e6c bytes) 000440[1] 1718927406.695264 mapwrite 0x5dcbb0 thru 0x5de8fe (0x1d4f bytes) 000443[1] 1718927406.735833 read 0x823ca8 thru 0x823eac (0x205 bytes) 000444[1] 1718927406.736150 mapwrite 0x184976 thru 0x1855cb (0xc56 bytes) 000446[1] 1718927406.765274 read 0x38c336 thru 0x38caf2 (0x7bd bytes) 000447[0] 1718927406.782501 mapwrite 0x8a6e0a thru 0x8a7433 (0x62a bytes) 000450[0] 1718927406.834278 mapread 0x4eed2d thru 0x4f0534 (0x1808 bytes) 000452[0] 1718927406.848201 write 0x92cabf thru 0x92d53e (0xa80 bytes) 000453[0] 1718927406.856979 mapwrite 0x536cf6 thru 0x538a5f (0x1d6a bytes) 000454[1] 1718927406.886347 trunc from 0x9d143a to 0x57b3cb (0x45606e bytes) 000456[0] 1718927406.900386 read 0x225fc4 thru 0x2265b3 (0x5f0 bytes) 000457[0] 1718927406.915481 read 0x3b219a thru 0x3b3de7 (0x1c4e bytes) 000458[1] 1718927406.927089 trunc from 0x57b3cc to 0x9487dd (0x3cd412 bytes) 000460[0] 1718927406.945373 write 0x157cbe thru 0x158da2 (0x10e5 bytes) 000461[1] 1718927406.957416 write 0x073f32 thru 0x075ef1 (0x1fc0 bytes) 000462[1] 1718927406.963623 write 0x8ef4ae thru 0x8f0630 (0x1183 bytes) 000463[0] 1718927406.971102 mapread 0x049ab1 thru 0x04a102 (0x652 bytes) 000466[1] 1718927407.003653 mapread 0x35f55f thru 0x35fa15 (0x4b7 bytes) 000467[1] 1718927407.011614 mapread 0x1cfc90 thru 0x1d15ff (0x1970 bytes) 000469[1] 1718927407.016406 read 0x7d2a12 thru 0x7d49d5 (0x1fc4 bytes) 000472[0] 1718927407.024368 mapwrite 0x00d48c thru 0x00dc67 (0x7dc bytes) 000475[1] 1718927407.049123 write 0x7d5d8a thru 0x7d6adb (0xd52 bytes) 000477[1] 1718927407.060737 mapwrite 0x9b7da7 thru 0x9b8d6a (0xfc4 bytes) 000478[0] 1718927407.090155 read 0x4fe419 thru 0x4ff316 (0xefe bytes) 000481[1] 1718927407.100275 read 0x9715ea thru 0x97324c (0x1c63 bytes) 000483[0] 1718927407.105204 write 0x91f0eb thru 0x920aee (0x1a04 bytes) 000484[0] 1718927407.138099 mapread 0x472a00 thru 0x472c79 (0x27a bytes) 000485[1] 1718927407.141248 mapwrite 0x07280b thru 0x0739d5 (0x11cb bytes) 000486[0] 1718927407.173421 trunc from 0x9b8d6b to 0x473d86 (0x544fe4 bytes) 000487[1] 1718927407.184038 write 0x8acfd8 thru 0x8ae67d (0x16a6 bytes) 000491[0] 1718927407.202731 trunc from 0x8ae67e to 0x7bb2cb (0xf33b2 bytes) 000492[1] 1718927407.211684 mapwrite 0x0d38e1 thru 0x0d4eb3 (0x15d3 bytes) 000494[0] 1718927407.236008 mapwrite 0x0090cf thru 0x00a758 (0x168a bytes) 000495[1] 1718927407.258507 trunc from 0x7bb2cc to 0x56760b (0x253cc0 bytes) 000497[1] 1718927407.272514 write 0x09350a thru 0x094851 (0x1348 bytes) 000498[0] 1718927407.275721 mapwrite 0x3a986e thru 0x3aaacf (0x1262 bytes) 000500[0] 1718927407.301664 write 0x8a5b51 thru 0x8a68f8 (0xda8 bytes) All operations completed A-OK! PASS 16b (6s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 19:50:08 (1718927408) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 19:50:09 (1718927409) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0892257 s, 23.5 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.066889 s, 31.4 MB/s ldlm.namespaces.MGC192.168.202.137@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800a9905800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b3a56800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800a9905800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b3a56800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800a9905800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b3a56800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.99326 s, 35.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.668979 s, 157 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.602259 s, 174 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.23314 s, 85.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.73523 s, 60.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.903268 s, 116 MB/s PASS 16d (11s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 19:50:20 (1718927420) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0557238 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357202 s, 29.4 MB/s PASS 16e (1s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 19:50:21 (1718927421) TIMEOUT PASS 16f (22s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 19:50:43 (1718927443) TIMEOUT PASS 16g (22s) == sanityn test 16i: read after truncate file ============ 19:51:05 (1718927465) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (1s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 19:51:06 (1718927466) fail_loc=0x8000030a PASS 17 (4s) == sanityn test 18: mmap sanity check =========================================================================================== 19:51:10 (1718927470) mmap test1: basic mmap operation (PASS, 0.014945s) mmap test2: MAP_PRIVATE not write back (PASS, 0.01103s) mmap test3: concurrent mmap ops on two nodes (PASS, 4.4948s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.0643s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.020028s) mmap test6: check mmap write/read content on two nodes (PASS, 0.023698s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0113s) PASS 18 (10s) == sanityn test 19: test concurrent uncached read races ========================================================================= 19:51:20 (1718927480) oleg237-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg237-client: oleg237-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 19:51:21 (1718927481) PASS 20 (1s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 19:51:22 (1718927482) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) == sanityn test 23: others should see updated atime while another read============================================================== 19:51:24 (1718927484) now is 1718927484 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6872 new atime is 1718927545 PASS 23 (62s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 19:52:26 (1718927546) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 374 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 375 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 374 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 375 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 8.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 5.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 13.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117974 374 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118071 375 117696 1% /mnt/lustre2[OST:1] filesystem_summary: 235665 369 235296 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 374 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 375 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 117974 374 117600 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118071 375 117696 1% /mnt/lustre[OST:1] filesystem_summary: 235665 369 235296 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117974 374 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118071 375 117696 1% /mnt/lustre2[OST:1] filesystem_summary: 235665 369 235296 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre2 PASS 24a (2s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 19:52:28 (1718927548) PASS 24b (1s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 19:52:29 (1718927549) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (2s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 19:52:31 (1718927551) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (0s) == sanityn test 26a: allow mtime to get older ============ 19:52:31 (1718927551) utime: good mknod times 1718927551 <= 1718927552 <= 1718927552 for /mnt/lustre/f26a utime: good mknod times 1718927551 <= 1718927552 <= 1718927552 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (3s) == sanityn test 26b: sync mtime between ost and mds ====== 19:52:34 (1718927554) PASS 26b (3s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 19:52:38 (1718927558) 0 PASS 30 (3s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 19:52:41 (1718927561) fail_loc=0x314 PASS 31a (3s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 19:52:44 (1718927564) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg237-server mds-ost sync done. sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.202.137@tcp:/lustre 7542784 14336 7524352 1% /mnt/lustre2 PASS 31b (18s) == sanityn test 31r: open-rename(replace) race =========== 19:53:02 (1718927582) -rw-r--r-- 1 root root 0 Jun 20 19:53 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (4s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 19:53:07 (1718927587) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (0s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 19:53:08 (1718927588) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (0s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 19:53:08 (1718927588) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 19:53:09 (1718927589) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 19:53:10 (1718927590) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg237-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a9905800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b3a56800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800a9905800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8800b3a56800.ost_server_uuid in FULL state after 0 sec oleg237-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800a9905800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b3a56800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800a9905800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800b3a56800.ost_server_uuid in FULL state after 0 sec oleg237-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a9905800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b3a56800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800a9905800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8800b3a56800.ost_server_uuid in FULL state after 0 sec oleg237-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800a9905800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b3a56800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800a9905800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8800b3a56800.ost_server_uuid in IDLE state after 0 sec oleg237-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a9905800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b3a56800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800a9905800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8800b3a56800.ost_server_uuid in IDLE state after 0 sec oleg237-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800a9905800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b3a56800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800a9905800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8800b3a56800.ost_server_uuid in IDLE state after 0 sec PASS 34 (71s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 19:54:21 (1718927661) fail_loc=0x80000317 Race attempt 0 Wait for 15565 15704 for 60 sec... total: 4000 open/close in 8.12 seconds: 492.90 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.202.137@tcp:/lustre 7.2G 15M 7.2G 1% /mnt/lustre 192.168.202.137@tcp:/lustre 7.2G 15M 7.2G 1% /mnt/lustre2 PASS 35 (63s) == sanityn test 36: handle ESTALE/open-unlink correctly == 19:55:24 (1718927724) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg237-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.317921 s, 165 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6872 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.296608 s, 177 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6872 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7523328) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.299472 s, 175 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6872 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7522304) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.313874 s, 167 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6872 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7522304) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.301451 s, 174 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6872 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7522304) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.372644 s, 141 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6872 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7522304) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.298749 s, 175 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6872 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7522304) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.28597 s, 183 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6872 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7522304) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.279896 s, 187 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6872 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7522304) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.273465 s, 192 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6872 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7522304) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.277368 s, 189 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6872 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7522304) after_dd(7471104) after(7522304) PASS 36 (231s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 19:59:15 (1718927955) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6872 - create 9533 (time 1718927966.77 total 10.00 last 953.28) total: 10000 create in 10.44 seconds: 958.03 ops/second PASS 37 (13s) == sanityn test 39a: test from 11063 ============================================================================================ 19:59:28 (1718927968) repeat after cancel_lru_locks PASS 39a (2s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 19:59:30 (1718927970) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 19:59:34 (1718927974) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 19:59:36 (1718927976) fail_loc=0x411 fail_loc=0 PASS 39d (2s) == sanityn test 40a: pdirops: create vs others ======================================================================== 19:59:38 (1718927978) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (7s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 19:59:45 (1718927985) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (8s) == sanityn test 40c: pdirops: link and others ======================================================================== 19:59:53 (1718927993) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (7s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 20:00:00 (1718928000) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (7s) == sanityn test 40e: pdirops: rename and others ======================================================================== 20:00:08 (1718928008) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (6s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 20:00:14 (1718928014) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (5s) == sanityn test 41b: pdirops: create vs create ======================================================================== 20:00:19 (1718928019) open: File exists Conflict PASS 41b (5s) == sanityn test 41c: pdirops: create vs link ======================================================================== 20:00:24 (1718928024) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (6s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 20:00:30 (1718928030) Conflict PASS 41d (5s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 20:00:35 (1718928035) Conflict PASS 41e (4s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 20:00:39 (1718928039) Conflict PASS 41f (5s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 20:00:44 (1718928044) Conflict PASS 41g (5s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 20:00:49 (1718928049) Conflict PASS 41h (5s) == sanityn test 41i: reint_open: create vs create ======== 20:00:54 (1718928054) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (676s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 20:12:10 (1718928730) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (5s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 20:12:15 (1718928735) open: File exists Conflict PASS 42b (4s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 20:12:19 (1718928739) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (5s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 20:12:24 (1718928744) Conflict PASS 42d (5s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 20:12:29 (1718928749) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (4s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 20:12:33 (1718928753) Conflict PASS 42f (5s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 20:12:38 (1718928758) Conflict PASS 42g (4s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 20:12:42 (1718928762) Conflict PASS 42h (4s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 20:12:46 (1718928766) PASS 43a (11s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 20:12:57 (1718928777) Conflict PASS 43b (4s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 20:13:01 (1718928781) Conflict PASS 43c (4s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 20:13:05 (1718928785) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (4s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 20:13:09 (1718928789) Conflict PASS 43e (4s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 20:13:13 (1718928793) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (5s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 20:13:18 (1718928798) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (4s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 20:13:22 (1718928802) Conflict PASS 43h (5s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 20:13:27 (1718928807) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (0s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 20:13:27 (1718928807) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (37s) == sanityn test 43k: unlink vs create ==================== 20:14:04 (1718928844) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (402s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 20:20:46 (1718929246) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Thu Jun 20 20:20:50 EDT 2024 PASS 44a (5s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 20:20:51 (1718929251) open: File exists Conflict PASS 44b (5s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 20:20:56 (1718929256) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (4s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 20:21:00 (1718929260) Conflict PASS 44d (5s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 20:21:05 (1718929265) Conflict PASS 44e (6s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 20:21:11 (1718929271) Conflict PASS 44f (5s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 20:21:16 (1718929276) Conflict PASS 44g (4s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 20:21:20 (1718929280) Conflict PASS 44h (5s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 20:21:25 (1718929285) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 20:21:26 (1718929286) PASS 45a (25s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 20:21:51 (1718929311) Conflict PASS 45b (4s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 20:21:55 (1718929315) Conflict PASS 45c (5s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 20:22:00 (1718929320) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (5s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 20:22:05 (1718929325) Conflict PASS 45e (5s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 20:22:11 (1718929331) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (6s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 20:22:16 (1718929336) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (5s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 20:22:21 (1718929341) Conflict PASS 45h (4s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 20:22:25 (1718929345) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 20:22:26 (1718929346) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (401s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 20:29:07 (1718929747) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (5s) == sanityn test 46b: pdirops: link vs create ======================================================================== 20:29:12 (1718929752) open: File exists Conflict PASS 46b (4s) == sanityn test 46c: pdirops: link vs link ======================================================================== 20:29:16 (1718929756) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (4s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 20:29:20 (1718929760) Conflict PASS 46d (4s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 20:29:24 (1718929764) Conflict PASS 46e (5s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 20:29:29 (1718929769) Conflict PASS 46f (4s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 20:29:33 (1718929773) Conflict PASS 46g (4s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 20:29:37 (1718929777) Conflict PASS 46h (4s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 20:29:41 (1718929781) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 20:29:42 (1718929782) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 20:29:43 (1718929783) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 20:29:44 (1718929784) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 20:29:45 (1718929785) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (0s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 20:29:45 (1718929785) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 20:29:46 (1718929786) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 20:29:47 (1718929787) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 20:29:48 (1718929788) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.00418147 s, 2.4 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (6s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 20:29:54 (1718929794) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00457523 s, 34.8 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3175: kill: (24637) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 20:29:58 (1718929798) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00299377 s, 342 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133492 s, 78.5 MB/s 3145728 PASS 51b (14s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 20:30:12 (1718929812) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00369505 s, 277 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 20:30:18 (1718929818) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008296 s, 126 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (2s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 20:30:20 (1718929820) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2435 0x983 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2435 0x983 0 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 20:30:24 (1718929824) llite.lustre-ffff8800a9905800.xattr_cache=1 llite.lustre-ffff8800b3a56800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504963 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-06-20 20:30:24.000000000 -0400 Modify: 2024-06-20 20:30:24.000000000 -0400 Change: 2024-06-20 20:30:24.000000000 -0400 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504966 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-06-20 20:30:30.000000000 -0400 Modify: 2024-06-20 20:30:30.000000000 -0400 Change: 2024-06-20 20:30:30.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504969 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-06-20 20:30:35.000000000 -0400 Modify: 2024-06-20 20:30:35.000000000 -0400 Change: 2024-06-20 20:30:35.000000000 -0400 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-06-20 20:30:35.000000000 -0400 Modify: 2024-06-20 20:30:35.000000000 -0400 Change: 2024-06-20 20:30:35.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-06-20 20:30:40.000000000 -0400 Modify: 2024-06-20 20:30:41.000000000 -0400 Change: 2024-06-20 20:30:41.000000000 -0400 Birth: - File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504972 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-06-20 20:30:40.000000000 -0400 Modify: 2024-06-20 20:30:41.000000000 -0400 Change: 2024-06-20 20:30:41.000000000 -0400 Birth: - PASS 54 (23s) == sanityn test 55a: rename vs unlink target dir ========= 20:30:47 (1718929847) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 20:30:54 (1718929854) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 20:31:01 (1718929861) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 20:31:13 (1718929873) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (8s) == sanityn test 60: Verify data_version behaviour ======== 20:31:21 (1718929881) PASS 60 (2s) == sanityn test 70a: cd directory && rm directory ======== 20:31:23 (1718929883) PASS 70a (2s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 20:31:25 (1718929885) PASS 70b (1s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 20:31:26 (1718929886) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00275643 s, 23.8 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 20:31:27 (1718929887) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00286217 s, 22.9 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 20:31:28 (1718929888) llite.lustre-ffff8800a9905800.xattr_cache=1 llite.lustre-ffff8800b3a56800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (1s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 20:31:30 (1718929890) llite.lustre-ffff8800a9905800.xattr_cache=1 llite.lustre-ffff8800b3a56800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (1s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 20:31:31 (1718929891) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0025642 s, 399 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00225722 s, 454 kB/s 2634: get lock2 2635: get lock1 2635: done 2635: get lock2 2634: done 2634: get lock1 fcntl cmd 7 failed: Resource deadlock avoided 2635: cannot get lock2: Resource deadlock avoided 2635: exit rc=0 fcntl cmd 7 failed: Resource deadlock avoided 2634: cannot get lock1: Resource deadlock avoided 2634: done 2634: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 20:31:37 (1718929897) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0136895 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00618676 s, 169 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3681: 3269 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (6s) == sanityn test 76: Verify MDT open_files listing ======== 20:31:43 (1718929903) opening files: ........ closing files: ........ PASS 76 (15s) == sanityn test 77a: check FIFO NRS policy =============== 20:31:58 (1718929918) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.100455 s, 167 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00898004 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00951014 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00898571 s, 117 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0120257 s, 87.2 MB/s , 0.00747312 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327304 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331066 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253498 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338761 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307931 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0385783 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104311 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0380897 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288889 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398255 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115303 s, 90.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3a56800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0020792 s, 504 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00971012 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00277078 s, 378 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00149473 s, 702 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116484 s, 90.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00892943 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00935357 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00898251 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00663658 s, 158 MB/s 1048576 bytes (1.0 MB) copied, 0.00585266 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00924801 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122765 s, 85.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109598 s, 95.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115466 s, 90.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119523 s, 87.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128336 s, 81.7 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg237-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) == sanityn test 77b: check CRR-N NRS policy ============== 20:32:00 (1718929920) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0938126 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00838213 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125839 s, 83.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171914 s, 61.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163492 s, 64.1 MB/s , 0.020813 s, 50.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149996 s, 69.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124572 s, 84.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172626 s, 60.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167552 s, 62.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202071 s, 51.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0318279 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0410678 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184293 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271873 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357739 s, 29.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3a56800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3a56800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294408 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00893486 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00825359 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127502 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00859192 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139549 s, 75.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104933 s, 99.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110978 s, 94.5 MB/s , 0.00941034 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104227 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113596 s, 92.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0156279 s, 67.1 MB/s , 0.014561 s, 72.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011351 s, 92.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110844 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128684 s, 81.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129177 s, 81.2 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg237-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.105214 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011136 s, 94.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127564 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274836 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292779 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295025 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364597 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275231 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319688 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034642 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0375154 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317374 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.035299 s, 29.7 MB/s , 0.036401 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0415787 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0453104 s, 23.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3a56800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3a56800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0438787 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00712889 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00138674 s, 756 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00623066 s, 168 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00646134 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00866403 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111818 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00778733 s, 135 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00636905 s, 165 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00895944 s, 117 MB/s , 0.00891131 s, 118 MB/s 1048576 bytes (1.0 MB) copied, 0.002388 s, 439 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00967417 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00943335 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101056 s, 104 MB/s 1048576 bytes (1.0 MB) copied, 0.0116445 s, 90.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3a56800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112226 s, 93.4 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg237-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) == sanityn test 77c: check ORR NRS policy ================ 20:32:03 (1718929923) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.138637 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00903873 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139702 s, 75.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154386 s, 67.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107262 s, 97.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192453 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306269 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159843 s, 65.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199741 s, 52.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3a56800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208722 s, 50.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0311201 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0380648 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0458309 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0403991 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0502025 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0439008 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030944 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00202353 s, 518 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00863738 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124866 s, 84.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00990222 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133349 s, 78.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00976389 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00988402 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117815 s, 89.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113633 s, 92.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117251 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00882917 s, 119 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0119403 s, 87.8 MB/s , 0.0130419 s, 80.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123931 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135462 s, 77.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137415 s, 76.3 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3a56800.lock_unused_count=2 removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg237-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.09887 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00891881 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161858 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306517 s, 34.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0253914 s, 41.3 MB/s , 0.0277421 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03485 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354209 s, 29.6 MB/s , 0.0299015 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.032756 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166199 s, 63.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265297 s, 39.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245022 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353579 s, 29.7 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.025906 s, 40.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0281901 s, 37.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3a56800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3a56800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286785 s, 36.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00772091 s, 136 MB/s , 0.00740573 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00454569 s, 231 MB/s , 0.00457458 s, 229 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00321685 s, 326 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00587427 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00886612 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00128617 s, 815 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121888 s, 86.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00204101 s, 514 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0100849 s, 104 MB/s 1048576 bytes (1.0 MB) copied, 0.00346743 s, 302 MB/s , 0.00166745 s, 629 MB/s 1048576 bytes (1.0 MB) copied, 0.00970855 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0015598 s, 672 MB/s 1048576 bytes (1.0 MB) copied, 0.00458617 s, 229 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3a56800.lock_unused_count=1 removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg237-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0931933 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00874846 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152637 s, 68.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153838 s, 68.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103578 s, 101 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131112 s, 80.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0227623 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228649 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145984 s, 71.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015214 s, 68.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184379 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206735 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0412753 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279908 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0529647 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.043561 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0594671 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00475524 s, 221 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00577311 s, 182 MB/s 1048576 bytes (1.0 MB) copied, 0.0127844 s, 82.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00737282 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00206514 s, 508 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00401325 s, 261 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00167837 s, 625 MB/s , 0.00458967 s, 228 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00155934 s, 672 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00160558 s, 653 MB/s 1048576 bytes (1.0 MB) copied, 0.0149459 s, 70.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00469053 s, 224 MB/s , 0.00262956 s, 399 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00406786 s, 258 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3a56800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00163428 s, 642 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00177943 s, 589 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg237-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (4s) == sanityn test 77d: check TRR nrs policy ================ 20:32:07 (1718929927) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.115402 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139322 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162146 s, 64.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285788 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395254 s, 26.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0436196 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297602 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0414862 s, 25.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0380088 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0379564 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464576 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0408992 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0424033 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.055072 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0513907 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0462311 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0542281 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00768224 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106026 s, 98.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00751825 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00944906 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00893843 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0090321 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00950107 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00997365 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0099845 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109997 s, 95.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118534 s, 88.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118686 s, 88.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114522 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123379 s, 85.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122769 s, 85.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127597 s, 82.2 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg237-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.119757 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00776111 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117068 s, 89.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207131 s, 50.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181719 s, 57.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0169571 s, 61.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00610446 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191765 s, 54.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0262273 s, 40.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0245284 s, 42.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0189851 s, 55.2 MB/s , 0.036646 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00723341 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0310758 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251446 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248453 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165765 s, 63.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3a56800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3a56800.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00957956 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984255 s, 107 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00905672 s, 116 MB/s , 0.0108845 s, 96.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0100331 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00923424 s, 114 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00703219 s, 149 MB/s , 0.0104695 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00845873 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00849411 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00180122 s, 582 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00324925 s, 323 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985211 s, 106 MB/s , 0.00780476 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100044 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00160019 s, 655 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg237-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0890265 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00859098 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162817 s, 64.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018192 s, 57.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191641 s, 54.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238687 s, 43.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0255289 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212973 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277707 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146338 s, 71.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182974 s, 57.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331917 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275694 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0325038 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014836 s, 70.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3a56800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3a56800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238762 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00797936 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00168798 s, 621 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00697685 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00841026 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00229677 s, 457 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00831038 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00854334 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00974078 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107142 s, 97.9 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0001-osc-ffff8800b3a56800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010631 s, 98.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0108991 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010073 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00162434 s, 646 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112766 s, 93.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114334 s, 91.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00787329 s, 133 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg237-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (3s) == sanityn test 77e: check TBF NID nrs policy ============ 20:32:10 (1718929930) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.202.37@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.10438 s, 161 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00699661 s, 150 MB/s 1048576 bytes (1.0 MB) copied, 0.00766733 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00795452 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014448 s, 72.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170691 s, 61.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016225 s, 64.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195586 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107242 s, 97.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00701017 s, 150 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288225 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130131 s, 80.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0299797 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304407 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193398 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285371 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255391 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00778756 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00681537 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105463 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00782752 s, 134 MB/s , 0.00694591 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00857793 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00889515 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00790054 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00962974 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00991788 s, 106 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00980968 s, 107 MB/s , 0.0099921 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102542 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104097 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105936 s, 99.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121281 s, 86.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg237-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.111854 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00841636 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01148 s, 91.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013882 s, 75.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109449 s, 95.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304334 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315495 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0244382 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288368 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356691 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0355891 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352641 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0415352 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331711 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0486272 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0501751 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0397667 s, 26.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00694531 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00699373 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918297 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00826695 s, 127 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110558 s, 94.8 MB/s , 0.00841387 s, 125 MB/s 1048576 bytes (1.0 MB) copied, 0.00851962 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00912398 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00977635 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00966625 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100033 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0098183 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107773 s, 97.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00999693 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100068 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115573 s, 90.7 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg237-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.107719 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00840926 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106309 s, 98.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137404 s, 76.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265086 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125162 s, 83.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239528 s, 43.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0224536 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306167 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306939 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017809 s, 58.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240706 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020933 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353678 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0313228 s, 33.5 MB/s , 0.0248496 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277986 s, 37.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3a56800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3a56800.lock_unused_count=4 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00810438 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116309 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00854782 s, 123 MB/s , 0.0100318 s, 105 MB/s 1048576 bytes (1.0 MB) copied, 0.00950336 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108893 s, 96.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010791 s, 97.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985022 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117148 s, 89.5 MB/s , 0.0100966 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111004 s, 94.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102706 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108143 s, 97.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3a56800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109062 s, 96.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114367 s, 91.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121878 s, 86.0 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg237-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0961332 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00673333 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107047 s, 98.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0190567 s, 55.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0210366 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288571 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0270046 s, 38.8 MB/s , 0.025633 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.022352 s, 46.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0167831 s, 62.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128226 s, 81.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016771 s, 62.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.027064 s, 38.7 MB/s , 0.0243849 s, 43.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0317286 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215889 s, 48.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217537 s, 48.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3a56800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3a56800.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00768772 s, 136 MB/s 1048576 bytes (1.0 MB) copied, 0.00782529 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00920425 s, 114 MB/s , 0.00308664 s, 340 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00956201 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00123631 s, 848 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00581338 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0062937 s, 167 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112422 s, 93.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110016 s, 95.3 MB/s 1048576 bytes (1.0 MB) copied, 0.00277839 s, 377 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152976 s, 68.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112901 s, 92.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00161545 s, 649 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00493724 s, 212 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00153486 s, 683 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg237-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (6s) == sanityn test 77f: check TBF JobID nrs policy ========== 20:32:16 (1718929936) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg237-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.127363 s, 132 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00743415 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128102 s, 81.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152113 s, 68.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170577 s, 61.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159426 s, 65.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00779727 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00773011 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270712 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259323 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290675 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0418587 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352917 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.028738 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0456752 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0223615 s, 46.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0418601 s, 25.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00804331 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00836935 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00820214 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00905358 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103428 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118191 s, 88.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101031 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010597 s, 99.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110883 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113063 s, 92.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111981 s, 93.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123316 s, 85.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120328 s, 87.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0127378 s, 82.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147434 s, 71.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142707 s, 73.5 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg237-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg237-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.123004 s, 136 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.007631 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117411 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281484 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360427 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285836 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338246 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0374773 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.031954 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0384495 s, 27.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0443049 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457442 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342849 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331649 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390848 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.048363 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0435805 s, 24.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00720872 s, 145 MB/s , 0.00619477 s, 169 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00893848 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00839397 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00834114 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00894735 s, 117 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00838203 s, 125 MB/s , 0.00953117 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00882454 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100705 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00898045 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00982113 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00929931 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121819 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110279 s, 95.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108309 s, 96.8 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg237-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg237-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0948402 s, 177 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972106 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157943 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273405 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289176 s, 36.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0344196 s, 30.5 MB/s , 0.0224407 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293239 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300234 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341705 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293037 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199302 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0296389 s, 35.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026251 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209524 s, 50.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321431 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319729 s, 32.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104753 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106753 s, 98.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00844242 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00886 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00932648 s, 112 MB/s 1048576 bytes (1.0 MB) copied, 0.013668 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00840295 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0086684 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100493 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106564 s, 98.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00955995 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011868 s, 88.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103304 s, 102 MB/s , 0.0145299 s, 72.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123317 s, 85.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123124 s, 85.2 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg237-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg237-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0909878 s, 184 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011051 s, 94.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011806 s, 88.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117883 s, 89.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208857 s, 50.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175622 s, 59.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022754 s, 46.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133696 s, 78.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0147581 s, 71.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225885 s, 46.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163751 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243581 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126074 s, 83.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183633 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265227 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156061 s, 67.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3a56800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213276 s, 49.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00772835 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00879483 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00150808 s, 695 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00773986 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00339279 s, 309 MB/s 1048576 bytes (1.0 MB) copied, 0.00344437 s, 304 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00933188 s, 112 MB/s , 0.00951554 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00190376 s, 551 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00121671 s, 862 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00463971 s, 226 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00876876 s, 120 MB/s , 0.00999732 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00962361 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.009993 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00998592 s, 105 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg237-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (5s) == sanityn test 77g: Change TBF type directly ============ 20:32:21 (1718929941) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (3s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 20:32:24 (1718929944) oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg237-client: oleg237-server: ssh exited with exit code 2 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg237-client: oleg237-server: ssh exited with exit code 12 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg237-client: oleg237-server: ssh exited with exit code 12 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg237-client: oleg237-server: ssh exited with exit code 12 PASS 77h (2s) == sanityn test 77i: Change rank of TBF rule ============= 20:32:26 (1718929946) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (6s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 20:32:32 (1718929952) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0940862 s, 178 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105684 s, 99.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117433 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00959137 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166161 s, 63.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153258 s, 68.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196552 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127155 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00732872 s, 143 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226089 s, 46.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0305987 s, 34.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0350641 s, 29.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0233531 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293146 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201383 s, 52.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266303 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211116 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00957437 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00797513 s, 131 MB/s 1048576 bytes (1.0 MB) copied, 0.00809885 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00917046 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00913189 s, 115 MB/s , 0.00878092 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00940306 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00950323 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101593 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106687 s, 98.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985389 s, 106 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109609 s, 95.7 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0117905 s, 88.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0109827 s, 95.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112598 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139755 s, 75.0 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg237-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94874 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4301 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (50s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 20:33:22 (1718930002) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg237-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.638322 s, 26.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.168857 s, 6.2 MB/s , 0.170722 s, 6.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.174453 s, 6.0 MB/s 1048576 bytes (1.0 MB) copied, 0.167996 s, 6.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.181159 s, 5.8 MB/s , 0.180001 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.186674 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.175441 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.36961 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.370773 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.390597 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.381364 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.393719 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.392646 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.397148 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.384137 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157955 s, 6.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153907 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155537 s, 6.7 MB/s 1048576 bytes (1.0 MB) copied, 0.157711 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156167 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.16112 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157775 s, 6.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158363 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159574 s, 6.6 MB/s 1048576 bytes (1.0 MB) copied, 0.159338 s, 6.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.159823 s, 6.6 MB/s , 0.159255 s, 6.6 MB/s 1048576 bytes (1.0 MB) copied, 0.158953 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159711 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.161987 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.161755 s, 6.5 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg237-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.837 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9576 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg237-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.646642 s, 25.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.163924 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.184879 s, 5.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.184691 s, 5.7 MB/s 1048576 bytes (1.0 MB) copied, 0.178141 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.189054 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.182923 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.190782 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.198682 s, 5.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.365531 s, 2.9 MB/s , 0.364137 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.380833 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.390303 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.383105 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.381777 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.393586 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.400877 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.150366 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148007 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147408 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148571 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149409 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154174 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15098 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149208 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149615 s, 7.0 MB/s , 0.148444 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151331 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151501 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151824 s, 6.9 MB/s 1048576 bytes (1.0 MB) copied, 0.152409 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152236 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15303 s, 6.9 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg237-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8322 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9657 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (130s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 20:35:32 (1718930132) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg237-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.196819 s, 85.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194372 s, 53.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119111 s, 88.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0286367 s, 36.6 MB/s , 0.0236297 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.039456 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234308 s, 44.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0394726 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356135 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0692753 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0827343 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0852423 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0881285 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0963942 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.098459 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.084077 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0809226 s, 13.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100066 s, 105 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00869587 s, 121 MB/s , 0.00695544 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100751 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00994058 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00838829 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00933943 s, 112 MB/s , 0.0043743 s, 240 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010083 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107634 s, 97.4 MB/s , 0.0102675 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104798 s, 100 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0105986 s, 98.9 MB/s , 0.0125381 s, 83.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120932 s, 86.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128902 s, 81.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg237-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.91828 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73274 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.37@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.37@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.145068 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107623 s, 97.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300442 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025136 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0323631 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271416 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241255 s, 43.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255776 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328338 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0620407 s, 16.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0844876 s, 12.4 MB/s , 0.0832502 s, 12.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0770936 s, 13.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.159762 s, 6.6 MB/s , 0.15747 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.250036 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.267126 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104658 s, 100 MB/s , 0.00937299 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115052 s, 91.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00686452 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00905872 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00930157 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115216 s, 91.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968059 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01034 s, 101 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113462 s, 92.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121067 s, 86.6 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0122737 s, 85.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0122832 s, 85.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119893 s, 87.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126761 s, 82.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127534 s, 82.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg237-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93102 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73022 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.37@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg237-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.142634 s, 118 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00768039 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163192 s, 64.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254589 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021903 s, 47.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331965 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338214 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316649 s, 33.1 MB/s , 0.0341904 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0660054 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0822345 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0773323 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0882904 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0838144 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0838808 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0809633 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0821416 s, 12.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0066822 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00777576 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00650592 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00902046 s, 116 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00893816 s, 117 MB/s , 0.00889673 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00925979 s, 113 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00934304 s, 112 MB/s , 0.00955879 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00864652 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00949162 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100029 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00929176 s, 113 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948584 s, 111 MB/s 1048576 bytes (1.0 MB) copied, 0.0106149 s, 98.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116767 s, 89.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg237-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93865 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.87897 s, 21.5 MB/s Read runtime is 5 s, speed is 20.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg237-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.2523 s, 66.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139418 s, 75.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0338523 s, 31.0 MB/s , 0.0257239 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191485 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03097 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0409399 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181054 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0339613 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.161597 s, 6.5 MB/s , 0.161578 s, 6.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.308124 s, 3.4 MB/s , 0.315587 s, 3.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.359385 s, 2.9 MB/s , 0.359987 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.353967 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.347562 s, 3.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00680303 s, 154 MB/s , 0.00829326 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00665602 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00651431 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115221 s, 91.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00772157 s, 136 MB/s , 0.00786741 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011464 s, 91.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00885106 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00995419 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103758 s, 101 MB/s , 0.0103303 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107519 s, 97.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106024 s, 98.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130054 s, 80.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012638 s, 83.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg237-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77803 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73136 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92803 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73181 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg237-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.645832 s, 26.0 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.159899 s, 6.6 MB/s , 0.165313 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.175043 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.164699 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.171501 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.165789 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.17425 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.167007 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.365058 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.372799 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.367223 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.383763 s, 2.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.383757 s, 2.7 MB/s , 0.394314 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.39464 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.380846 s, 2.8 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00566357 s, 185 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00843756 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00487358 s, 215 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00836124 s, 125 MB/s , 0.00550927 s, 190 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0104962 s, 99.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0120061 s, 87.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00521313 s, 201 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00711097 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00757163 s, 138 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00877431 s, 120 MB/s , 0.00877778 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948843 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00963622 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105174 s, 99.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107453 s, 97.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg237-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8335 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4298 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg237-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.147119 s, 114 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118657 s, 88.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0314045 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247532 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197557 s, 53.1 MB/s , 0.026843 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0351859 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335539 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018195 s, 57.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0640515 s, 16.4 MB/s , 0.0618634 s, 16.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0717133 s, 14.6 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0752465 s, 13.9 MB/s , 0.0670725 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0652702 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0883205 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0866766 s, 12.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0072911 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00644131 s, 163 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00798815 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00819378 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00980309 s, 107 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.011091 s, 94.5 MB/s , 0.00829215 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00822995 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00963812 s, 109 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0104524 s, 100 MB/s 1048576 bytes (1.0 MB) copied, 0.00912986 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00978185 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103831 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107814 s, 97.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0117121 s, 89.5 MB/s , 0.0126304 s, 83.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg237-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94741 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72957 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg237-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.25334 s, 66.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0093323 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00920454 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00947221 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184625 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00855275 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00615453 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117657 s, 89.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00513906 s, 204 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.16265 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.160586 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.172806 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.164324 s, 6.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.347093 s, 3.0 MB/s , 0.350141 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.366041 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.369988 s, 2.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0079267 s, 132 MB/s , 0.00755781 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101341 s, 103 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0105167 s, 99.7 MB/s , 0.00964096 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00994762 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00982378 s, 107 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00955768 s, 110 MB/s 1048576 bytes (1.0 MB) copied, 0.00972451 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112709 s, 93.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109076 s, 96.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out , 0.0123636 s, 84.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0117599 s, 89.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0114921 s, 91.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116323 s, 90.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141953 s, 73.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg237-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.79841 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73173 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77368 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73482 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (351s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 20:41:23 (1718930483) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (2s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 20:41:25 (1718930485) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg237-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.53767 s, 296 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg237-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0197 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 20:42:15 (1718930535) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.10248 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00775763 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134323 s, 78.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00929677 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181979 s, 57.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172814 s, 60.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162861 s, 64.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113809 s, 92.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.035594 s, 29.5 MB/s , 0.0338926 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165893 s, 63.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353329 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241636 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0394015 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0325436 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0344896 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0359624 s, 29.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3a56800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3a56800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00807102 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00839228 s, 125 MB/s 1048576 bytes (1.0 MB) copied, 0.00929963 s, 113 MB/s , 0.00850579 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101909 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012814 s, 81.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00929445 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00461534 s, 227 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010555 s, 99.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0128613 s, 81.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0104331 s, 101 MB/s , 0.0111246 s, 94.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119086 s, 88.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116132 s, 90.3 MB/s , 0.0117206 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010086 s, 104 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg237-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95127 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97348 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.189493 s, 88.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010123 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100089 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00824846 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149536 s, 70.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134646 s, 77.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142725 s, 73.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115218 s, 91.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102654 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.070899 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0674139 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0886552 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0896342 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.164744 s, 6.4 MB/s , 0.167251 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.2721 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.25898 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148325 s, 70.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138056 s, 76.0 MB/s , 0.0125363 s, 83.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133935 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123896 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115561 s, 90.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187653 s, 55.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0121429 s, 86.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126122 s, 83.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139053 s, 75.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014473 s, 72.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146481 s, 71.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150237 s, 69.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162591 s, 64.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0168772 s, 62.1 MB/s , 0.0169245 s, 62.0 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg237-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg237-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92095 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97093 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (62s) == sanityn test 77o: Changing rank should not panic ====== 20:43:17 (1718930597) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (3s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 20:43:20 (1718930600) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (26s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 20:43:46 (1718930626) mds.MDS.mdt.nrs_policies=tbf oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg237-client: oleg237-server: ssh exited with exit code 36 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg237-client: oleg237-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (9s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 20:43:55 (1718930635) oleg237-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: Resource temporarily unavailable pdsh@oleg237-client: oleg237-server: ssh exited with exit code 11 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (2s) == sanityn test 79: xattr: intent error ================== 20:43:57 (1718930637) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505430 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-06-20 20:43:58.000000000 -0400 Modify: 2024-06-20 20:43:58.000000000 -0400 Change: 2024-06-20 20:43:58.000000000 -0400 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 20:44:09 (1718930649) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 20:44:10 (1718930650) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 20:44:11 (1718930651) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (0s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 20:44:11 (1718930651) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 20:44:12 (1718930652) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 20:44:13 (1718930653) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6872 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6872 PASS 82 (1s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 20:44:15 (1718930655) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (0s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 20:44:15 (1718930655) fail_loc=0x8000060b PASS 84 (7s) == sanityn test 90: open/create and unlink striped directory ========================================================== 20:44:22 (1718930662) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 20:44:23 (1718930663) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 20:44:24 (1718930664) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 20:44:25 (1718930665) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2643 0xa53 0 1 2630 0xa46 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2644 0xa54 0 1 2631 0xa47 0 rc1=0 and rc2=0 PASS 93 (8s) == sanityn test 94: signal vs CP callback race =========== 20:44:33 (1718930673) osc.lustre-OST0000-osc-ffff8800a9905800.idle_timeout=0 osc.lustre-OST0000-osc-ffff8800b3a56800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800a9905800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b3a56800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0406186 s, 202 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 3.00123 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5142: 3858 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0147134 s, 278 kB/s osc.lustre-OST0000-osc-ffff8800a9905800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b3a56800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800a9905800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b3a56800.idle_timeout=debug PASS 94 (10s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 20:44:43 (1718930683) llite.lustre-ffff8800a9905800.fast_read=0 llite.lustre-ffff8800b3a56800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00171619 s, 7.2 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800a9905800.fast_read=1 llite.lustre-ffff8800b3a56800.fast_read=1 PASS 95b (8s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 20:44:51 (1718930691) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 20:44:52 (1718930692) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00135758 s, 3.0 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 20:44:54 (1718930694) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0286654 s, 73.2 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 20:44:56 (1718930696) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0257635 s, 81.4 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (1s) == sanityn test 100e: DoM: read on open and file size ==== 20:44:57 (1718930697) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00148145 s, 8.1 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 20:44:59 (1718930699) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000511708 s, 8.0 MB/s PASS 101a (1s) == sanityn test 101b: Discard DoM data on rename ========= 20:45:01 (1718930701) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000536629 s, 7.6 MB/s PASS 101b (1s) == sanityn test 101c: Discard DoM data on close-unlink === 20:45:02 (1718930702) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000596286 s, 6.9 MB/s PASS 101c (3s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 20:45:05 (1718930705) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1718930706 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1718930708 Finishing test test23 at 1718930709 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 20:45:12 (1718930712) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (0s) == sanityn test 105: Glimpse and lock cancel race ======== 20:45:13 (1718930713) fail_loc=0x416 PASS 105 (48s) == sanityn test 106a: Verify the btime via statx() ======= 20:46:00 (1718930760) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (0s) == sanityn test 106b: Glimpse RPCs test for statx ======== 20:46:00 (1718930760) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) == sanityn test 106c: Verify statx attributes mask ======= 20:46:01 (1718930761) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) == sanityn test 107a: Basic grouplock conflict =========== 20:46:02 (1718930762) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0605688 s, 173 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2651 0xa5b 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6872 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6872 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 20:46:06 (1718930766) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0742503 s, 141 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2635 0xa4b 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6872 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (7s) == sanityn test 108a: lseek: parallel updates ============ 20:46:14 (1718930774) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00142451 s, 7.0 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.23974 s, 2.0 MB/s PASS 108a (5s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 20:46:19 (1718930779) 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions oleg237-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 6 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions oleg237-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 7 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions oleg237-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 8 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions oleg237-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 10 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions oleg237-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 14 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions oleg237-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 15 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions oleg237-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 18 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions oleg237-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 21 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions oleg237-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg237-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 24 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions oleg237-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 26 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions oleg237-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 31 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions oleg237-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 35 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions oleg237-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 41 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions oleg237-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 43 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions oleg237-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 45 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 192.168.202.137@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.202.137@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg237-server' oleg237-server: oleg237-server.virtnet: executing load_modules_local oleg237-server: Loading modules from /home/green/git/lustre-release/lustre oleg237-server: detected 4 online CPUs by sysfs oleg237-server: Force libcfs to create 2 CPU partitions oleg237-server: libkmod: kmod_module_get_refcnt: could not read integer from '/sys/module/intel_rapl/refcnt': 'No such device' oleg237-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre Starting client: oleg237-client.virtnet: -o user_xattr,flock oleg237-server@tcp:/lustre /mnt/lustre2 PASS 109 (445s) == sanityn test 110: do not grant another lock on resend ========================================================== 20:53:44 (1718931224) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-06-20 20:53:45.000000000 -0400 Modify: 2024-06-20 20:53:45.000000000 -0400 Change: 2024-06-20 20:53:45.000000000 -0400 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg237-client.virtnet oleg237-client.virtnet PASS 110 (36s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 20:54:20 (1718931260) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 20:54:22 (1718931262) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (0s) == sanityn test 113: check servers of specified fs ======= 20:54:22 (1718931262) lustre-OST0000-osc-ffff88009e8b8800 active. lustre-OST0001-osc-ffff88009e8b8800 active. lustre-OST0000-osc-ffff88009e8b8800 active. lustre-OST0001-osc-ffff88009e8b8800 active. MGC192.168.202.137@tcp active. PASS 113 (3s) cleanup: ====================================================== == sanityn test complete, duration 4091 sec ============== 20:54:26 (1718931266) Stopping clients: oleg237-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg237-client.virtnet /mnt/lustre2 opts: