-----============= acceptance-small: sanityn ============----- Wed Jan 15 18:37:24 EST 2025 excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 Started clients oleg458-client.virtnet: 192.168.204.158@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt) oleg458-client.virtnet: executing check_config_client /mnt/lustre oleg458-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg458-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800aa144800.idle_timeout=debug osc.lustre-OST0000-osc-ffff88013731d000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aa144800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88013731d000.idle_timeout=debug disable quota as required oleg458-server: oleg458-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102247 s, 103 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6873] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 18:37:34 (1736984254) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (3s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 18:37:37 (1736984257) -rw-r--r-- 1 root root 0 Jan 15 18:37 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (3s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 18:37:40 (1736984260) -rw-r--r-- 1 root root 0 Jan 15 18:37 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (4s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 18:37:44 (1736984264) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (3s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 18:37:47 (1736984267) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (3s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 18:37:50 (1736984270) total 1 -rwxrwxrwx 1 root root 0 Jan 15 18:37 f2a -rwxrwxrwx 1 root root 0 Jan 15 18:37 f2b total 1 -rwxrwxrwx 1 root root 0 Jan 15 18:37 f2a -rwxrwxrwx 1 root root 0 Jan 15 18:37 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00770104 s, 66.5 kB/s PASS 2e (3s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 18:37:53 (1736984273) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (2s) == sanityn test 2g: check blocks update on sync write ==== 18:37:55 (1736984275) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.266462 s, 7.9 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (3s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 18:37:58 (1736984278) PASS 3 (3s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 18:38:01 (1736984281) PASS 4 (4s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 18:38:05 (1736984285) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (3s) == sanityn test 6: remove of open file on other node ============================================================================ 18:38:08 (1736984288) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (4s) == sanityn test 7: remove of open directory on other node ======================================================================= 18:38:12 (1736984292) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (3s) == sanityn test 8: remove of open special file on other node ==================================================================== 18:38:15 (1736984295) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (3s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 18:38:18 (1736984298) PASS 9a (4s) == sanityn test 9b: append to striped sparse file ======== 18:38:22 (1736984302) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256406 s, 40.9 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0398441 s, 0.1 kB/s Data read (expecting 'foo'): foo PASS 9b (3s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 18:38:25 (1736984305) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00416569 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00591168 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00390633 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00437629 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00330944 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0040616 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00298864 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00608461 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00473796 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00504235 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0106186 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0110927 s, 0.1 kB/s PASS 10a (4s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 18:38:29 (1736984309) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.0135695 s, 226 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0227546 s, 180 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00101694 s, 3.0 MB/s PASS 10b (4s) == sanityn test 11: execution of file opened for write should return error ============================================================== 18:38:33 (1736984313) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6873 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (4s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 18:38:37 (1736984317) warning: '-runtime' deprecated, use '-t runtime' instead - link 707 (time 1736984328.14 total 10.01 last 70.66) - link 1391 (time 1736984338.14 total 20.01 last 68.34) - link 1999 (time 1736984348.16 total 30.03 last 60.72) - link 2636 (time 1736984358.16 total 40.03 last 63.69) - link 3335 (time 1736984368.17 total 50.04 last 69.84) - link 4131 (time 1736984378.17 total 60.04 last 79.57) - link 4898 (time 1736984388.18 total 70.05 last 76.63) - link 5711 (time 1736984398.19 total 80.06 last 81.25) - link 6585 (time 1736984408.19 total 90.06 last 87.40) total: 7545 link in 100.01 seconds: 75.44 ops/second using seed 3867410072 running for 100 seconds - stat 10000 (time 1736984324 ; total 5 ; last 5) - stat 20000 (time 1736984329 ; total 10 ; last 5) - stat 30000 (time 1736984334 ; total 15 ; last 5) - stat 40000 (time 1736984340 ; total 21 ; last 6) - stat 50000 (time 1736984345 ; total 26 ; last 5) - stat 60000 (time 1736984351 ; total 32 ; last 6) - stat 70000 (time 1736984356 ; total 37 ; last 5) - stat 80000 (time 1736984361 ; total 42 ; last 5) - stat 90000 (time 1736984366 ; total 47 ; last 5) - stat 100000 (time 1736984376 ; total 57 ; last 10) - stat 105717 (time 1736984387 ; total 68 ; last 11) - stat 110000 (time 1736984394 ; total 75 ; last 7) - stat 117049 (time 1736984405 ; total 86 ; last 11) - stat 120000 (time 1736984409 ; total 90 ; last 4) total: 126255 stats in 100 seconds: 1262.550049 stats/second - unlinked 0 (time 1736984369 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile7545) error: No such file or directory total: 7544 unlinks in 78 seconds: 96.717949 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (20496) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (20499) - No such process PASS 12 (132s) == sanityn test 13: test directory page revocation ======= 18:40:49 (1736984449) f13 f13 PASS 13 (4s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 18:40:54 (1736984454) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6873 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4260: 21872 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (3s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 18:40:57 (1736984457) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4260: 22502 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (3s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 18:41:00 (1736984460) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 23130 Terminated $DIR1/$tdir/sleep 60 PASS 14b (3s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 18:41:03 (1736984463) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 23760 Terminated $DIR1/$tdir/sleep 60 PASS 14c (3s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 18:41:06 (1736984466) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 24389 Terminated $DIR1/$tdir/sleep 60 PASS 14d (4s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 18:41:10 (1736984470) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7517184 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7517184kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (14s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 18:41:24 (1736984484) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.215109 s, 9.7 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.142999 s, 14.7 MB/s Chance of close/open is 1 in 50 Seed set to 4486 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x5e4297 000001[0] 1736984486.783611 trunc from 00000000 to 0x5e4296 (0x5e4297 bytes) 000005[0] 1736984486.828989 write 0x7ade09 thru 0x7b64b1 (0x86a9 bytes) 000007[0] 1736984486.832001 read 0x352027 thru 0x359ce3 (0x7cbd bytes) 000008[0] 1736984486.860282 mapread 0x2646eb thru 0x26db64 (0x947a bytes) 000009[0] 1736984486.893478 read 0x053128 thru 0x062438 (0xf311 bytes) 000010[0] 1736984486.944130 trunc from 0x7b64b2 to 0x2baa50 (0x4fba61 bytes) 000012[1] 1736984487.012165 write 0x296518 thru 0x296a9e (0x587 bytes) 000015[0] 1736984487.057635 write 0x978097 thru 0x97bb4f (0x3ab9 bytes) 000017[0] 1736984487.079917 read 0x40a48b thru 0x40e6e2 (0x4258 bytes) 000020[0] 1736984487.146689 trunc from 0x97bb50 to 0x58cfc6 (0x3eeb89 bytes) 000022[1] 1736984487.173345 write 0x4f72a1 thru 0x4f955a (0x22ba bytes) truncating to largest ever: 0x6b18d1 000024[1] 1736984487.232111 trunc from 0x58cfc7 to 0x6b18d0 (0x12490a bytes) 000025[0] 1736984487.278181 mapwrite 0x6bada5 thru 0x6be375 (0x35d1 bytes) 000028[1] 1736984487.421013 mapread 0x13ed3f thru 0x14d3a4 (0xe666 bytes) 000029[0] 1736984487.461888 mapread 0x5e4eb9 thru 0x5e8443 (0x358b bytes) 000030[0] 1736984487.482454 write 0x53cbe4 thru 0x54333e (0x675b bytes) 000031[1] 1736984487.513814 read 0x6a552c thru 0x6ad705 (0x81da bytes) truncating to largest ever: 0x8dfc14 000035[0] 1736984487.557124 trunc from 0x6be376 to 0x8dfc13 (0x22189e bytes) 000036[0] 1736984487.601916 write 0x199bb0 thru 0x19d223 (0x3674 bytes) 000037[1] 1736984487.618922 trunc from 0x8dfc14 to 0x0d8611 (0x807602 bytes) 000039[0] 1736984487.715369 write 0x31f979 thru 0x3223fa (0x2a82 bytes) 000040[1] 1736984487.737728 write 0x1eb083 thru 0x1f87f7 (0xd775 bytes) 000041[1] 1736984487.757764 write 0x549688 thru 0x557200 (0xdb79 bytes) 000043[0] 1736984487.771769 trunc from 0x557201 to 0x07eebc (0x4d8344 bytes) 000046[1] 1736984487.837446 mapread 0x070934 thru 0x07419b (0x3868 bytes) 000051[1] 1736984487.893770 mapread 0x05873a thru 0x068082 (0xf949 bytes) 000052[0] 1736984487.917720 mapread 0x0081b1 thru 0x00ff51 (0x7da1 bytes) 000053[1] 1736984487.951569 write 0x6bd457 thru 0x6c35cf (0x6179 bytes) 000057[1] 1736984488.008275 write 0x8ff566 thru 0x9013e6 (0x1e81 bytes) 000059[1] 1736984488.024852 write 0x4d5b98 thru 0x4e526d (0xf6d6 bytes) 000061[1] 1736984488.054564 trunc from 0x9013e7 to 0x2740ee (0x68d2f8 bytes) 000069[1] 1736984488.129095 write 0x896d2e thru 0x8a2002 (0xb2d5 bytes) 000070[0] 1736984488.131517 mapread 0x002143 thru 0x00e19c (0xc05a bytes) 000071[0] 1736984488.215536 write 0x0b38f3 thru 0x0c05d2 (0xcce0 bytes) 000073[0] 1736984488.263624 write 0x315b6c thru 0x31adf1 (0x5286 bytes) 000079[1] 1736984488.326644 read 0x19ad4e thru 0x19cce0 (0x1f93 bytes) 000080[1] 1736984488.357759 read 0x6f33a7 thru 0x6fee00 (0xba5a bytes) 000082[0] 1736984488.401852 trunc from 0x8a2003 to 0x459fef (0x448013 bytes) 000084[0] 1736984488.450575 trunc from 0x459ff0 to 0x27a1e7 (0x1dfe08 bytes) 000085[1] 1736984488.479208 mapread 0x2786db thru 0x27a1e7 (0x1b0d bytes) 000087[1] 1736984488.517859 mapwrite 0x36f5f8 thru 0x373c73 (0x467c bytes) 000089[0] 1736984488.622412 read 0x2a3077 thru 0x2ae9c8 (0xb952 bytes) 000093[1] 1736984488.667865 write 0x1614c0 thru 0x16fbec (0xe72d bytes) 000094[1] 1736984488.686408 write 0x46afc0 thru 0x47917e (0xe1bf bytes) 000096[1] 1736984488.725604 mapread 0x047788 thru 0x04e2ba (0x6b33 bytes) 000097[1] 1736984488.762075 read 0x2161b3 thru 0x21bb44 (0x5992 bytes) 000099[1] 1736984488.802378 trunc from 0x47917f to 0x105857 (0x373927 bytes) 000103[0] 1736984488.853999 mapwrite 0x6b3373 thru 0x6c1fcc (0xec5a bytes) 000104[0] 1736984489.000886 mapread 0x047976 thru 0x052506 (0xab91 bytes) 000105[1] 1736984489.055124 mapwrite 0x94feb6 thru 0x9546bb (0x4806 bytes) 000108[0] 1736984489.235224 mapwrite 0x228adf thru 0x231825 (0x8d47 bytes) 000109[0] 1736984489.329217 read 0x716b25 thru 0x71feb3 (0x938f bytes) 000110[1] 1736984489.386304 mapread 0x15752a thru 0x15cd67 (0x583e bytes) 000111[1] 1736984489.412028 mapread 0x1703a8 thru 0x1730c5 (0x2d1e bytes) 000112[1] 1736984489.431286 mapread 0x2aa2b6 thru 0x2ae501 (0x424c bytes) 000113[0] 1736984489.469868 mapwrite 0x5dd535 thru 0x5e48e0 (0x73ac bytes) 000114[0] 1736984489.565518 mapread 0x469aab thru 0x475b72 (0xc0c8 bytes) 000115[0] 1736984489.620972 mapread 0x3e8721 thru 0x3ef6ff (0x6fdf bytes) 000120[0] 1736984489.672114 trunc from 0x9546bc to 0x1bd629 (0x797092 bytes) 000122[0] 1736984489.705581 trunc from 0x1bd62a to 0x468181 (0x2aab58 bytes) 000130[1] 1736984489.787714 read 0x37f0f1 thru 0x3822e2 (0x31f2 bytes) 000131[0] 1736984489.814444 mapwrite 0x4faabf thru 0x503849 (0x8d8b bytes) 000132[0] 1736984489.952794 mapread 0x4b6780 thru 0x4c5fe5 (0xf866 bytes) 000134[0] 1736984490.040907 mapwrite 0x5e77ff thru 0x5f3692 (0xbe94 bytes) 000136[1] 1736984490.214270 write 0x722ea1 thru 0x732988 (0xfae8 bytes) 000137[0] 1736984490.241739 read 0x2e644e thru 0x2f4b11 (0xe6c4 bytes) 000138[1] 1736984490.258091 read 0x45448c thru 0x464114 (0xfc89 bytes) 000141[0] 1736984490.311287 write 0x826a4e thru 0x82f5a8 (0x8b5b bytes) 000143[1] 1736984490.360729 trunc from 0x82f5a9 to 0x67ddc5 (0x1b17e3 bytes) 000144[0] 1736984490.392357 trunc from 0x67ddc6 to 0x137872 (0x546553 bytes) 000146[0] 1736984490.417094 write 0x0ea3aa thru 0x0ec237 (0x1e8e bytes) 000150[1] 1736984490.452731 read 0x0c8c68 thru 0x0d130e (0x86a7 bytes) 000151[0] 1736984490.475879 write 0x7daec9 thru 0x7dbd26 (0xe5e bytes) 000157[1] 1736984490.517570 read 0x745022 thru 0x7476e8 (0x26c7 bytes) 000158[1] 1736984490.578604 trunc from 0x7dbd27 to 0x008ea9 (0x7d2e7d bytes) 000162[1] 1736984490.685806 mapwrite 0x8e45e6 thru 0x8eab34 (0x654f bytes) 000163[1] 1736984490.814690 write 0x8971ca thru 0x8a3ec3 (0xccfa bytes) 000164[1] 1736984490.815572 trunc from 0x8eab35 to 0x58ed75 (0x35bdbf bytes) 000166[0] 1736984490.867496 mapwrite 0x703b7e thru 0x70b306 (0x7789 bytes) 000169[1] 1736984491.103016 mapwrite 0x8a6488 thru 0x8ab00a (0x4b83 bytes) 000172[0] 1736984491.252094 mapwrite 0x46d1b4 thru 0x46f02a (0x1e77 bytes) 000173[0] 1736984491.333972 mapread 0x0cd979 thru 0x0d8734 (0xadbc bytes) 000174[1] 1736984491.380696 mapwrite 0x16a932 thru 0x17573b (0xae0a bytes) 000175[1] 1736984491.483412 mapwrite 0x3d8df9 thru 0x3e7557 (0xe75f bytes) 000177[1] 1736984491.620196 write 0x597552 thru 0x5a6a48 (0xf4f7 bytes) 000178[1] 1736984491.637013 trunc from 0x8ab00b to 0x140d92 (0x76a278 bytes) 000179[0] 1736984491.670032 mapread 0x102c9c thru 0x10a6fb (0x7a60 bytes) 000180[1] 1736984491.712395 mapread 0x0dbcf6 thru 0x0dcaaf (0xdba bytes) 000181[1] 1736984491.726148 trunc from 0x140d93 to 0x2f18d5 (0x1b0b43 bytes) 000182[1] 1736984491.762068 trunc from 0x2f18d6 to 0x35c74c (0x6ae77 bytes) 000184[0] 1736984491.785167 read 0x0ced0f thru 0x0d6143 (0x7435 bytes) 000186[0] 1736984491.803545 read 0x0d65a6 thru 0x0e1214 (0xac6f bytes) 000187[1] 1736984491.820335 read 0x2a8918 thru 0x2ac6fe (0x3de7 bytes) 000190[1] 1736984491.845351 mapread 0x291952 thru 0x2949e8 (0x3097 bytes) 000192[1] 1736984491.876885 read 0x304dc5 thru 0x313b61 (0xed9d bytes) 000194[1] 1736984491.877524 write 0x221c54 thru 0x22e997 (0xcd44 bytes) 000198[1] 1736984491.925381 write 0x4fc0a7 thru 0x504ade (0x8a38 bytes) 000207[0] 1736984491.954409 read 0x4e11b6 thru 0x4e7e60 (0x6cab bytes) 000209[0] 1736984491.987113 mapread 0x09d0de thru 0x0a8c3b (0xbb5e bytes) 000210[1] 1736984492.036496 trunc from 0x504adf to 0x59f892 (0x9adb4 bytes) 000212[1] 1736984492.057657 trunc from 0x59f893 to 0x724f6d (0x1856db bytes) 000216[1] 1736984492.101511 mapwrite 0x7d0cf4 thru 0x7da19e (0x94ab bytes) 000218[1] 1736984492.244160 write 0x4c62d9 thru 0x4d2bad (0xc8d5 bytes) 000219[0] 1736984492.253391 mapwrite 0x28c3d0 thru 0x298fc4 (0xcbf5 bytes) 000220[0] 1736984492.387464 mapread 0x0129c0 thru 0x01cf15 (0xa556 bytes) 000223[1] 1736984492.465235 trunc from 0x7da19f to 0x484d79 (0x355425 bytes) 000226[0] 1736984492.517689 trunc from 0x484d7a to 0x7a591c (0x320ba3 bytes) 000229[1] 1736984492.551230 trunc from 0x7a591d to 0x29af17 (0x50aa05 bytes) 000231[1] 1736984492.600757 mapwrite 0x232265 thru 0x23e5bf (0xc35b bytes) 000232[0] 1736984492.692105 write 0x0bd9ef thru 0x0c4f9e (0x75b0 bytes) 000233[0] 1736984492.741762 mapread 0x14aac7 thru 0x14cecb (0x2405 bytes) 000234[1] 1736984492.774024 mapread 0x04f970 thru 0x053e1c (0x44ad bytes) 000236[0] 1736984492.813309 write 0x3623f4 thru 0x365702 (0x330f bytes) 000237[0] 1736984492.828625 trunc from 0x365703 to 0x82fe8e (0x4ca78c bytes) 000238[1] 1736984492.850572 trunc from 0x82fe8f to 0x894fe1 (0x65153 bytes) 000239[1] 1736984492.898895 write 0x51d89f thru 0x52546e (0x7bd0 bytes) 000240[0] 1736984492.923509 mapwrite 0x905034 thru 0x9098b0 (0x487d bytes) 000241[1] 1736984493.045685 read 0x0d51b6 thru 0x0df694 (0xa4df bytes) 000242[1] 1736984493.091730 mapwrite 0x8eb65b thru 0x8fb0b8 (0xfa5e bytes) 000243[0] 1736984493.242094 read 0x2d4e70 thru 0x2de1be (0x934f bytes) 000245[1] 1736984493.323261 write 0x11e48d thru 0x123b93 (0x5707 bytes) 000246[0] 1736984493.422067 mapwrite 0x592c39 thru 0x5a049c (0xd864 bytes) 000247[1] 1736984493.591147 trunc from 0x9098b1 to 0x350970 (0x5b8f40 bytes) 000248[1] 1736984493.649753 trunc from 0x350971 to 0x24584f (0x10b121 bytes) 000249[1] 1736984493.679020 trunc from 0x245850 to 0x02013c (0x225713 bytes) 000250[1] 1736984493.690851 read 0x017d9e thru 0x02013c (0x839f bytes) 000251[0] 1736984493.715588 mapwrite 0x7b081c thru 0x7bbfea (0xb7cf bytes) 000253[0] 1736984493.855184 write 0x76a780 thru 0x76b8c8 (0x1149 bytes) 000255[0] 1736984493.862221 read 0x5dea5d thru 0x5e886a (0x9e0e bytes) 000256[1] 1736984493.891749 trunc from 0x7bbfeb to 0x830e50 (0x74e66 bytes) 000257[0] 1736984493.955083 write 0x1b5c01 thru 0x1c4f82 (0xf382 bytes) 000258[1] 1736984493.982373 read 0x14c7e5 thru 0x1593ef (0xcc0b bytes) 000259[0] 1736984494.076718 trunc from 0x830e51 to 0x5cd6aa (0x2637a6 bytes) 000261[1] 1736984494.138191 write 0x51fb35 thru 0x52df34 (0xe400 bytes) 000262[1] 1736984494.163954 read 0x3cd2f5 thru 0x3d746d (0xa179 bytes) 000264[1] 1736984494.187761 write 0x2a3c8f thru 0x2af423 (0xb795 bytes) 000265[0] 1736984494.230107 read 0x245839 thru 0x24c76a (0x6f32 bytes) 000266[0] 1736984494.288394 mapread 0x1e4a53 thru 0x1e5573 (0xb21 bytes) 000267[0] 1736984494.349002 write 0x017e85 thru 0x018971 (0xaed bytes) 000268[1] 1736984494.376012 read 0x26ffc4 thru 0x27fa4b (0xfa88 bytes) 000272[1] 1736984494.440195 trunc from 0x5cd6ab to 0x0daceb (0x4f29bf bytes) 000273[1] 1736984494.480242 mapread 0x08786b thru 0x08c511 (0x4ca7 bytes) 000274[1] 1736984494.531111 write 0x0cbb56 thru 0x0cff23 (0x43ce bytes) 000275[1] 1736984494.541839 mapread 0x0a2234 thru 0x0a81fc (0x5fc9 bytes) 000277[0] 1736984494.594082 trunc from 0x0dacec to 0x39f993 (0x2c4ca8 bytes) 000278[1] 1736984494.643742 write 0x9cbc60 thru 0x9cfe55 (0x41f6 bytes) 000279[1] 1736984494.672443 mapwrite 0x740dda thru 0x74f1e1 (0xe408 bytes) 000283[1] 1736984494.880347 mapwrite 0x15333a thru 0x160f0e (0xdbd5 bytes) 000284[1] 1736984495.059855 mapwrite 0x1ecee5 thru 0x1ef1ae (0x22ca bytes) 000285[1] 1736984495.146473 mapwrite 0x640954 thru 0x649d9a (0x9447 bytes) 000287[1] 1736984495.257219 read 0x6f611d thru 0x6f85f6 (0x24da bytes) 000288[1] 1736984495.257826 trunc from 0x9cfe56 to 0x7501e5 (0x27fc70 bytes) 000291[1] 1736984495.320919 mapwrite 0x54cac6 thru 0x5591f2 (0xc72d bytes) 000292[1] 1736984495.454321 write 0x9b53a2 thru 0x9bdc7d (0x88dc bytes) 000296[1] 1736984495.477579 mapread 0x3574fe thru 0x35847c (0xf7f bytes) 000297[0] 1736984495.506785 mapread 0x280fd2 thru 0x28dfea (0xd019 bytes) 000298[0] 1736984495.576539 write 0x497266 thru 0x498670 (0x140b bytes) 000303[1] 1736984495.665654 read 0x1292dc thru 0x12e6d1 (0x53f6 bytes) 000306[1] 1736984495.722275 mapread 0x426442 thru 0x4294e0 (0x309f bytes) 000310[1] 1736984495.848773 read 0x87a7de thru 0x88922b (0xea4e bytes) 000311[0] 1736984495.859966 write 0x76ef7d thru 0x77b854 (0xc8d8 bytes) 000312[0] 1736984495.919166 trunc from 0x9bdc7e to 0x2648e3 (0x75939a bytes) 000313[1] 1736984495.961351 trunc from 0x2648e4 to 0x502bd9 (0x29e2f6 bytes) 000314[1] 1736984495.986302 read 0x0ebbfc thru 0x0f32b6 (0x76bb bytes) 000316[0] 1736984496.024969 trunc from 0x502bda to 0x792b10 (0x28ff37 bytes) 000317[0] 1736984496.073177 trunc from 0x792b11 to 0x17a766 (0x6183aa bytes) 000318[1] 1736984496.081126 trunc from 0x17a767 to 0x56376b (0x3e9005 bytes) 000319[1] 1736984496.130220 mapwrite 0x5e380a thru 0x5ee4ca (0xacc1 bytes) 000320[1] 1736984496.230346 read 0x07f15a thru 0x08a1da (0xb081 bytes) 000321[1] 1736984496.261977 mapread 0x439073 thru 0x43f98c (0x691a bytes) 000322[1] 1736984496.289019 trunc from 0x5ee4cb to 0x6fb736 (0x10d26c bytes) 000323[1] 1736984496.313483 mapwrite 0x567fb7 thru 0x568581 (0x5cb bytes) 000324[1] 1736984496.364875 mapread 0x0c0ca5 thru 0x0cf0b3 (0xe40f bytes) 000326[0] 1736984496.376877 trunc from 0x6fb737 to 0x6613a0 (0x9a396 bytes) 000327[0] 1736984496.428086 write 0x6cc7e7 thru 0x6d7f5b (0xb775 bytes) 000331[1] 1736984496.450780 mapwrite 0x8eeef3 thru 0x8f9dae (0xaebc bytes) 000332[0] 1736984496.585837 trunc from 0x8f9daf to 0x2e891c (0x611492 bytes) 000333[1] 1736984496.618374 write 0x239634 thru 0x2404d9 (0x6ea6 bytes) 000334[0] 1736984496.649192 read 0x1d3021 thru 0x1e0353 (0xd333 bytes) 000335[1] 1736984496.664823 trunc from 0x2e891d to 0x1b9d10 (0x12ec0c bytes) 000337[0] 1736984496.710898 mapread 0x190594 thru 0x195e28 (0x5895 bytes) 000340[0] 1736984496.762866 read 0x1694f6 thru 0x16ab57 (0x1662 bytes) 000341[0] 1736984496.770166 mapread 0x15d684 thru 0x16b862 (0xe1df bytes) 000342[1] 1736984496.840983 write 0x7a9198 thru 0x7b5c6d (0xcad6 bytes) 000343[1] 1736984496.853071 read 0x5bb3e5 thru 0x5c1ff1 (0x6c0d bytes) 000345[0] 1736984496.865626 mapwrite 0x7c4487 thru 0x7cc5b2 (0x812c bytes) 000346[1] 1736984497.009402 write 0x474df5 thru 0x47d993 (0x8b9f bytes) 000348[0] 1736984497.046175 mapread 0x7bb75c thru 0x7c5787 (0xa02c bytes) 000349[0] 1736984497.099647 trunc from 0x7cc5b3 to 0x5a5e62 (0x226750 bytes) 000351[1] 1736984497.159249 write 0x95cfaa thru 0x96ac82 (0xdcd9 bytes) 000353[1] 1736984497.186176 write 0x9b5349 thru 0x9c02a4 (0xaf5c bytes) 000355[0] 1736984497.204947 write 0x3acb0f thru 0x3b073b (0x3c2d bytes) 000357[0] 1736984497.240985 trunc from 0x9c02a5 to 0x21dece (0x7a23d6 bytes) 000358[0] 1736984497.255211 mapwrite 0x301c92 thru 0x30723e (0x55ad bytes) 000359[0] 1736984497.361377 read 0x06b227 thru 0x070e0d (0x5be7 bytes) 000360[0] 1736984497.426205 mapwrite 0x042efa thru 0x04bc4a (0x8d51 bytes) 000364[0] 1736984497.547389 write 0x4f5bc3 thru 0x4f7070 (0x14ae bytes) 000365[0] 1736984497.548976 trunc from 0x4f7071 to 0x444fca (0xb20a6 bytes) 000367[1] 1736984497.578094 write 0x8a2e91 thru 0x8b17c3 (0xe933 bytes) 000371[1] 1736984497.632498 read 0x116e9e thru 0x123999 (0xcafc bytes) 000372[0] 1736984497.722267 trunc from 0x8b17c4 to 0x2caebd (0x5e6906 bytes) 000374[1] 1736984497.796549 read 0x16953e thru 0x16d3c1 (0x3e84 bytes) 000375[1] 1736984497.817339 trunc from 0x2caebe to 0x0f63e0 (0x1d4add bytes) 000376[1] 1736984497.842706 mapread 0x0962f6 thru 0x0a15f1 (0xb2fc bytes) 000377[1] 1736984497.892976 write 0x24d03c thru 0x2543e2 (0x73a7 bytes) 000381[1] 1736984497.939369 trunc from 0x2543e3 to 0x10ef79 (0x145469 bytes) 000383[0] 1736984497.975714 read 0x0d1b1f thru 0x0ddec7 (0xc3a9 bytes) 000384[0] 1736984497.997997 write 0x4deb2c thru 0x4e7f46 (0x941b bytes) 000385[1] 1736984498.022944 read 0x15734a thru 0x1630d8 (0xbd8f bytes) 000386[1] 1736984498.088364 mapwrite 0x8fac89 thru 0x904b73 (0x9eeb bytes) 000390[0] 1736984498.374809 write 0x2d3aec thru 0x2e1775 (0xdc8a bytes) 000393[1] 1736984498.433859 mapwrite 0x20c2d6 thru 0x210f7d (0x4ca8 bytes) 000394[0] 1736984498.521772 read 0x3d5158 thru 0x3e4643 (0xf4ec bytes) 000395[1] 1736984498.558861 mapread 0x60d53c thru 0x610d32 (0x37f7 bytes) 000400[1] 1736984498.599871 trunc from 0x904b74 to 0x3337a7 (0x5d13cc bytes) 000401[1] 1736984498.631674 read 0x0d1f7f thru 0x0d9c2e (0x7cb0 bytes) 000403[0] 1736984498.669063 trunc from 0x3337a8 to 0x6a6507 (0x372d60 bytes) 000405[0] 1736984498.738634 write 0x0b84f1 thru 0x0bbcae (0x37be bytes) 000406[0] 1736984498.784852 mapwrite 0x36ecd7 thru 0x3758d8 (0x6c02 bytes) 000407[0] 1736984498.885544 write 0x0e0f70 thru 0x0ee5d0 (0xd661 bytes) 000409[0] 1736984498.917910 read 0x69207b thru 0x69905f (0x6fe5 bytes) 000410[0] 1736984498.920763 mapwrite 0x38606b thru 0x38e1e9 (0x817f bytes) 000411[1] 1736984499.000756 read 0x4f811e thru 0x504724 (0xc607 bytes) 000413[1] 1736984499.058302 write 0x8a97b8 thru 0x8aade0 (0x1629 bytes) 000414[1] 1736984499.065222 write 0x708f80 thru 0x70c4a7 (0x3528 bytes) 000415[1] 1736984499.107085 write 0x7f6cd6 thru 0x80188c (0xabb7 bytes) 000416[0] 1736984499.120512 mapread 0x86759d thru 0x86e3bd (0x6e21 bytes) 000419[0] 1736984499.179994 trunc from 0x8aade1 to 0x67d53f (0x22d8a1 bytes) 000420[1] 1736984499.212044 read 0x4981d8 thru 0x49e9e6 (0x680f bytes) 000421[0] 1736984499.243020 write 0x6ee75d thru 0x6f9b6e (0xb412 bytes) 000422[1] 1736984499.264686 write 0x03c4de thru 0x04087d (0x43a0 bytes) 000423[0] 1736984499.289527 mapwrite 0x87c32e thru 0x882672 (0x6345 bytes) 000424[1] 1736984499.467681 mapwrite 0x55afdc thru 0x55d749 (0x276e bytes) 000425[0] 1736984499.584701 read 0x44a4dc thru 0x453753 (0x9278 bytes) 000427[0] 1736984499.638281 write 0x7b0734 thru 0x7b73b4 (0x6c81 bytes) 000428[0] 1736984499.687412 read 0x71f62a thru 0x72a526 (0xaefd bytes) 000431[0] 1736984499.719359 read 0x3de520 thru 0x3de8dc (0x3bd bytes) truncating to largest ever: 0x95dbd7 000432[1] 1736984499.737133 trunc from 0x882673 to 0x95dbd6 (0xdb564 bytes) 000433[0] 1736984499.801832 read 0x6d7d2c thru 0x6d99e1 (0x1cb6 bytes) 000434[1] 1736984499.828627 mapwrite 0x177c34 thru 0x17c401 (0x47ce bytes) 000435[1] 1736984499.952524 mapread 0x84d0bc thru 0x84d72a (0x66f bytes) 000436[0] 1736984499.971860 write 0x62864c thru 0x63053a (0x7eef bytes) 000438[0] 1736984500.022385 trunc from 0x95dbd7 to 0x3d5e51 (0x587d85 bytes) truncating to largest ever: 0x98727f 000440[1] 1736984500.067091 trunc from 0x3d5e52 to 0x98727e (0x5b142d bytes) 000441[0] 1736984500.102932 mapwrite 0x98728b thru 0x996265 (0xefdb bytes) 000443[1] 1736984500.278513 mapwrite 0x08da69 thru 0x09d9b8 (0xff50 bytes) 000444[0] 1736984500.458779 write 0x8db94d thru 0x8e38a8 (0x7f5c bytes) 000445[1] 1736984500.473629 write 0x79565f thru 0x79cde0 (0x7782 bytes) 000447[0] 1736984500.517796 trunc from 0x996266 to 0x75fa53 (0x236812 bytes) 000448[1] 1736984500.565723 trunc from 0x75fa54 to 0x041238 (0x71e81b bytes) 000449[1] 1736984500.601102 read 0x01b0fb thru 0x0202fc (0x5202 bytes) 000452[1] 1736984500.641048 write 0x4ab30d thru 0x4b0e90 (0x5b84 bytes) 000453[1] 1736984500.645216 write 0x6ad329 thru 0x6ae98d (0x1665 bytes) 000454[1] 1736984500.649484 mapread 0x56f2ff thru 0x5718f5 (0x25f7 bytes) 000455[1] 1736984500.710888 trunc from 0x6ae98e to 0x4c8c87 (0x1e5d06 bytes) 000456[1] 1736984500.749010 read 0x0dfffd thru 0x0e2f17 (0x2f1b bytes) 000457[1] 1736984500.765226 mapwrite 0x1178b3 thru 0x11fc8f (0x83dd bytes) 000459[1] 1736984500.820941 write 0x4a25d0 thru 0x4a56f4 (0x3125 bytes) 000460[0] 1736984500.821525 trunc from 0x4c8c88 to 0x81d070 (0x3543e9 bytes) 000462[0] 1736984500.881540 read 0x10efa1 thru 0x11cbd0 (0xdc30 bytes) truncating to largest ever: 0x9d45b8 000463[0] 1736984500.918526 trunc from 0x81d071 to 0x9d45b7 (0x1b7547 bytes) 000464[0] 1736984500.951620 read 0x3c118d thru 0x3c9e1c (0x8c90 bytes) 000468[0] 1736984500.995386 write 0x1c67ef thru 0x1cfc63 (0x9475 bytes) 000469[1] 1736984501.014918 trunc from 0x9d45b8 to 0x645ca1 (0x38e916 bytes) 000470[1] 1736984501.079219 mapwrite 0x9e3d0e thru 0x9e5699 (0x198c bytes) 000471[1] 1736984501.166228 trunc from 0x9e569a to 0x3fdbe5 (0x5e7ab4 bytes) 000472[0] 1736984501.179308 mapread 0x314236 thru 0x31fd4e (0xbb19 bytes) 000474[0] 1736984501.254283 write 0x6b215c thru 0x6be1f2 (0xc097 bytes) 000477[1] 1736984501.299748 write 0x504e9b thru 0x5139c1 (0xeb27 bytes) 000481[1] 1736984501.340237 trunc from 0x6be1f3 to 0x0a9102 (0x6150f0 bytes) 000483[1] 1736984501.391699 mapwrite 0x3f94ee thru 0x408773 (0xf286 bytes) 000484[0] 1736984501.537714 write 0x72f250 thru 0x73e462 (0xf213 bytes) 000485[1] 1736984501.558134 trunc from 0x73e463 to 0x0727eb (0x6cbc77 bytes) 000486[0] 1736984501.584973 write 0x552f46 thru 0x553fd2 (0x108d bytes) 000487[1] 1736984501.612880 mapwrite 0x25184a thru 0x253032 (0x17e9 bytes) 000488[0] 1736984501.706747 trunc from 0x553fd3 to 0x7fe3dc (0x2aa40a bytes) 000489[0] 1736984501.732511 read 0x24283f thru 0x24e1d2 (0xb994 bytes) 000490[1] 1736984501.769533 trunc from 0x7fe3dd to 0x54a9e0 (0x2b39fc bytes) 000491[1] 1736984501.848465 read 0x4e30ca thru 0x4e6073 (0x2faa bytes) 000495[1] 1736984501.915246 mapread 0x0ee4cf thru 0x0f5a07 (0x7539 bytes) 000497[1] 1736984501.970080 trunc from 0x54a9e1 to 0x287450 (0x2c3590 bytes) 000500[1] 1736984502.014651 mapread 0x1d045f thru 0x1d1951 (0x14f3 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 4502 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[1] 1736984502.420667 mapwrite 0x6fa000 thru 0x6fb01f (0x1020 bytes) 000004[0] 1736984502.500826 write_OD 0x21b000 thru 0x226fff (0xc000 bytes) 000005[0] 1736984502.573305 read_OD 0x3fc000 thru 0x3fdfff (0x2000 bytes) 000006[0] 1736984502.578827 mapwrite 0x7dd000 thru 0x7e4e15 (0x7e16 bytes) 000007[0] 1736984502.677489 mapread 0x7e1000 thru 0x7e4bf5 (0x3bf6 bytes) truncating to largest ever: 0x3ebf8d 000008[1] 1736984502.679938 trunc from 0x7e4e16 to 0x3ebf8c (0x3f8e89 bytes) 000010[1] 1736984502.712894 mapread 0x02f000 thru 0x02f2ff (0x300 bytes) 000012[0] 1736984502.724671 mapread 0x187000 thru 0x18a482 (0x3483 bytes) 000014[1] 1736984502.754768 mapwrite 0x1d2000 thru 0x1d3d49 (0x1d4a bytes) 000015[1] 1736984502.833697 trunc from 0x3ebf8d to 0x0661dd (0x385daf bytes) 000016[1] 1736984502.853820 write_OD 0x461000 thru 0x467fff (0x7000 bytes) 000017[1] 1736984502.899781 trunc from 0x468000 to 0x2b05a8 (0x1b7a57 bytes) 000019[1] 1736984502.912684 mapwrite 0x2cf000 thru 0x2cf4da (0x4db bytes) truncating to largest ever: 0x6311df 000021[0] 1736984502.977847 trunc from 0x2cf4db to 0x6311de (0x361d04 bytes) 000023[0] 1736984503.003168 read_OD 0x263000 thru 0x263fff (0x1000 bytes) 000024[1] 1736984503.018328 trunc from 0x6311df to 0x5f305c (0x3e182 bytes) 000026[1] 1736984503.061645 write_OD 0x3d4000 thru 0x3d9fff (0x6000 bytes) 000029[0] 1736984503.115201 write_OD 0x374000 thru 0x381fff (0xe000 bytes) 000031[1] 1736984503.164631 mapwrite 0x373000 thru 0x378767 (0x5768 bytes) 000032[0] 1736984503.270764 write_OD 0x40f000 thru 0x410fff (0x2000 bytes) truncating to largest ever: 0x75acf9 000033[1] 1736984503.310591 trunc from 0x5f305d to 0x75acf8 (0x167c9c bytes) truncating to largest ever: 0x7eb935 000035[0] 1736984503.331848 trunc from 0x75acf9 to 0x7eb934 (0x90c3c bytes) 000039[1] 1736984503.373126 mapread 0x410000 thru 0x4102e6 (0x2e7 bytes) 000040[0] 1736984503.402804 write_OD 0x4b4000 thru 0x4b8fff (0x5000 bytes) 000041[1] 1736984503.457064 write_OD 0x0b6000 thru 0x0bbfff (0x6000 bytes) truncating to largest ever: 0x9c1b60 000042[0] 1736984503.513838 trunc from 0x7eb935 to 0x9c1b5f (0x1d622b bytes) 000047[1] 1736984503.566679 write_OD 0x7b6000 thru 0x7bdfff (0x8000 bytes) 000048[0] 1736984503.612917 write_OD 0x7fa000 thru 0x803fff (0xa000 bytes) 000051[1] 1736984503.670718 mapwrite 0x030000 thru 0x03a383 (0xa384 bytes) 000052[1] 1736984503.768577 write_OD 0x18c000 thru 0x191fff (0x6000 bytes) 000053[0] 1736984503.831264 trunc from 0x9c1b60 to 0x85ee52 (0x162d0d bytes) 000055[0] 1736984503.864264 mapwrite 0x846000 thru 0x853b40 (0xdb41 bytes) 000056[0] 1736984503.964928 read_OD 0x696000 thru 0x6a1fff (0xc000 bytes) 000057[0] 1736984503.989859 mapread 0x55e000 thru 0x56ce61 (0xee62 bytes) 000059[1] 1736984504.066642 write_OD 0x62c000 thru 0x637fff (0xc000 bytes) 000060[1] 1736984504.125664 write_OD 0x822000 thru 0x828fff (0x7000 bytes) 000063[0] 1736984504.174814 trunc from 0x85ee53 to 0x13c1a5 (0x722cad bytes) 000071[1] 1736984504.240462 read_OD 0x13a000 thru 0x13afff (0x1000 bytes) 000073[1] 1736984504.256791 mapread 0x12d000 thru 0x135b5c (0x8b5d bytes) 000075[0] 1736984504.274075 read_OD 0x0b1000 thru 0x0b6fff (0x6000 bytes) 000076[0] 1736984504.278220 read_OD 0x076000 thru 0x07dfff (0x8000 bytes) 000079[0] 1736984504.286178 write_OD 0x092000 thru 0x09ffff (0xe000 bytes) 000080[0] 1736984504.356463 mapread 0x098000 thru 0x0a31fe (0xb1ff bytes) 000082[1] 1736984504.374183 mapwrite 0x810000 thru 0x816831 (0x6832 bytes) 000086[0] 1736984504.523164 mapwrite 0x58d000 thru 0x5989cf (0xb9d0 bytes) 000087[1] 1736984504.663130 trunc from 0x816832 to 0x71a34f (0xfc4e2 bytes) 000089[1] 1736984504.683926 read_OD 0x2e2000 thru 0x2effff (0xe000 bytes) 000090[1] 1736984504.701147 read_OD 0x203000 thru 0x210fff (0xe000 bytes) 000091[0] 1736984504.711892 write_OD 0x0d0000 thru 0x0d2fff (0x3000 bytes) 000096[1] 1736984504.775225 write_OD 0x049000 thru 0x049fff (0x1000 bytes) 000097[1] 1736984504.837354 write_OD 0x4b6000 thru 0x4bcfff (0x7000 bytes) 000098[0] 1736984504.890527 write_OD 0x4b3000 thru 0x4b6fff (0x4000 bytes) 000099[0] 1736984504.948647 mapwrite 0x512000 thru 0x520f05 (0xef06 bytes) 000101[0] 1736984505.078949 mapread 0x6b7000 thru 0x6c12ff (0xa300 bytes) 000103[1] 1736984505.109742 write_OD 0x8e7000 thru 0x8e9fff (0x3000 bytes) 000104[1] 1736984505.161834 read_OD 0x0e8000 thru 0x0f0fff (0x9000 bytes) 000109[1] 1736984505.174620 mapwrite 0x593000 thru 0x59f3b0 (0xc3b1 bytes) 000113[1] 1736984505.334722 trunc from 0x8ea000 to 0x63c8db (0x2ad724 bytes) 000115[0] 1736984505.366702 mapwrite 0x4f2000 thru 0x4f57ca (0x37cb bytes) 000116[0] 1736984505.449017 mapwrite 0x060000 thru 0x06a0d4 (0xa0d5 bytes) 000117[0] 1736984505.541001 mapwrite 0x6a8000 thru 0x6b50f1 (0xd0f2 bytes) 000119[1] 1736984505.711472 write_OD 0x748000 thru 0x749fff (0x2000 bytes) 000121[1] 1736984505.771653 write_OD 0x9c7000 thru 0x9cdfff (0x7000 bytes) 000122[0] 1736984505.830038 mapread 0x22a000 thru 0x22a988 (0x989 bytes) 000123[1] 1736984505.835502 mapwrite 0x171000 thru 0x174306 (0x3307 bytes) 000126[1] 1736984505.953028 trunc from 0x9ce000 to 0x164c95 (0x86936a bytes) 000127[1] 1736984505.975876 write_OD 0x131000 thru 0x132fff (0x2000 bytes) 000128[1] 1736984506.024965 mapwrite 0x1a8000 thru 0x1b0a8e (0x8a8f bytes) 000131[0] 1736984506.129562 mapread 0x181000 thru 0x18ee88 (0xde89 bytes) 000132[1] 1736984506.180093 mapwrite 0x2a3000 thru 0x2b08db (0xd8dc bytes) 000134[0] 1736984506.296122 mapwrite 0x31c000 thru 0x31d02b (0x102c bytes) 000135[0] 1736984506.372156 write_OD 0x165000 thru 0x171fff (0xd000 bytes) 000137[0] 1736984506.425770 read_OD 0x133000 thru 0x135fff (0x3000 bytes) 000139[0] 1736984506.435610 mapread 0x28f000 thru 0x290385 (0x1386 bytes) 000140[0] 1736984506.463959 write_OD 0x98e000 thru 0x990fff (0x3000 bytes) 000141[0] 1736984506.552090 mapread 0x7e4000 thru 0x7e86e0 (0x46e1 bytes) 000142[1] 1736984506.576506 mapread 0x83d000 thru 0x83eeac (0x1ead bytes) 000146[0] 1736984506.588036 mapread 0x3ad000 thru 0x3b5af1 (0x8af2 bytes) 000147[0] 1736984506.617183 mapread 0x7c7000 thru 0x7cd8e6 (0x68e7 bytes) 000148[0] 1736984506.634153 read_OD 0x933000 thru 0x93ffff (0xd000 bytes) 000150[0] 1736984506.640720 mapwrite 0x9c5000 thru 0x9d0d45 (0xbd46 bytes) 000152[0] 1736984506.798013 mapread 0x5e6000 thru 0x5f18ee (0xb8ef bytes) 000153[0] 1736984506.873873 write_OD 0x736000 thru 0x736fff (0x1000 bytes) 000154[0] 1736984506.927202 write_OD 0x692000 thru 0x69afff (0x9000 bytes) 000155[1] 1736984506.994896 read_OD 0x350000 thru 0x35bfff (0xc000 bytes) 000157[0] 1736984507.004643 write_OD 0x1eb000 thru 0x1eefff (0x4000 bytes) 000158[1] 1736984507.074463 read_OD 0x7b2000 thru 0x7b7fff (0x6000 bytes) 000159[1] 1736984507.088546 read_OD 0x6ff000 thru 0x702fff (0x4000 bytes) 000160[1] 1736984507.103300 read_OD 0x7f1000 thru 0x7f8fff (0x8000 bytes) 000161[0] 1736984507.119065 mapread 0x13c000 thru 0x140573 (0x4574 bytes) 000164[0] 1736984507.153160 mapread 0x8c5000 thru 0x8cceac (0x7ead bytes) 000167[1] 1736984507.184886 mapwrite 0x372000 thru 0x3722e5 (0x2e6 bytes) 000169[0] 1736984507.291644 mapread 0x8d2000 thru 0x8dcb04 (0xab05 bytes) 000173[1] 1736984507.348036 mapread 0x49d000 thru 0x4a8897 (0xb898 bytes) 000177[1] 1736984507.397205 trunc from 0x9d0d46 to 0x2da135 (0x6f6c10 bytes) 000179[1] 1736984507.428464 read_OD 0x0e1000 thru 0x0eafff (0xa000 bytes) 000181[0] 1736984507.440553 mapwrite 0x01a000 thru 0x01cfa1 (0x2fa2 bytes) 000184[1] 1736984507.522162 trunc from 0x2da136 to 0x857920 (0x57d7eb bytes) 000187[0] 1736984507.565611 read_OD 0x135000 thru 0x13bfff (0x7000 bytes) 000189[1] 1736984507.581060 mapwrite 0x2c5000 thru 0x2cc0ba (0x70bb bytes) 000190[0] 1736984507.736300 write_OD 0x5e6000 thru 0x5eefff (0x9000 bytes) 000193[0] 1736984507.810400 mapread 0x303000 thru 0x30d545 (0xa546 bytes) 000195[1] 1736984507.894665 trunc from 0x857921 to 0x45c6a5 (0x3fb27b bytes) 000197[1] 1736984507.936586 mapread 0x415000 thru 0x4157ff (0x800 bytes) 000198[0] 1736984507.944184 trunc from 0x45c6a6 to 0x5845d9 (0x127f34 bytes) 000199[1] 1736984507.978897 mapwrite 0x39f000 thru 0x3a7f45 (0x8f46 bytes) 000203[1] 1736984508.130152 write_OD 0x259000 thru 0x261fff (0x9000 bytes) 000204[1] 1736984508.187304 write_OD 0x187000 thru 0x190fff (0xa000 bytes) 000205[0] 1736984508.243031 trunc from 0x5845da to 0x3a9506 (0x1db0d3 bytes) 000206[0] 1736984508.266319 mapread 0x174000 thru 0x174c0a (0xc0b bytes) 000207[1] 1736984508.279987 mapwrite 0x9e8000 thru 0x9ec27f (0x4280 bytes) 000208[1] 1736984508.404453 write_OD 0x36c000 thru 0x36efff (0x3000 bytes) 000209[1] 1736984508.457704 mapwrite 0x14b000 thru 0x14b8c8 (0x8c9 bytes) truncating to largest ever: 0x9d13fb 000210[1] 1736984508.522020 trunc from 0x9ec280 to 0x9d13fa (0x1ae85 bytes) 000211[1] 1736984508.538317 read_OD 0x6ca000 thru 0x6cdfff (0x4000 bytes) 000212[0] 1736984508.549159 read_OD 0x4c6000 thru 0x4ccfff (0x7000 bytes) 000217[0] 1736984508.561931 mapread 0x21c000 thru 0x22b658 (0xf659 bytes) 000218[0] 1736984508.656009 read_OD 0x148000 thru 0x151fff (0xa000 bytes) 000219[0] 1736984508.662185 mapwrite 0x74e000 thru 0x754732 (0x6733 bytes) 000221[0] 1736984508.856021 mapwrite 0x568000 thru 0x575027 (0xd028 bytes) 000223[1] 1736984508.955907 read_OD 0x8ae000 thru 0x8b4fff (0x7000 bytes) 000225[0] 1736984508.977638 mapwrite 0x5db000 thru 0x5db0bf (0xc0 bytes) 000226[0] 1736984509.048208 mapread 0x811000 thru 0x81e4b8 (0xd4b9 bytes) 000229[0] 1736984509.107710 trunc from 0x9d13fb to 0x240ff7 (0x790403 bytes) 000230[1] 1736984509.126262 read_OD 0x136000 thru 0x141fff (0xc000 bytes) 000231[1] 1736984509.156383 mapread 0x142000 thru 0x14752d (0x552e bytes) 000232[1] 1736984509.176211 read_OD 0x18d000 thru 0x19bfff (0xf000 bytes) 000234[1] 1736984509.186395 write_OD 0x56f000 thru 0x570fff (0x2000 bytes) 000235[0] 1736984509.238689 trunc from 0x571000 to 0x278169 (0x2f8e96 bytes) 000236[0] 1736984509.264924 mapread 0x216000 thru 0x2245e3 (0xe5e4 bytes) 000238[0] 1736984509.342493 read_OD 0x128000 thru 0x135fff (0xe000 bytes) 000239[0] 1736984509.356795 mapwrite 0x347000 thru 0x35056d (0x956e bytes) 000241[1] 1736984509.479985 write_OD 0x748000 thru 0x751fff (0xa000 bytes) 000242[1] 1736984509.545004 mapread 0x6c1000 thru 0x6c1669 (0x66a bytes) 000244[0] 1736984509.617225 write_OD 0x69a000 thru 0x6a6fff (0xd000 bytes) 000245[1] 1736984509.686565 mapread 0x24f000 thru 0x25dee2 (0xeee3 bytes) 000246[1] 1736984509.732049 read_OD 0x1c3000 thru 0x1cefff (0xc000 bytes) 000248[1] 1736984509.737401 mapread 0x300000 thru 0x30a5b3 (0xa5b4 bytes) 000249[1] 1736984509.813160 write_OD 0x306000 thru 0x30afff (0x5000 bytes) 000251[0] 1736984509.890035 read_OD 0x04b000 thru 0x04ffff (0x5000 bytes) 000252[1] 1736984509.896185 mapwrite 0x09b000 thru 0x09bbf4 (0xbf5 bytes) 000254[1] 1736984509.954985 mapwrite 0x0dd000 thru 0x0dfebd (0x2ebe bytes) 000258[1] 1736984510.024382 write_OD 0x1bc000 thru 0x1c5fff (0xa000 bytes) 000259[1] 1736984510.088404 read_OD 0x1a6000 thru 0x1b0fff (0xb000 bytes) 000260[1] 1736984510.095108 write_OD 0x3de000 thru 0x3e9fff (0xc000 bytes) 000261[1] 1736984510.145146 mapread 0x0f2000 thru 0x0ff5ca (0xd5cb bytes) 000262[1] 1736984510.240187 mapread 0x478000 thru 0x47a3a7 (0x23a8 bytes) 000263[1] 1736984510.242803 trunc from 0x752000 to 0x6b6898 (0x9b767 bytes) 000264[0] 1736984510.286049 trunc from 0x6b6899 to 0x3b8c95 (0x2fdc03 bytes) 000266[1] 1736984510.319766 mapread 0x34a000 thru 0x34dd54 (0x3d55 bytes) 000267[0] 1736984510.350003 mapread 0x2f7000 thru 0x2f7c7c (0xc7d bytes) 000268[0] 1736984510.381159 trunc from 0x3b8c96 to 0x00cb6d (0x3ac128 bytes) 000269[1] 1736984510.420283 mapwrite 0x7df000 thru 0x7eb3dc (0xc3dd bytes) 000271[1] 1736984510.515076 mapread 0x230000 thru 0x2333af (0x33b0 bytes) 000273[1] 1736984510.571018 read_OD 0x22f000 thru 0x22ffff (0x1000 bytes) 000275[0] 1736984510.588617 mapread 0x11d000 thru 0x12a419 (0xd41a bytes) 000276[0] 1736984510.705114 trunc from 0x7eb3dd to 0x5103df (0x2daffd bytes) 000277[1] 1736984510.736308 trunc from 0x5103e0 to 0x8a0e73 (0x390a94 bytes) 000278[0] 1736984510.767500 trunc from 0x8a0e74 to 0x339ae5 (0x56738e bytes) 000279[1] 1736984510.793361 write_OD 0x1ab000 thru 0x1b0fff (0x6000 bytes) 000282[1] 1736984510.852270 mapwrite 0x700000 thru 0x706826 (0x6827 bytes) 000291[0] 1736984511.056157 mapwrite 0x66e000 thru 0x678b04 (0xab05 bytes) 000292[1] 1736984511.154684 trunc from 0x706827 to 0x5aed55 (0x157ad1 bytes) 000294[0] 1736984511.177168 trunc from 0x5aed56 to 0x1a1df4 (0x40cf61 bytes) 000296[0] 1736984511.207210 mapwrite 0x9e7000 thru 0x9ee0d5 (0x70d6 bytes) 000297[1] 1736984511.337207 read_OD 0x84f000 thru 0x859fff (0xb000 bytes) 000301[1] 1736984511.361687 read_OD 0x385000 thru 0x38efff (0xa000 bytes) 000303[0] 1736984511.366737 read_OD 0x13d000 thru 0x14bfff (0xf000 bytes) 000304[1] 1736984511.372510 mapread 0x2ee000 thru 0x2f3200 (0x5201 bytes) 000305[0] 1736984511.458944 write_OD 0x831000 thru 0x83ffff (0xf000 bytes) 000311[1] 1736984511.543005 trunc from 0x9ee0d6 to 0x692473 (0x35bc62 bytes) 000312[0] 1736984511.588330 read_OD 0x072000 thru 0x072fff (0x1000 bytes) 000316[0] 1736984511.626085 trunc from 0x692474 to 0x5c1d7d (0xd06f6 bytes) 000317[1] 1736984511.657056 mapread 0x460000 thru 0x46068b (0x68c bytes) 000318[1] 1736984511.693249 write_OD 0x7a9000 thru 0x7b2fff (0xa000 bytes) 000319[1] 1736984511.757919 write_OD 0x1d4000 thru 0x1dffff (0xc000 bytes) 000320[0] 1736984511.832399 read_OD 0x6ca000 thru 0x6d0fff (0x7000 bytes) 000322[1] 1736984511.847953 mapread 0x289000 thru 0x291b65 (0x8b66 bytes) 000323[1] 1736984511.889532 read_OD 0x080000 thru 0x089fff (0xa000 bytes) 000324[0] 1736984511.894221 mapread 0x2c7000 thru 0x2d173f (0xa740 bytes) 000326[0] 1736984511.935808 read_OD 0x0c5000 thru 0x0c6fff (0x2000 bytes) 000327[1] 1736984511.941059 write_OD 0x0e5000 thru 0x0edfff (0x9000 bytes) 000328[1] 1736984512.004534 mapread 0x4d9000 thru 0x4d9ffa (0xffb bytes) 000329[1] 1736984512.068944 trunc from 0x7b3000 to 0x2c24a8 (0x4f0b57 bytes) 000330[1] 1736984512.118519 trunc from 0x2c24a9 to 0x3b34b9 (0xf1011 bytes) 000331[0] 1736984512.140264 write_OD 0x83f000 thru 0x848fff (0xa000 bytes) 000334[1] 1736984512.218678 mapwrite 0x555000 thru 0x5595d2 (0x45d3 bytes) 000335[1] 1736984512.311359 mapread 0x71e000 thru 0x71eeb9 (0xeba bytes) 000336[1] 1736984512.317633 read_OD 0x7c2000 thru 0x7c8fff (0x7000 bytes) 000337[0] 1736984512.345302 write_OD 0x2d3000 thru 0x2d7fff (0x5000 bytes) 000338[0] 1736984512.416768 mapread 0x3df000 thru 0x3e3527 (0x4528 bytes) 000339[0] 1736984512.508343 trunc from 0x849000 to 0x557911 (0x2f16ee bytes) 000340[1] 1736984512.557779 trunc from 0x557912 to 0x5fd95f (0xa604e bytes) 000342[0] 1736984512.584254 read_OD 0x3ba000 thru 0x3c6fff (0xd000 bytes) 000346[0] 1736984512.605123 mapread 0x038000 thru 0x0477b7 (0xf7b8 bytes) 000347[1] 1736984512.684311 read_OD 0x387000 thru 0x38bfff (0x5000 bytes) 000348[1] 1736984512.688375 write_OD 0x712000 thru 0x71efff (0xd000 bytes) 000350[1] 1736984512.750857 read_OD 0x2aa000 thru 0x2b1fff (0x8000 bytes) 000351[1] 1736984512.763791 write_OD 0x01f000 thru 0x02dfff (0xf000 bytes) 000353[1] 1736984512.835522 mapread 0x482000 thru 0x48b232 (0x9233 bytes) 000355[1] 1736984512.888077 mapread 0x042000 thru 0x04a05e (0x805f bytes) 000357[1] 1736984512.910738 read_OD 0x3f6000 thru 0x3f6fff (0x1000 bytes) 000360[0] 1736984512.913664 trunc from 0x71f000 to 0x3070c7 (0x417f38 bytes) 000361[1] 1736984512.944282 trunc from 0x3070c8 to 0x3e3d4b (0xdcc84 bytes) 000362[0] 1736984512.971500 write_OD 0x694000 thru 0x699fff (0x6000 bytes) 000363[1] 1736984513.028759 mapread 0x14d000 thru 0x15604d (0x904e bytes) 000367[0] 1736984513.102788 trunc from 0x69a000 to 0x71d0be (0x830bf bytes) 000368[0] 1736984513.152311 read_OD 0x671000 thru 0x672fff (0x2000 bytes) 000369[1] 1736984513.164072 mapread 0x1fe000 thru 0x203c63 (0x5c64 bytes) 000370[1] 1736984513.217303 read_OD 0x059000 thru 0x05cfff (0x4000 bytes) 000376[0] 1736984513.226843 trunc from 0x71d0bf to 0x67be1e (0xa12a0 bytes) 000377[1] 1736984513.263041 read_OD 0x0de000 thru 0x0ecfff (0xf000 bytes) 000378[1] 1736984513.279503 mapread 0x09c000 thru 0x09d2c1 (0x12c2 bytes) 000379[0] 1736984513.359500 write_OD 0x679000 thru 0x684fff (0xc000 bytes) 000385[1] 1736984513.453093 trunc from 0x685000 to 0x83d57b (0x1b857c bytes) 000388[0] 1736984513.498575 write_OD 0x209000 thru 0x209fff (0x1000 bytes) 000390[0] 1736984513.548602 mapwrite 0x5f3000 thru 0x5fbdf3 (0x8df4 bytes) 000393[0] 1736984513.688166 trunc from 0x83d57c to 0x13439a (0x7091e1 bytes) 000394[1] 1736984513.707319 trunc from 0x13439b to 0x2efbe8 (0x1bb84e bytes) 000397[1] 1736984513.742534 mapwrite 0x76a000 thru 0x76c4a6 (0x24a7 bytes) 000399[0] 1736984513.824990 write_OD 0x8ff000 thru 0x90afff (0xc000 bytes) 000404[0] 1736984513.893560 trunc from 0x90b000 to 0x3ec140 (0x51eebf bytes) 000407[0] 1736984513.955853 write_OD 0x442000 thru 0x44ffff (0xe000 bytes) 000409[1] 1736984514.003321 mapwrite 0x33b000 thru 0x348208 (0xd209 bytes) 000412[0] 1736984514.117363 trunc from 0x450000 to 0x4e75c1 (0x975c2 bytes) 000413[0] 1736984514.147217 trunc from 0x4e75c2 to 0x969d5e (0x48279d bytes) 000415[1] 1736984514.171328 read_OD 0x1d2000 thru 0x1d8fff (0x7000 bytes) 000416[0] 1736984514.200041 mapread 0x32b000 thru 0x331443 (0x6444 bytes) 000418[0] 1736984514.250332 write_OD 0x5f5000 thru 0x601fff (0xd000 bytes) 000419[0] 1736984514.314562 write_OD 0x198000 thru 0x1a0fff (0x9000 bytes) 000422[0] 1736984514.398667 mapwrite 0x3a0000 thru 0x3a229b (0x229c bytes) 000423[0] 1736984514.467759 read_OD 0x1ee000 thru 0x1effff (0x2000 bytes) 000424[1] 1736984514.480012 mapread 0x8a5000 thru 0x8ac3d4 (0x73d5 bytes) 000425[0] 1736984514.494458 mapread 0x8e8000 thru 0x8f6d26 (0xed27 bytes) 000426[1] 1736984514.561615 mapread 0x5f8000 thru 0x607a30 (0xfa31 bytes) 000427[0] 1736984514.587220 read_OD 0x35e000 thru 0x367fff (0xa000 bytes) 000431[0] 1736984514.594242 mapwrite 0x130000 thru 0x136d46 (0x6d47 bytes) 000433[1] 1736984514.766805 write_OD 0x530000 thru 0x53cfff (0xd000 bytes) 000434[1] 1736984514.843656 trunc from 0x969d5f to 0x86e3a5 (0xfb9b9 bytes) 000438[1] 1736984514.909328 write_OD 0x0cf000 thru 0x0dafff (0xc000 bytes) 000439[0] 1736984514.956348 write_OD 0x43f000 thru 0x448fff (0xa000 bytes) 000442[1] 1736984515.022379 trunc from 0x86e3a6 to 0x71bbc6 (0x1527df bytes) 000443[1] 1736984515.055853 write_OD 0x0f6000 thru 0x101fff (0xc000 bytes) 000445[0] 1736984515.112343 trunc from 0x71bbc7 to 0x391207 (0x38a9bf bytes) 000449[1] 1736984515.165295 mapread 0x09c000 thru 0x0aa544 (0xe545 bytes) 000450[1] 1736984515.192751 trunc from 0x391208 to 0x1777b8 (0x219a4f bytes) 000451[0] 1736984515.222723 mapwrite 0x6a5000 thru 0x6a83ca (0x33cb bytes) 000452[0] 1736984515.318528 mapread 0x332000 thru 0x340604 (0xe605 bytes) 000454[0] 1736984515.355309 mapread 0x619000 thru 0x61de15 (0x4e16 bytes) 000457[1] 1736984515.400476 mapread 0x40e000 thru 0x413536 (0x5537 bytes) 000459[0] 1736984515.421554 mapread 0x091000 thru 0x09255d (0x155e bytes) 000460[1] 1736984515.428823 write_OD 0x241000 thru 0x248fff (0x8000 bytes) 000462[0] 1736984515.526286 mapread 0x3fa000 thru 0x3fbbe9 (0x1bea bytes) 000463[0] 1736984515.532935 mapread 0x4dd000 thru 0x4de1d7 (0x11d8 bytes) 000465[0] 1736984515.543114 mapread 0x1d6000 thru 0x1de18e (0x818f bytes) 000466[1] 1736984515.576766 trunc from 0x6a83cb to 0x6ccd98 (0x249ce bytes) 000469[1] 1736984515.629270 read_OD 0x131000 thru 0x13efff (0xe000 bytes) 000470[1] 1736984515.648014 trunc from 0x6ccd99 to 0x290baa (0x43c1ee bytes) 000472[0] 1736984515.687453 trunc from 0x290bab to 0x4b3a70 (0x222ec6 bytes) 000475[1] 1736984515.725830 trunc from 0x4b3a71 to 0x810add (0x35d06d bytes) 000476[1] 1736984515.746878 mapwrite 0x206000 thru 0x20fae0 (0x9ae1 bytes) 000477[1] 1736984515.890801 trunc from 0x810ade to 0x446ce6 (0x3c9df7 bytes) 000479[1] 1736984515.919950 mapwrite 0x58b000 thru 0x591645 (0x6646 bytes) 000481[0] 1736984516.011820 mapread 0x195000 thru 0x19c009 (0x700a bytes) 000482[0] 1736984516.081559 read_OD 0x3c6000 thru 0x3cafff (0x5000 bytes) 000483[1] 1736984516.091277 mapread 0x4bb000 thru 0x4bc836 (0x1837 bytes) 000484[1] 1736984516.099660 write_OD 0x1fc000 thru 0x208fff (0xd000 bytes) 000486[1] 1736984516.171702 write_OD 0x32f000 thru 0x33bfff (0xd000 bytes) 000490[1] 1736984516.241974 read_OD 0x282000 thru 0x287fff (0x6000 bytes) 000493[1] 1736984516.260227 read_OD 0x1ab000 thru 0x1affff (0x5000 bytes) 000494[0] 1736984516.270595 mapwrite 0x206000 thru 0x209174 (0x3175 bytes) 000495[0] 1736984516.366344 write_OD 0x047000 thru 0x04cfff (0x6000 bytes) 000498[1] 1736984516.412738 read_OD 0x0e9000 thru 0x0f1fff (0x9000 bytes) 000499[0] 1736984516.420069 read_OD 0x1e5000 thru 0x1f2fff (0xe000 bytes) All operations completed A-OK! PASS 16a (34s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 18:41:58 (1736984518) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.237776 s, 8.8 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.178363 s, 11.8 MB/s Chance of close/open is 1 in 50 Seed set to 4520 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[1] 1736984521.012764 write 0x3e2606 thru 0x3e3139 (0xb34 bytes) 000003[0] 1736984521.036078 write 0x6b7a96 thru 0x6b9566 (0x1ad1 bytes) truncating to largest ever: 0x82f777 000004[0] 1736984521.053749 trunc from 0x6b9567 to 0x82f776 (0x176210 bytes) 000005[0] 1736984521.097095 trunc from 0x82f777 to 0x4e5b93 (0x349be3 bytes) truncating to largest ever: 0x96309c 000006[1] 1736984521.112677 trunc from 0x4e5b94 to 0x96309b (0x47d508 bytes) 000007[0] 1736984521.172626 write 0x24c0f7 thru 0x24cc16 (0xb20 bytes) 000008[1] 1736984521.192219 mapwrite 0x892229 thru 0x8922c9 (0xa1 bytes) 000010[1] 1736984521.345610 mapwrite 0x6c2cd0 thru 0x6c388b (0xbbc bytes) 000011[1] 1736984521.440473 trunc from 0x96309c to 0x41a7a6 (0x5488f5 bytes) 000012[0] 1736984521.472665 trunc from 0x41a7a7 to 0x45008d (0x358e7 bytes) 000014[0] 1736984521.520200 write 0x357ff3 thru 0x35817e (0x18c bytes) 000015[0] 1736984521.529952 mapread 0x3b17cb thru 0x3b1d03 (0x539 bytes) 000016[0] 1736984521.535895 mapwrite 0x1c40c2 thru 0x1c4a27 (0x966 bytes) 000017[0] 1736984521.619724 mapwrite 0x9abe66 thru 0x9adb64 (0x1cff bytes) 000018[0] 1736984521.700401 read 0x17e051 thru 0x17f8b0 (0x1860 bytes) truncating to largest ever: 0x9d33d4 000019[1] 1736984521.715677 trunc from 0x9adb65 to 0x9d33d3 (0x2586f bytes) 000021[1] 1736984521.757131 read 0x54b37f thru 0x54c3e5 (0x1067 bytes) 000022[1] 1736984521.762862 write 0x3a38b4 thru 0x3a4fab (0x16f8 bytes) 000025[0] 1736984521.785716 trunc from 0x9d33d4 to 0x2eae1b (0x6e85b8 bytes) 000026[1] 1736984521.822048 read 0x033d87 thru 0x03451d (0x797 bytes) 000027[1] 1736984521.856061 trunc from 0x2eae1c to 0x5874f9 (0x29c6de bytes) 000030[1] 1736984521.906700 write 0x70e1b7 thru 0x70e9e1 (0x82b bytes) 000031[0] 1736984521.907825 mapread 0x046d9b thru 0x047cfe (0xf64 bytes) 000032[0] 1736984521.927895 read 0x133b6f thru 0x134c8e (0x1120 bytes) 000033[0] 1736984521.982344 read 0x481577 thru 0x482f99 (0x1a23 bytes) 000035[1] 1736984521.987330 trunc from 0x70e9e2 to 0x42efe3 (0x2df9fe bytes) 000036[1] 1736984522.024344 mapread 0x3b9a16 thru 0x3b9cf8 (0x2e3 bytes) 000037[0] 1736984522.083908 read 0x1fa912 thru 0x1fb05a (0x749 bytes) 000038[0] 1736984522.107042 write 0x3487d5 thru 0x34897c (0x1a8 bytes) 000039[0] 1736984522.126659 write 0x3b1e66 thru 0x3b32ca (0x1465 bytes) 000041[1] 1736984522.149060 trunc from 0x42efe4 to 0x919582 (0x4ea59f bytes) 000042[1] 1736984522.176701 trunc from 0x919583 to 0x4ef7a3 (0x429ddf bytes) 000043[0] 1736984522.197324 read 0x0c8ce1 thru 0x0ca3da (0x16fa bytes) 000044[0] 1736984522.224050 mapread 0x4ef0d4 thru 0x4ef7a3 (0x6d0 bytes) 000045[0] 1736984522.236333 mapwrite 0x7554b5 thru 0x75683a (0x1386 bytes) 000048[1] 1736984522.376072 read 0x002d41 thru 0x004500 (0x17c0 bytes) 000050[1] 1736984522.424988 mapwrite 0x94fe14 thru 0x9500d9 (0x2c6 bytes) 000051[0] 1736984522.567665 mapwrite 0x8ffbce thru 0x9000b8 (0x4eb bytes) 000052[1] 1736984522.697827 mapread 0x2ba583 thru 0x2bc431 (0x1eaf bytes) 000053[0] 1736984522.731329 write 0x0678cb thru 0x067ab5 (0x1eb bytes) 000054[1] 1736984522.745952 read 0x73d52b thru 0x73e059 (0xb2f bytes) 000055[1] 1736984522.767995 mapread 0x2c3a8f thru 0x2c5522 (0x1a94 bytes) 000056[1] 1736984522.805799 write 0x649d3b thru 0x64ad35 (0xffb bytes) 000057[1] 1736984522.819124 mapread 0x31fce2 thru 0x321a33 (0x1d52 bytes) 000059[1] 1736984522.855209 mapwrite 0x48bf81 thru 0x48d862 (0x18e2 bytes) 000061[0] 1736984522.935722 trunc from 0x9500da to 0x8c188c (0x8e84d bytes) 000062[0] 1736984522.976777 write 0x4325d8 thru 0x434327 (0x1d50 bytes) 000064[1] 1736984523.005785 write 0x94642d thru 0x9467bc (0x390 bytes) 000065[1] 1736984523.033223 trunc from 0x9467bd to 0x44ad58 (0x4fba64 bytes) 000067[0] 1736984523.091639 read 0x1bf771 thru 0x1bffb2 (0x842 bytes) 000068[1] 1736984523.124986 mapread 0x035e5c thru 0x036c17 (0xdbc bytes) 000069[0] 1736984523.141339 mapread 0x3f6608 thru 0x3f7efb (0x18f4 bytes) 000071[0] 1736984523.154529 write 0x37b35e thru 0x37ca22 (0x16c5 bytes) 000074[1] 1736984523.194122 trunc from 0x44ad59 to 0x47b0eb (0x30393 bytes) 000075[1] 1736984523.220828 mapread 0x479425 thru 0x47983b (0x417 bytes) 000076[0] 1736984523.251199 mapread 0x1078e8 thru 0x10808a (0x7a3 bytes) 000078[0] 1736984523.273725 write 0x64e780 thru 0x64f089 (0x90a bytes) 000079[1] 1736984523.293929 read 0x134f7b thru 0x1363a1 (0x1427 bytes) 000080[0] 1736984523.312575 trunc from 0x64f08a to 0x8013a1 (0x1b2318 bytes) 000082[1] 1736984523.338352 mapwrite 0x3fbc9f thru 0x3fc93f (0xca1 bytes) 000083[1] 1736984523.424621 read 0x25d8a9 thru 0x25de18 (0x570 bytes) 000087[0] 1736984523.520660 trunc from 0x8013a2 to 0x1f9e55 (0x60754c bytes) 000092[0] 1736984523.589853 write 0x4d9af3 thru 0x4da2eb (0x7f9 bytes) 000093[0] 1736984523.591059 write 0x0dde78 thru 0x0de75b (0x8e4 bytes) 000094[1] 1736984523.608938 mapread 0x1f911d thru 0x1fae19 (0x1cfd bytes) 000098[0] 1736984523.671615 read 0x44ad64 thru 0x44c24c (0x14e9 bytes) 000100[1] 1736984523.672149 write 0x9c9d6d thru 0x9ca815 (0xaa9 bytes) 000101[0] 1736984523.695659 read 0x99abbd thru 0x99ae64 (0x2a8 bytes) 000104[1] 1736984523.724697 read 0x871f19 thru 0x873596 (0x167e bytes) 000105[1] 1736984523.758733 trunc from 0x9ca816 to 0x0b7979 (0x912e9c bytes) 000106[1] 1736984523.795330 mapread 0x07175f thru 0x072264 (0xb06 bytes) 000107[1] 1736984523.805802 mapread 0x0316a6 thru 0x032585 (0xee0 bytes) 000108[0] 1736984523.829831 read 0x06fa27 thru 0x06fdd2 (0x3ac bytes) 000110[0] 1736984523.851902 trunc from 0x0b797a to 0x17273d (0xbadc4 bytes) 000114[0] 1736984523.914118 mapwrite 0x7b3f2e thru 0x7b4819 (0x8ec bytes) 000115[1] 1736984524.046998 mapread 0x5f90cc thru 0x5f95bb (0x4f0 bytes) 000119[0] 1736984524.102111 read 0x0652b9 thru 0x066463 (0x11ab bytes) 000124[1] 1736984524.192686 write 0x995d7e thru 0x99653e (0x7c1 bytes) 000133[1] 1736984524.281113 mapwrite 0x89098d thru 0x8924cb (0x1b3f bytes) 000134[1] 1736984524.382522 write 0x04717d thru 0x0489a4 (0x1828 bytes) 000136[1] 1736984524.393944 write 0x8f5991 thru 0x8f7889 (0x1ef9 bytes) 000138[1] 1736984524.431465 write 0x0dcfcc thru 0x0dd314 (0x349 bytes) 000141[1] 1736984524.453632 write 0x3796dd thru 0x37a87b (0x119f bytes) 000142[0] 1736984524.474115 mapwrite 0x822a84 thru 0x8245f2 (0x1b6f bytes) 000143[1] 1736984524.572073 write 0x823987 thru 0x82492d (0xfa7 bytes) 000145[1] 1736984524.584862 read 0x5c20ac thru 0x5c33dd (0x1332 bytes) 000148[0] 1736984524.614319 read 0x771102 thru 0x772736 (0x1635 bytes) 000149[1] 1736984524.645019 read 0x24c312 thru 0x24d040 (0xd2f bytes) 000150[0] 1736984524.730156 mapread 0x7a8665 thru 0x7a8d39 (0x6d5 bytes) 000152[0] 1736984524.739794 mapwrite 0x977aa5 thru 0x97816d (0x6c9 bytes) 000153[1] 1736984524.833727 write 0x857fbb thru 0x8596df (0x1725 bytes) 000155[1] 1736984524.855837 read 0x2f6980 thru 0x2f7b44 (0x11c5 bytes) 000156[1] 1736984524.884084 read 0x795733 thru 0x7969ae (0x127c bytes) 000157[1] 1736984524.914585 write 0x438559 thru 0x43a538 (0x1fe0 bytes) 000160[0] 1736984524.915478 mapwrite 0x4d9aac thru 0x4da284 (0x7d9 bytes) 000161[1] 1736984524.988106 read 0x0ce6e9 thru 0x0ce877 (0x18f bytes) 000164[0] 1736984525.093259 mapread 0x5c65f3 thru 0x5c6f79 (0x987 bytes) 000165[1] 1736984525.097928 mapread 0x4adfd4 thru 0x4ae61c (0x649 bytes) 000166[0] 1736984525.099146 mapwrite 0x3c6aa5 thru 0x3c7fe7 (0x1543 bytes) 000167[1] 1736984525.173867 mapwrite 0x01cc43 thru 0x01dbab (0xf69 bytes) 000168[1] 1736984525.266523 mapread 0x73f139 thru 0x73f6f7 (0x5bf bytes) 000169[1] 1736984525.280493 trunc from 0x99653f to 0x51ba89 (0x47aab5 bytes) 000172[0] 1736984525.316502 read 0x4c2853 thru 0x4c38ed (0x109b bytes) 000173[0] 1736984525.327453 write 0x1cabec thru 0x1cb535 (0x94a bytes) 000175[1] 1736984525.341286 mapread 0x04cb90 thru 0x04cdaa (0x21b bytes) 000176[1] 1736984525.347572 trunc from 0x51ba8a to 0x6dd90b (0x1c1e82 bytes) 000177[0] 1736984525.405373 write 0x6f6c0b thru 0x6f83b1 (0x17a7 bytes) 000179[0] 1736984525.443970 trunc from 0x6f83b2 to 0x25bef6 (0x49c4bb bytes) 000182[0] 1736984525.502258 mapread 0x0b7868 thru 0x0b8b00 (0x1299 bytes) 000183[1] 1736984525.526770 mapread 0x11ffea thru 0x1209e0 (0x9f7 bytes) 000185[0] 1736984525.570137 mapwrite 0x4e78fd thru 0x4e8865 (0xf69 bytes) 000186[1] 1736984525.662506 read 0x29f24a thru 0x2a0c4a (0x1a01 bytes) 000187[0] 1736984525.695169 mapread 0x1acfb4 thru 0x1ae7c8 (0x1815 bytes) 000189[1] 1736984525.708535 mapwrite 0x8b3df4 thru 0x8b5b60 (0x1d6d bytes) 000190[1] 1736984525.884194 mapread 0x206c4f thru 0x208540 (0x18f2 bytes) 000192[0] 1736984525.919531 write 0x807959 thru 0x807eb5 (0x55d bytes) 000195[0] 1736984525.972930 read 0x6ed324 thru 0x6ed6c8 (0x3a5 bytes) 000196[1] 1736984525.994154 mapread 0x0ce785 thru 0x0d03c1 (0x1c3d bytes) 000198[1] 1736984526.028153 trunc from 0x8b5b61 to 0x185c7e (0x72fee2 bytes) 000200[1] 1736984526.056869 trunc from 0x185c7f to 0x6afee7 (0x52a269 bytes) 000202[0] 1736984526.082230 trunc from 0x6afee8 to 0x6db0ed (0x2b206 bytes) 000204[1] 1736984526.125409 mapread 0x44e102 thru 0x44ffc4 (0x1ec3 bytes) 000205[1] 1736984526.164399 read 0x6656e1 thru 0x666f27 (0x1847 bytes) 000206[0] 1736984526.191981 write 0x6aabf9 thru 0x6ac586 (0x198e bytes) 000207[0] 1736984526.208133 write 0x069f55 thru 0x06b053 (0x10ff bytes) 000208[0] 1736984526.245307 mapwrite 0x82d1d7 thru 0x82ed0b (0x1b35 bytes) 000210[1] 1736984526.372464 read 0x32abcf thru 0x32be0c (0x123e bytes) 000212[0] 1736984526.403916 mapwrite 0x4b5f22 thru 0x4b6273 (0x352 bytes) 000214[1] 1736984526.494597 read 0x03884c thru 0x03a63f (0x1df4 bytes) 000215[0] 1736984526.564633 write 0x3a3d16 thru 0x3a4c22 (0xf0d bytes) 000216[0] 1736984526.586978 mapread 0x2045c7 thru 0x2058c1 (0x12fb bytes) 000217[0] 1736984526.594083 mapwrite 0x262ecf thru 0x264672 (0x17a4 bytes) 000224[0] 1736984526.677533 read 0x7be90e thru 0x7c0602 (0x1cf5 bytes) 000225[1] 1736984526.703150 mapwrite 0x86e0fa thru 0x86fd4f (0x1c56 bytes) 000226[1] 1736984526.813405 read 0x6109c2 thru 0x61203c (0x167b bytes) 000228[1] 1736984526.877640 read 0x435b46 thru 0x436b02 (0xfbd bytes) 000230[0] 1736984526.923595 read 0x173edc thru 0x174236 (0x35b bytes) 000231[1] 1736984526.950374 read 0x2c41dc thru 0x2c6161 (0x1f86 bytes) 000232[0] 1736984526.982773 read 0x3c0445 thru 0x3c182f (0x13eb bytes) 000234[1] 1736984527.031693 trunc from 0x86fd50 to 0x40687b (0x4694d4 bytes) 000235[0] 1736984527.078577 read 0x12da34 thru 0x12e867 (0xe34 bytes) 000236[1] 1736984527.105460 read 0x12f0f4 thru 0x12f6c4 (0x5d1 bytes) 000237[1] 1736984527.144535 write 0x68aa2c thru 0x68c2fd (0x18d2 bytes) 000238[0] 1736984527.149570 mapwrite 0x1b924c thru 0x1b9f12 (0xcc7 bytes) 000245[0] 1736984527.297251 trunc from 0x68c2fe to 0x296a23 (0x3f58da bytes) 000249[1] 1736984527.347897 trunc from 0x296a24 to 0x309a0a (0x72fe7 bytes) 000250[0] 1736984527.367689 mapread 0x289cda thru 0x289f85 (0x2ac bytes) 000252[0] 1736984527.400467 trunc from 0x309a0b to 0x54a3af (0x2409a5 bytes) 000253[1] 1736984527.431593 mapwrite 0x392ac4 thru 0x3945d6 (0x1b13 bytes) 000254[1] 1736984527.516213 mapread 0x4bba82 thru 0x4bd88d (0x1e0c bytes) 000255[1] 1736984527.567860 mapwrite 0x0aa549 thru 0x0aad3e (0x7f6 bytes) 000257[1] 1736984527.649083 read 0x1aba43 thru 0x1ac937 (0xef5 bytes) 000258[1] 1736984527.685622 write 0x33e639 thru 0x33fa7a (0x1442 bytes) 000260[1] 1736984527.686245 read 0x13088d thru 0x131f8d (0x1701 bytes) 000263[1] 1736984527.709534 trunc from 0x54a3b0 to 0x49c28f (0xae120 bytes) 000266[1] 1736984527.728468 read 0x3e6cef thru 0x3e7abe (0xdd0 bytes) 000268[0] 1736984527.748818 trunc from 0x49c290 to 0x9a5685 (0x5093f6 bytes) 000270[1] 1736984527.819927 read 0x98edc0 thru 0x98faa4 (0xce5 bytes) 000271[1] 1736984527.839485 trunc from 0x9a5686 to 0x0de365 (0x8c7320 bytes) 000273[1] 1736984527.880017 write 0x44ce24 thru 0x44df47 (0x1124 bytes) 000274[0] 1736984527.894142 read 0x185d6e thru 0x1879ef (0x1c82 bytes) 000275[1] 1736984527.943072 read 0x3d414b thru 0x3d6012 (0x1ec8 bytes) 000276[0] 1736984527.961734 mapread 0x06eb00 thru 0x06eb2e (0x2f bytes) 000277[0] 1736984528.016367 read 0x3d6afb thru 0x3d8808 (0x1d0e bytes) 000278[0] 1736984528.016738 write 0x6bd613 thru 0x6bda3a (0x428 bytes) 000282[1] 1736984528.046462 write 0x27eebe thru 0x27f713 (0x856 bytes) 000283[0] 1736984528.097702 mapwrite 0x832b05 thru 0x8337f3 (0xcef bytes) 000284[0] 1736984528.200669 mapread 0x60feba thru 0x610008 (0x14f bytes) 000285[0] 1736984528.214198 read 0x7072a3 thru 0x708526 (0x1284 bytes) 000287[0] 1736984528.231049 write 0x1a99d5 thru 0x1aac43 (0x126f bytes) 000292[0] 1736984528.260205 write 0x8dca04 thru 0x8de653 (0x1c50 bytes) 000293[1] 1736984528.261270 read 0x12fbc8 thru 0x12fe23 (0x25c bytes) 000294[1] 1736984528.305511 read 0x65f92a thru 0x6601ee (0x8c5 bytes) 000298[0] 1736984528.346104 mapread 0x5d86fd thru 0x5d9c62 (0x1566 bytes) 000299[0] 1736984528.364863 read 0x72db4c thru 0x72f697 (0x1b4c bytes) 000300[0] 1736984528.383477 trunc from 0x8de654 to 0x990ba9 (0xb2556 bytes) 000301[0] 1736984528.417197 trunc from 0x990baa to 0x1a7f8c (0x7e8c1d bytes) 000302[1] 1736984528.428333 mapread 0x0e3b32 thru 0x0e55f7 (0x1ac6 bytes) 000303[0] 1736984528.465803 write 0x7b9e01 thru 0x7bb3cf (0x15cf bytes) 000304[1] 1736984528.481872 read 0x070177 thru 0x0706d0 (0x55a bytes) 000305[1] 1736984528.506530 trunc from 0x7bb3d0 to 0x2bc9ba (0x4fea15 bytes) 000307[0] 1736984528.544237 read 0x29d659 thru 0x29ed8a (0x1732 bytes) 000310[1] 1736984528.581907 trunc from 0x2bc9bb to 0x41fde5 (0x16342b bytes) 000311[0] 1736984528.614741 mapwrite 0x66f53d thru 0x66f541 (0x5 bytes) 000313[1] 1736984528.731591 mapwrite 0x081974 thru 0x082075 (0x702 bytes) 000317[1] 1736984528.856555 mapread 0x1f77f0 thru 0x1f78e5 (0xf6 bytes) 000319[1] 1736984528.970022 trunc from 0x66f542 to 0x6d47ec (0x652ab bytes) 000320[0] 1736984529.001283 mapread 0x23c95c thru 0x23e1c4 (0x1869 bytes) 000322[1] 1736984529.057599 write 0x6ed05e thru 0x6ed612 (0x5b5 bytes) 000324[1] 1736984529.077416 trunc from 0x6ed613 to 0x408cbf (0x2e4953 bytes) 000328[1] 1736984529.115863 read 0x0efd7d thru 0x0f12a1 (0x1525 bytes) 000329[1] 1736984529.172519 read 0x241941 thru 0x242270 (0x930 bytes) 000330[0] 1736984529.172933 read 0x2e9766 thru 0x2ea320 (0xbbb bytes) 000332[1] 1736984529.187601 mapread 0x0c6eac thru 0x0c7b52 (0xca7 bytes) 000333[1] 1736984529.192225 mapwrite 0x582238 thru 0x583a9c (0x1865 bytes) 000335[1] 1736984529.280463 write 0x1d5c01 thru 0x1d72f4 (0x16f4 bytes) 000336[1] 1736984529.280907 write 0x397734 thru 0x398c8c (0x1559 bytes) 000337[1] 1736984529.290805 trunc from 0x583a9d to 0x9ce70e (0x44ac72 bytes) 000338[0] 1736984529.311953 write 0x345b8e thru 0x346b89 (0xffc bytes) 000339[0] 1736984529.349814 write 0x7542a1 thru 0x754409 (0x169 bytes) 000344[1] 1736984529.400103 read 0x3c43f8 thru 0x3c5183 (0xd8c bytes) 000348[0] 1736984529.486638 mapread 0x07dd32 thru 0x07e177 (0x446 bytes) 000349[1] 1736984529.574794 write 0x5f0dd1 thru 0x5f2d65 (0x1f95 bytes) 000350[0] 1736984529.607622 mapread 0x360332 thru 0x360b88 (0x857 bytes) 000353[0] 1736984529.632667 read 0x397116 thru 0x398dee (0x1cd9 bytes) 000359[0] 1736984529.669312 mapread 0x127097 thru 0x128760 (0x16ca bytes) 000360[0] 1736984529.678565 mapwrite 0x2f1a98 thru 0x2f3996 (0x1eff bytes) 000362[1] 1736984529.767682 trunc from 0x9ce70f to 0x2b9c96 (0x714a78 bytes) 000364[1] 1736984529.822923 mapread 0x17d5a0 thru 0x17e4d0 (0xf31 bytes) 000365[0] 1736984529.843226 mapread 0x0c66fd thru 0x0c796a (0x126e bytes) 000366[0] 1736984529.872689 trunc from 0x2b9c97 to 0x0b8aa0 (0x2011f6 bytes) 000369[1] 1736984529.930253 mapwrite 0x2c7da5 thru 0x2c837e (0x5da bytes) 000370[1] 1736984530.004288 read 0x1278e4 thru 0x128d72 (0x148f bytes) 000372[0] 1736984530.043172 write 0x3bdc73 thru 0x3bdcd6 (0x64 bytes) 000373[0] 1736984530.062646 read 0x38a9cd thru 0x38abc1 (0x1f5 bytes) 000375[1] 1736984530.087057 read 0x1a33a7 thru 0x1a50b4 (0x1d0e bytes) 000376[1] 1736984530.131245 read 0x00d8e3 thru 0x00f858 (0x1f76 bytes) 000378[0] 1736984530.170617 write 0x59ac9e thru 0x59c1a9 (0x150c bytes) 000379[1] 1736984530.193836 read 0x329697 thru 0x32991a (0x284 bytes) 000380[0] 1736984530.250457 read 0x006845 thru 0x006915 (0xd1 bytes) 000381[1] 1736984530.280943 mapread 0x0d5553 thru 0x0d6445 (0xef3 bytes) 000385[0] 1736984530.290541 mapread 0x224e93 thru 0x22582a (0x998 bytes) 000388[0] 1736984530.329113 mapwrite 0x60f860 thru 0x6112db (0x1a7c bytes) 000389[1] 1736984530.458613 mapwrite 0x55f03d thru 0x560ab7 (0x1a7b bytes) 000393[0] 1736984530.575437 write 0x2482e1 thru 0x24992c (0x164c bytes) 000394[0] 1736984530.592277 read 0x5ecbed thru 0x5ee244 (0x1658 bytes) 000395[1] 1736984530.616999 read 0x34ef4b thru 0x34f76d (0x823 bytes) 000396[1] 1736984530.630543 mapwrite 0x64739d thru 0x648691 (0x12f5 bytes) 000398[0] 1736984530.770074 read 0x018807 thru 0x019ffd (0x17f7 bytes) 000399[1] 1736984530.808733 read 0x53e7df thru 0x53fe15 (0x1637 bytes) 000402[0] 1736984530.826718 trunc from 0x648692 to 0x774a9b (0x12c40a bytes) 000404[1] 1736984530.893392 mapread 0x1f8e24 thru 0x1f9489 (0x666 bytes) 000405[1] 1736984530.924887 mapread 0x67433b thru 0x6760d4 (0x1d9a bytes) 000408[1] 1736984530.946330 trunc from 0x774a9c to 0x401546 (0x373555 bytes) 000410[1] 1736984530.991290 mapread 0x061c34 thru 0x062d65 (0x1132 bytes) 000411[1] 1736984531.029687 mapwrite 0x3c388b thru 0x3c3f1b (0x691 bytes) 000412[0] 1736984531.094121 write 0x7a6939 thru 0x7a7beb (0x12b3 bytes) 000413[0] 1736984531.121090 mapread 0x5ec2e7 thru 0x5ed7cb (0x14e5 bytes) 000417[1] 1736984531.230409 mapread 0x1f58e9 thru 0x1f7054 (0x176c bytes) 000421[1] 1736984531.268647 mapwrite 0x9d1607 thru 0x9d2ea3 (0x189d bytes) 000422[0] 1736984531.361213 read 0x5c65fe thru 0x5c7dc5 (0x17c8 bytes) 000424[0] 1736984531.412220 trunc from 0x9d2ea4 to 0x5429c9 (0x4904da bytes) 000425[0] 1736984531.449493 read 0x22d968 thru 0x22f4b4 (0x1b4d bytes) 000428[1] 1736984531.508067 mapwrite 0x67daf1 thru 0x67f8c7 (0x1dd7 bytes) 000430[1] 1736984531.663222 write 0x3d34a8 thru 0x3d380a (0x363 bytes) 000431[1] 1736984531.668324 mapread 0x04bdcd thru 0x04cff8 (0x122c bytes) 000432[0] 1736984531.689748 trunc from 0x67f8c8 to 0x08a484 (0x5f5443 bytes) 000433[0] 1736984531.723698 trunc from 0x08a485 to 0x7ec718 (0x762294 bytes) 000434[0] 1736984531.740845 trunc from 0x7ec719 to 0x59575e (0x256fba bytes) 000435[0] 1736984531.752208 trunc from 0x59575f to 0x6906e3 (0xfaf85 bytes) 000437[1] 1736984531.781028 mapwrite 0x38d93c thru 0x38f5fc (0x1cc1 bytes) 000438[1] 1736984531.891250 write 0x9d7436 thru 0x9d82c3 (0xe8e bytes) 000440[1] 1736984531.919990 mapread 0x7c8cc5 thru 0x7caba7 (0x1ee3 bytes) 000441[0] 1736984531.955632 write 0x9a7b87 thru 0x9a9ac5 (0x1f3f bytes) 000442[0] 1736984532.027330 mapwrite 0x72e693 thru 0x72e74d (0xbb bytes) 000447[0] 1736984532.083471 read 0x2b8f13 thru 0x2bad65 (0x1e53 bytes) 000449[0] 1736984532.116286 mapread 0x009d25 thru 0x00a10b (0x3e7 bytes) 000450[0] 1736984532.152181 trunc from 0x9d82c4 to 0x1a1b72 (0x836751 bytes) 000452[0] 1736984532.211502 write 0x93c312 thru 0x93d4c1 (0x11b0 bytes) 000453[1] 1736984532.237153 mapread 0x89322d thru 0x8948c9 (0x169d bytes) 000454[1] 1736984532.262812 mapwrite 0x339cf1 thru 0x33af83 (0x1293 bytes) 000455[1] 1736984532.328173 read 0x107f7c thru 0x10869b (0x720 bytes) 000456[0] 1736984532.403819 write 0x82c7a2 thru 0x82e08e (0x18ed bytes) 000459[1] 1736984532.467235 read 0x791bf2 thru 0x791f7d (0x38c bytes) 000460[1] 1736984532.485933 write 0x7ffe34 thru 0x8015e3 (0x17b0 bytes) 000461[0] 1736984532.501281 write 0x05c98e thru 0x05d1c8 (0x83b bytes) 000462[0] 1736984532.525826 read 0x31c400 thru 0x31cd66 (0x967 bytes) 000463[1] 1736984532.613502 trunc from 0x93d4c2 to 0x74f282 (0x1ee23f bytes) 000464[1] 1736984532.646893 trunc from 0x74f283 to 0x77b572 (0x2c2f0 bytes) 000465[0] 1736984532.656628 mapread 0x52dc9b thru 0x52f822 (0x1b88 bytes) 000466[0] 1736984532.677698 mapread 0x0ed919 thru 0x0eec5a (0x1342 bytes) 000468[0] 1736984532.700635 trunc from 0x77b573 to 0x875c7b (0xfa709 bytes) 000469[0] 1736984532.734075 write 0x43e8f6 thru 0x43ffc4 (0x16cf bytes) 000470[0] 1736984532.757039 mapwrite 0x2bceec thru 0x2be5c2 (0x16d7 bytes) 000475[1] 1736984532.871124 mapwrite 0x9ad217 thru 0x9ad3d4 (0x1be bytes) 000476[1] 1736984532.960084 mapread 0x44e962 thru 0x450832 (0x1ed1 bytes) 000480[1] 1736984532.991601 read 0x4f3eef thru 0x4f417a (0x28c bytes) 000482[1] 1736984533.065068 write 0x4e5816 thru 0x4e6844 (0x102f bytes) 000483[1] 1736984533.072813 read 0x3b809e thru 0x3b8d34 (0xc97 bytes) 000484[0] 1736984533.091225 read 0x6ab3fa thru 0x6aba60 (0x667 bytes) 000485[1] 1736984533.131516 trunc from 0x9ad3d5 to 0x4fb848 (0x4b1b8c bytes) 000490[0] 1736984533.181699 write 0x013ba3 thru 0x014406 (0x864 bytes) 000491[1] 1736984533.202722 mapwrite 0x63273d thru 0x6336ad (0xf71 bytes) 000494[0] 1736984533.307168 mapwrite 0x09661d thru 0x098175 (0x1b59 bytes) 000497[1] 1736984533.401508 mapread 0x5db515 thru 0x5dd2c7 (0x1db3 bytes) All operations completed A-OK! PASS 16b (17s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 18:42:15 (1736984535) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (2s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 18:42:17 (1736984537) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.182815 s, 11.5 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.169388 s, 12.4 MB/s ldlm.namespaces.MGC192.168.204.158@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aa144800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff88013731d000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800aa144800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800aa144800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff88013731d000.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 7.33055 s, 14.3 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.60956 s, 65.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.14194 s, 91.8 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 3.02482 s, 34.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.48106 s, 19.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.43958 s, 72.8 MB/s PASS 16d (26s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 18:42:43 (1736984563) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0834336 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0603444 s, 17.4 MB/s PASS 16e (3s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 18:42:46 (1736984566) TIMEOUT PASS 16f (22s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 18:43:09 (1736984589) TIMEOUT PASS 16g (22s) == sanityn test 16i: read after truncate file ============ 18:43:31 (1736984611) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (2s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 18:43:33 (1736984613) fail_loc=0x8000030a PASS 17 (4s) == sanityn test 18: mmap sanity check =========================================================================================== 18:43:37 (1736984617) mmap test1: basic mmap operation (PASS, 0.032333s) mmap test2: MAP_PRIVATE not write back (PASS, 0.022613s) mmap test3: concurrent mmap ops on two nodes (PASS, 9.443s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.1615s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.050047s) mmap test6: check mmap write/read content on two nodes (PASS, 0.054062s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0274s) PASS 18 (16s) == sanityn test 19: test concurrent uncached read races ========================================================================= 18:43:54 (1736984634) oleg458-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg458-client: oleg458-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (2s) == sanityn test 20: test extra readahead page left in cache ============================================================== 18:43:55 (1736984635) PASS 20 (2s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 18:43:58 (1736984638) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (3s) == sanityn test 23: others should see updated atime while another read============================================================== 18:44:00 (1736984640) now is 1736984641 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6873 new atime is 1736984702 PASS 23 (63s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 18:45:03 (1736984703) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 374 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 375 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.1K 368 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 374 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 375 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 368 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 8.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 5.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 13.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117974 374 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118071 375 117696 1% /mnt/lustre2[OST:1] filesystem_summary: 235664 368 235296 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 374 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 375 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 368 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 117974 374 117600 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118071 375 117696 1% /mnt/lustre[OST:1] filesystem_summary: 235664 368 235296 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117974 374 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118071 375 117696 1% /mnt/lustre2[OST:1] filesystem_summary: 235664 368 235296 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre2 PASS 24a (3s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 18:45:06 (1736984706) PASS 24b (2s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 18:45:08 (1736984708) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (2s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 18:45:10 (1736984710) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 18:45:11 (1736984711) utime: good mknod times 1736984710 <= 1736984711 <= 1736984711 for /mnt/lustre/f26a utime: good mknod times 1736984710 <= 1736984711 <= 1736984711 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (2s) == sanityn test 26b: sync mtime between ost and mds ====== 18:45:13 (1736984713) PASS 26b (4s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 18:45:18 (1736984718) 0 PASS 30 (5s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 18:45:23 (1736984723) fail_loc=0x314 PASS 31a (3s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 18:45:26 (1736984726) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg458-server mds-ost sync done. sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.158@tcp:/lustre 7542784 13312 7525376 1% /mnt/lustre2 PASS 31b (12s) == sanityn test 31r: open-rename(replace) race =========== 18:45:38 (1736984738) -rw-r--r-- 1 root root 0 Jan 15 18:45 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (5s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 18:45:44 (1736984744) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 18:45:45 (1736984745) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 18:45:46 (1736984746) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 18:45:47 (1736984747) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 18:45:48 (1736984748) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg458-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa144800.ost_server_uuid,osc.lustre-OST0000-osc-ffff88013731d000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800aa144800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff88013731d000.ost_server_uuid in FULL state after 0 sec oleg458-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa144800.ost_server_uuid,osc.lustre-OST0001-osc-ffff88013731d000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800aa144800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff88013731d000.ost_server_uuid in IDLE state after 0 sec oleg458-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa144800.ost_server_uuid,osc.lustre-OST0000-osc-ffff88013731d000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800aa144800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff88013731d000.ost_server_uuid in IDLE state after 0 sec oleg458-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa144800.ost_server_uuid,osc.lustre-OST0001-osc-ffff88013731d000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800aa144800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff88013731d000.ost_server_uuid in IDLE state after 0 sec oleg458-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa144800.ost_server_uuid,osc.lustre-OST0000-osc-ffff88013731d000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800aa144800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff88013731d000.ost_server_uuid in IDLE state after 0 sec oleg458-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa144800.ost_server_uuid,osc.lustre-OST0001-osc-ffff88013731d000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800aa144800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff88013731d000.ost_server_uuid in IDLE state after 0 sec PASS 34 (89s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 18:47:17 (1736984837) fail_loc=0x80000317 Race attempt 0 Wait for 15539 15637 for 60 sec... - open/close 1478 (time 1736984851.03 total 10.01 last 147.70) - open/close 3084 (time 1736984861.04 total 20.01 last 160.52) total: 4000 open/close in 25.40 seconds: 157.47 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.204.158@tcp:/lustre 7.2G 15M 7.2G 1% /mnt/lustre 192.168.204.158@tcp:/lustre 7.2G 15M 7.2G 1% /mnt/lustre2 PASS 35 (66s) == sanityn test 36: handle ESTALE/open-unlink correctly == 18:48:23 (1736984903) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg458-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.633519 s, 82.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6873 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7523328) after_dd(7470080) after(7521280) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.442002 s, 119 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6873 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7521280) after_dd(7468032) after(7519232) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.400829 s, 131 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6873 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7519232) after_dd(7470080) after(7521280) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.525886 s, 99.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6873 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7521280) after_dd(7468032) after(7519232) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.492533 s, 106 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6873 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7519232) after_dd(7468032) after(7519232) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.347061 s, 151 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6873 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7519232) after_dd(7468032) after(7519232) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.370599 s, 141 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6873 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7519232) after_dd(7468032) after(7519232) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.424809 s, 123 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6873 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7519232) after_dd(7468032) after(7519232) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.417743 s, 126 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6873 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7519232) after_dd(7470080) after(7521280) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.352004 s, 149 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6873 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7521280) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.376826 s, 139 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6873 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7523328) after_dd(7470080) after(7521280) PASS 36 (236s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 18:52:19 (1736985139) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6873 - create 3541 (time 1736985150.82 total 10.00 last 354.00) - create 6825 (time 1736985160.82 total 20.00 last 328.40) total: 10000 create in 29.06 seconds: 344.09 ops/second PASS 37 (33s) == sanityn test 39a: test from 11063 ============================================================================================ 18:52:52 (1736985172) repeat after cancel_lru_locks PASS 39a (3s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 18:52:55 (1736985175) repeat after cancel_lru_locks PASS 39b (4s) == sanityn test 39c: check truncate mtime update ================================================================================ 18:52:59 (1736985179) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 18:53:02 (1736985182) fail_loc=0x411 fail_loc=0 PASS 39d (3s) == sanityn test 40a: pdirops: create vs others ======================================================================== 18:53:05 (1736985185) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (9s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 18:53:14 (1736985194) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (8s) == sanityn test 40c: pdirops: link and others ======================================================================== 18:53:22 (1736985202) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (9s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 18:53:31 (1736985211) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (8s) == sanityn test 40e: pdirops: rename and others ======================================================================== 18:53:39 (1736985219) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (9s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 18:53:48 (1736985228) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (5s) == sanityn test 41b: pdirops: create vs create ======================================================================== 18:53:53 (1736985233) open: File exists Conflict PASS 41b (6s) == sanityn test 41c: pdirops: create vs link ======================================================================== 18:53:59 (1736985239) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (5s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 18:54:04 (1736985244) Conflict PASS 41d (5s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 18:54:09 (1736985249) Conflict PASS 41e (6s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 18:54:15 (1736985255) Conflict PASS 41f (5s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 18:54:20 (1736985260) Conflict PASS 41g (6s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 18:54:26 (1736985266) Conflict PASS 41h (5s) == sanityn test 41i: reint_open: create vs create ======== 18:54:31 (1736985271) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (689s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 19:06:00 (1736985960) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (5s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 19:06:05 (1736985965) open: File exists Conflict PASS 42b (4s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 19:06:09 (1736985969) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (5s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 19:06:14 (1736985974) Conflict PASS 42d (4s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 19:06:18 (1736985978) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (5s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 19:06:23 (1736985983) Conflict PASS 42f (5s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 19:06:28 (1736985988) Conflict PASS 42g (4s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 19:06:32 (1736985992) Conflict PASS 42h (5s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 19:06:37 (1736985997) PASS 43a (12s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 19:06:49 (1736986009) Conflict PASS 43b (5s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 19:06:54 (1736986014) Conflict PASS 43c (4s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 19:06:58 (1736986018) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (5s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 19:07:03 (1736986023) Conflict PASS 43e (4s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 19:07:07 (1736986027) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (5s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 19:07:12 (1736986032) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (4s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 19:07:16 (1736986036) Conflict PASS 43h (4s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 19:07:20 (1736986040) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 19:07:21 (1736986041) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (33s) == sanityn test 43k: unlink vs create ==================== 19:07:54 (1736986074) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (353s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 19:13:47 (1736986427) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Wed Jan 15 19:13:50 EST 2025 PASS 44a (4s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 19:13:52 (1736986432) open: File exists Conflict PASS 44b (4s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 19:13:56 (1736986436) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (4s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 19:14:00 (1736986440) Conflict PASS 44d (5s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 19:14:05 (1736986445) Conflict PASS 44e (4s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 19:14:09 (1736986449) Conflict PASS 44f (4s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 19:14:13 (1736986453) Conflict PASS 44g (5s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 19:14:18 (1736986458) Conflict PASS 44h (4s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 19:14:22 (1736986462) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 19:14:23 (1736986463) PASS 45a (21s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 19:14:44 (1736986484) Conflict PASS 45b (4s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 19:14:48 (1736986488) Conflict PASS 45c (4s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 19:14:52 (1736986492) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (4s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 19:14:56 (1736986496) Conflict PASS 45e (5s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 19:15:01 (1736986501) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (4s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 19:15:05 (1736986505) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (4s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 19:15:09 (1736986509) Conflict PASS 45h (4s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 19:15:13 (1736986513) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 19:15:14 (1736986514) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (356s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 19:21:10 (1736986870) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (4s) == sanityn test 46b: pdirops: link vs create ======================================================================== 19:21:15 (1736986875) open: File exists Conflict PASS 46b (5s) == sanityn test 46c: pdirops: link vs link ======================================================================== 19:21:19 (1736986879) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (4s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 19:21:23 (1736986883) Conflict PASS 46d (4s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 19:21:27 (1736986887) Conflict PASS 46e (4s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 19:21:31 (1736986891) Conflict PASS 46f (4s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 19:21:35 (1736986895) Conflict PASS 46g (4s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 19:21:39 (1736986899) Conflict PASS 46h (5s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 19:21:44 (1736986904) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (0s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 19:21:44 (1736986904) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 19:21:45 (1736986905) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 19:21:46 (1736986906) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 19:21:47 (1736986907) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (0s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 19:21:47 (1736986907) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 19:21:48 (1736986908) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 19:21:49 (1736986909) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 19:21:50 (1736986910) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.00535579 s, 1.9 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (6s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 19:21:56 (1736986916) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.0031567 s, 50.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (24097) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 19:22:00 (1736986920) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00273129 s, 375 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151092 s, 69.4 MB/s 3145728 PASS 51b (14s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 19:22:14 (1736986934) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00317065 s, 323 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 19:22:20 (1736986940) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00754851 s, 139 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (2s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 19:22:22 (1736986942) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2435 0x983 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2435 0x983 0 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 19:22:26 (1736986946) llite.lustre-ffff8800aa144800.xattr_cache=1 llite.lustre-ffff88013731d000.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504963 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-15 19:22:26.000000000 -0500 Modify: 2025-01-15 19:22:26.000000000 -0500 Change: 2025-01-15 19:22:26.000000000 -0500 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504966 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-15 19:22:31.000000000 -0500 Modify: 2025-01-15 19:22:31.000000000 -0500 Change: 2025-01-15 19:22:31.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504969 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-15 19:22:37.000000000 -0500 Modify: 2025-01-15 19:22:37.000000000 -0500 Change: 2025-01-15 19:22:37.000000000 -0500 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-15 19:22:37.000000000 -0500 Modify: 2025-01-15 19:22:37.000000000 -0500 Change: 2025-01-15 19:22:37.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-15 19:22:42.000000000 -0500 Modify: 2025-01-15 19:22:42.000000000 -0500 Change: 2025-01-15 19:22:42.000000000 -0500 Birth: - File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504972 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-15 19:22:42.000000000 -0500 Modify: 2025-01-15 19:22:42.000000000 -0500 Change: 2025-01-15 19:22:42.000000000 -0500 Birth: - PASS 54 (22s) == sanityn test 55a: rename vs unlink target dir ========= 19:22:48 (1736986968) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 19:22:55 (1736986975) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 19:23:02 (1736986982) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 19:23:14 (1736986994) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) == sanityn test 60: Verify data_version behaviour ======== 19:23:23 (1736987003) PASS 60 (2s) == sanityn test 70a: cd directory && rm directory ======== 19:23:25 (1736987005) PASS 70a (1s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 19:23:26 (1736987006) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 19:23:28 (1736987008) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00237646 s, 27.6 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 19:23:29 (1736987009) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00400317 s, 16.4 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 19:23:30 (1736987010) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (0s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 19:23:30 (1736987010) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.0033524 s, 19.5 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 19:23:31 (1736987011) llite.lustre-ffff8800aa144800.xattr_cache=1 llite.lustre-ffff88013731d000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 19:23:33 (1736987013) llite.lustre-ffff8800aa144800.xattr_cache=1 llite.lustre-ffff88013731d000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (2s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 19:23:35 (1736987015) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00221747 s, 462 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00222142 s, 461 kB/s 2756: get lock2 2757: get lock1 2756: done 2756: get lock1 2757: done 2757: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 2757: cannot get lock2: Resource deadlock avoided 2757: exit rc=0 fcntl cmd 7 failed: Resource deadlock avoided 2756: cannot get lock1: Resource deadlock avoided 2756: done 2756: exit rc=0 PASS 74 (5s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 19:23:40 (1736987020) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0138004 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00730738 s, 143 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 3390 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 19:23:47 (1736987027) opening files: ........ closing files: ........ PASS 76 (13s) == sanityn test 77a: check FIFO NRS policy =============== 19:24:00 (1736987040) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.102255 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848801 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191755 s, 54.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174607 s, 60.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259281 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259633 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178982 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282119 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411253 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273137 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0417252 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446292 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400512 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288329 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.039145 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159935 s, 65.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300945 s, 34.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff88013731d000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104379 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104304 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127929 s, 82.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117077 s, 89.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115323 s, 90.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126355 s, 83.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0033506 s, 313 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012047 s, 87.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116312 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104385 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109111 s, 96.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142573 s, 73.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133643 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104315 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110492 s, 94.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136686 s, 76.7 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) == sanityn test 77b: check CRR-N NRS policy ============== 19:24:03 (1736987043) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0940033 s, 178 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0105993 s, 98.9 MB/s , 0.0106611 s, 98.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275961 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0230909 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400426 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282909 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389117 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398692 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0355182 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0455723 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0520051 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0574658 s, 18.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0494491 s, 21.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0568812 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0536145 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.057323 s, 18.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff88013731d000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00760498 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150615 s, 69.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136029 s, 77.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00788964 s, 133 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0159767 s, 65.6 MB/s , 0.0056902 s, 184 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00448806 s, 234 MB/s , 0.00826168 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00787471 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00863091 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00862396 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010723 s, 97.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198495 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101603 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108078 s, 97.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111725 s, 93.9 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0932744 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100924 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114432 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136672 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175224 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307061 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160424 s, 65.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194991 s, 53.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278227 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0312834 s, 33.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00747363 s, 140 MB/s 1048576 bytes (1.0 MB) copied, 0.0316597 s, 33.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0345127 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321058 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0298323 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129477 s, 81.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88013731d000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0465939 s, 22.5 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88013731d000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00916586 s, 114 MB/s , 0.0015221 s, 689 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00270034 s, 388 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00766201 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00830379 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0084778 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00583812 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108278 s, 96.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00399191 s, 263 MB/s , 0.0025863 s, 405 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00135071 s, 776 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00987517 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00145729 s, 720 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00149109 s, 703 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00122284 s, 857 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0011431 s, 917 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg458-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (2s) == sanityn test 77c: check ORR NRS policy ================ 19:24:06 (1736987046) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.137304 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00858838 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106937 s, 98.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143889 s, 72.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201786 s, 52.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172549 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173866 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014751 s, 71.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281688 s, 37.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447103 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0475605 s, 22.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0302869 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142851 s, 73.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0392421 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293907 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372843 s, 28.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88013731d000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026663 s, 39.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109789 s, 95.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00827238 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00714268 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00761887 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108423 s, 96.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00895062 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00808215 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00923056 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00150291 s, 698 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00749078 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00971379 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102189 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120647 s, 86.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120437 s, 87.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110035 s, 95.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00134604 s, 779 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0869536 s, 193 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00814469 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146171 s, 71.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157747 s, 66.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154477 s, 67.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169177 s, 62.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198747 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190248 s, 55.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0309335 s, 33.9 MB/s , 0.0384499 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0426531 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0375397 s, 27.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0355691 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286656 s, 36.6 MB/s 1048576 bytes (1.0 MB) copied, 0.01368 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196444 s, 53.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236825 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00699023 s, 150 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00687711 s, 152 MB/s , 0.0133242 s, 78.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00727972 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00947481 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00987233 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00939265 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100589 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107038 s, 98.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109018 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115733 s, 90.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127513 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012891 s, 81.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132238 s, 79.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140974 s, 74.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144436 s, 72.6 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0851454 s, 197 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00734158 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168651 s, 62.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166464 s, 63.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278522 s, 37.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017383 s, 60.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0132122 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215922 s, 48.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288885 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291687 s, 35.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023574 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354225 s, 29.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0419277 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316556 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185818 s, 56.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121717 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423656 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00620896 s, 169 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107204 s, 97.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00328421 s, 319 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00830925 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0083436 s, 126 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00147122 s, 713 MB/s , 0.013909 s, 75.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00867707 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109094 s, 96.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0100729 s, 104 MB/s , 0.0110081 s, 95.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101998 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126477 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0128681 s, 81.5 MB/s 1048576 bytes (1.0 MB) copied, 0.012434 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133167 s, 78.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lock_unused_count=2 removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (4s) == sanityn test 77d: check TRR nrs policy ================ 19:24:10 (1736987050) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0846268 s, 198 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00823285 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0098851 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143121 s, 73.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156006 s, 67.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188125 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215865 s, 48.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173552 s, 60.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361682 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166596 s, 62.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0219823 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346328 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164669 s, 63.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216943 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155171 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0422608 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0320488 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00866677 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00824308 s, 127 MB/s 1048576 bytes (1.0 MB) copied, 0.010875 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00773597 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107928 s, 97.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00867503 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110877 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119438 s, 87.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00148988 s, 704 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010265 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00979475 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00738701 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112613 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118574 s, 88.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131235 s, 79.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152726 s, 68.7 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.124363 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112243 s, 93.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121735 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172803 s, 60.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245741 s, 42.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278995 s, 37.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411024 s, 25.5 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0370628 s, 28.3 MB/s , 0.0402438 s, 26.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0439332 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388929 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0406917 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.051023 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0434424 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0498428 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0566327 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0590003 s, 17.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133767 s, 78.4 MB/s , 0.00903592 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139049 s, 75.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00835328 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00876334 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014498 s, 72.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0134825 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00970332 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110202 s, 95.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111306 s, 94.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119786 s, 87.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121493 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117474 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013017 s, 80.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109039 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139074 s, 75.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lock_unused_count=4 removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.091308 s, 184 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131318 s, 79.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134703 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235232 s, 44.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200135 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216069 s, 48.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278119 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176763 s, 59.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187505 s, 55.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172765 s, 60.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020578 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190991 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0287105 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328025 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0310037 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279253 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.046937 s, 22.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00818327 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00850187 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00642442 s, 163 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00814588 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122791 s, 85.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125478 s, 83.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00159025 s, 659 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00969369 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00614729 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106424 s, 98.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102433 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104724 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106402 s, 98.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110398 s, 95.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117768 s, 89.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116689 s, 89.9 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (3s) == sanityn test 77e: check TBF NID nrs policy ============ 19:24:13 (1736987053) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.204.58@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0952363 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108725 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124028 s, 84.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276475 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0374056 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361098 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0400003 s, 26.2 MB/s , 0.0345313 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0418692 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319267 s, 32.8 MB/s , 0.0437949 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0471124 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0397434 s, 26.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0332713 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446653 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0437483 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446784 s, 23.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0109331 s, 95.9 MB/s , 0.010019 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00752773 s, 139 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124364 s, 84.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0113946 s, 92.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117366 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984808 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121292 s, 86.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.011143 s, 94.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127083 s, 82.5 MB/s , 0.0106602 s, 98.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126342 s, 83.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0116681 s, 89.9 MB/s , 0.0129411 s, 81.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136236 s, 77.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156259 s, 67.1 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg458-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.13647 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00812031 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124931 s, 83.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122565 s, 85.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012985 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198739 s, 52.8 MB/s , 0.0315141 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189328 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338716 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219682 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218661 s, 48.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0494339 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0505121 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389808 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.051874 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371906 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0431146 s, 24.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104854 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0082007 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00978431 s, 107 MB/s 1048576 bytes (1.0 MB) copied, 0.00889959 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110603 s, 94.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101063 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00855288 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100148 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110865 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00956543 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112246 s, 93.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0108588 s, 96.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0109853 s, 95.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102627 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010395 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118486 s, 88.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.101026 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00748967 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00809549 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108714 s, 96.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185466 s, 56.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228481 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255979 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229586 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0320271 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194601 s, 53.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266854 s, 39.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207179 s, 50.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247666 s, 42.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292966 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0311817 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145337 s, 72.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222318 s, 47.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88013731d000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110055 s, 95.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0083496 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130446 s, 80.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00865554 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111791 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00894368 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114387 s, 91.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147829 s, 70.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0106762 s, 98.2 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117918 s, 88.9 MB/s , 0.00774374 s, 135 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0119246 s, 87.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lock_unused_count=1 , 0.0125484 s, 83.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012176 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128109 s, 81.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131029 s, 80.0 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.107133 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00887155 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150265 s, 69.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209159 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227819 s, 46.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0349287 s, 30.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027195 s, 38.6 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241654 s, 43.4 MB/s , 0.0267878 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0401437 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217515 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322165 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0339107 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291822 s, 35.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019228 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.028568 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.032647 s, 32.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88013731d000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105014 s, 99.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0112507 s, 93.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00799571 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011686 s, 89.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103887 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100019 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00176436 s, 594 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00711027 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012762 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116126 s, 90.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125384 s, 83.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125907 s, 83.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127796 s, 82.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0158186 s, 66.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0136441 s, 76.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134623 s, 77.9 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (6s) == sanityn test 77f: check TBF JobID nrs policy ========== 19:24:19 (1736987059) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg458-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.126876 s, 132 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104264 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120666 s, 86.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156242 s, 67.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012225 s, 85.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272656 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0340775 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0339846 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0351258 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286313 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360666 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293506 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447827 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255418 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0311406 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0368026 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117503 s, 89.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0066905 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00550906 s, 190 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00912222 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00727916 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122692 s, 85.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00933661 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00591375 s, 177 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0064475 s, 163 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00759425 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00801905 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00841256 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00872269 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985584 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101885 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010296 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108614 s, 96.5 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg458-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.118483 s, 142 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00940815 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143138 s, 73.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137953 s, 76.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0248125 s, 42.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00842623 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246436 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020303 s, 51.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232498 s, 45.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360532 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167723 s, 62.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0467442 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457583 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0408085 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0393962 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0369378 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392049 s, 26.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00735846 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00888329 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0083264 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00771921 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012157 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00954277 s, 110 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00892906 s, 117 MB/s , 0.0119636 s, 87.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103732 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107917 s, 97.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129232 s, 81.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125946 s, 83.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129054 s, 81.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139282 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126882 s, 82.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141242 s, 74.2 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg458-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0993927 s, 169 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0096692 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023541 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.024347 s, 43.1 MB/s , 0.0200533 s, 52.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392822 s, 26.7 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0377501 s, 27.8 MB/s , 0.0389361 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300951 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361383 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411989 s, 25.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0489975 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333958 s, 31.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0305328 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352376 s, 29.8 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88013731d000.lock_unused_count=6 1048576 bytes (1.0 MB) copied, 0.033935 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123381 s, 85.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101633 s, 103 MB/s , 0.014156 s, 74.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131897 s, 79.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107959 s, 97.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115225 s, 91.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013732 s, 76.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155023 s, 67.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0133797 s, 78.4 MB/s , 0.0141064 s, 74.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124355 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134458 s, 78.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132555 s, 79.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0130226 s, 80.5 MB/s , 0.0133077 s, 78.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160682 s, 65.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142631 s, 73.5 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg458-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.107381 s, 156 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104347 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126171 s, 83.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236068 s, 44.4 MB/s , 0.0176883 s, 59.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.029325 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0323316 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291641 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0394911 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0415074 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0418004 s, 25.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0377141 s, 27.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0425816 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390431 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0407118 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0418767 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.042954 s, 24.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff88013731d000.lock_unused_count=2 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00589332 s, 178 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00877044 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00600383 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.009051 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00727072 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00178483 s, 587 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00779637 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108743 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00878832 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00822528 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00668523 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00611162 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984695 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00954157 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00947064 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00974337 s, 108 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (6s) == sanityn test 77g: Change TBF type directly ============ 19:24:25 (1736987065) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (3s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 19:24:28 (1736987068) oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg458-client: oleg458-server: ssh exited with exit code 2 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg458-client: oleg458-server: ssh exited with exit code 12 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg458-client: oleg458-server: ssh exited with exit code 12 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg458-client: oleg458-server: ssh exited with exit code 12 PASS 77h (3s) == sanityn test 77i: Change rank of TBF rule ============= 19:24:31 (1736987071) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (6s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 19:24:37 (1736987077) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0923789 s, 182 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100253 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018734 s, 56.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0212945 s, 49.2 MB/s , 0.0194648 s, 53.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0201303 s, 52.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137165 s, 76.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0190758 s, 55.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02455 s, 42.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196735 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197942 s, 53.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188927 s, 55.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019242 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0658388 s, 15.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0680009 s, 15.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0569034 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0872204 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00731943 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985006 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00783377 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00734402 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00925424 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00809871 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00887007 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102119 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00939132 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101069 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105342 s, 99.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110709 s, 94.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117541 s, 89.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123859 s, 84.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122605 s, 85.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130624 s, 80.3 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg458-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93203 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4329 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (52s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 19:25:29 (1736987129) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg458-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.648198 s, 25.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.165504 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.168801 s, 6.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.177665 s, 5.9 MB/s , 0.171426 s, 6.1 MB/s , 0.163164 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.168292 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.181054 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.184636 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.368007 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.384 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.384146 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.375794 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.382225 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.385178 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.394334 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.391351 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149017 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149069 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151266 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15109 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152224 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151087 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154665 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152882 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150997 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155016 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152684 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151802 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152838 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153209 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153213 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153352 s, 6.8 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg458-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8282 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9681 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg458-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.638729 s, 26.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.169521 s, 6.2 MB/s , 0.170862 s, 6.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.164878 s, 6.4 MB/s , 0.166193 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178689 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166742 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.182147 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.174793 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.370311 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.376496 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.384442 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.381893 s, 2.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.39299 s, 2.7 MB/s , 0.380353 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.394682 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.389895 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152465 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153482 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152554 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152825 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.153776 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153771 s, 6.8 MB/s 1048576 bytes (1.0 MB) copied, 0.156922 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154822 s, 6.8 MB/s , 0.153923 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154297 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154723 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154973 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155646 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154582 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154865 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157447 s, 6.7 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8289 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9682 s, 5.3 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (136s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 19:27:45 (1736987265) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg458-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.192648 s, 87.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018183 s, 57.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201984 s, 51.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242711 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.036106 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233501 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0384983 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0355848 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0416016 s, 25.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0647302 s, 16.2 MB/s , 0.0660315 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0762681 s, 13.7 MB/s , 0.0765672 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0708747 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.060891 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0821266 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0747949 s, 14.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00751749 s, 139 MB/s 1+0 records in 1+0 records out , 0.0069148 s, 152 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0074739 s, 140 MB/s , 0.00869954 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111823 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00930671 s, 113 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00992393 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00947573 s, 111 MB/s , 0.0104828 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948822 s, 111 MB/s 1048576 bytes (1.0 MB) copied, 0.0117455 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984361 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120072 s, 87.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012113 s, 86.6 MB/s , 0.0123861 s, 84.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144273 s, 72.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg458-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92799 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73039 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.58@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.58@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.186006 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111443 s, 94.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158418 s, 66.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241698 s, 43.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0169524 s, 61.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182667 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0230946 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123533 s, 84.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194494 s, 53.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0741566 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0736487 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.17417 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.161196 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.27578 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.265176 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.362898 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.363583 s, 2.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00994091 s, 105 MB/s 1048576 bytes (1.0 MB) copied, 0.0194106 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101465 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100449 s, 104 MB/s 1048576 bytes (1.0 MB) copied, 0.010411 s, 101 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111654 s, 93.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103923 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0213522 s, 49.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0100962 s, 104 MB/s 1048576 bytes (1.0 MB) copied, 0.0224059 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127867 s, 82.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116043 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119645 s, 87.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126091 s, 83.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123782 s, 84.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139141 s, 75.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.90544 s, 21.4 MB/s Write runtime is 5 s, speed is 20.000000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72898 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.58@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg458-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.194965 s, 86.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179268 s, 58.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279288 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166156 s, 63.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0244538 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217701 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016058 s, 65.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214875 s, 48.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153523 s, 68.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0648165 s, 16.2 MB/s , 0.0657679 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0765103 s, 13.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0647415 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0605054 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0601599 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.082557 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0600986 s, 17.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00869891 s, 121 MB/s , 0.00867441 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972638 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00889697 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119473 s, 87.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00969951 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00896457 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00963232 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010936 s, 95.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110973 s, 94.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116636 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011915 s, 88.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011448 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125381 s, 83.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126922 s, 82.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109751 s, 95.5 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94504 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.87921 s, 21.5 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg458-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.34071 s, 49.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110629 s, 94.8 MB/s , 0.0104537 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0204814 s, 51.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138535 s, 75.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265278 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277695 s, 37.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00646419 s, 162 MB/s , 0.00611441 s, 171 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.170377 s, 6.2 MB/s , 0.169181 s, 6.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.362987 s, 2.9 MB/s , 0.358752 s, 2.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.373158 s, 2.8 MB/s , 0.369226 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.429503 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.407441 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00668805 s, 157 MB/s 1048576 bytes (1.0 MB) copied, 0.00582794 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0091014 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0085117 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00763508 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117839 s, 89.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00456769 s, 230 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107182 s, 97.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00943569 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968842 s, 108 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0101759 s, 103 MB/s , 0.0101091 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110021 s, 95.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108486 s, 96.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105485 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119198 s, 88.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg458-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.76246 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72832 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.9118 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72962 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg458-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.641015 s, 26.2 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169481 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.168579 s, 6.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.162373 s, 6.5 MB/s 1+0 records in 1+0 records out , 0.170686 s, 6.1 MB/s 1048576 bytes (1.0 MB) copied, 0.162705 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.170058 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.185545 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178333 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.368814 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.372208 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.384601 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.379916 s, 2.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.385964 s, 2.7 MB/s , 0.388311 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.388749 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.383989 s, 2.7 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00581921 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00717335 s, 146 MB/s 1048576 bytes (1.0 MB) copied, 0.00747474 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00653587 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00947586 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00891578 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.009331 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114807 s, 91.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00865961 s, 121 MB/s 1048576 bytes (1.0 MB) copied, 0.00888109 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00884319 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00889209 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00912801 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00911411 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00987799 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110166 s, 95.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.828 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4303 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg458-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.192264 s, 87.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00757621 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109224 s, 96.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216672 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116194 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0170923 s, 61.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0205963 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142038 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163578 s, 64.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.063868 s, 16.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0711444 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0597059 s, 17.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0651701 s, 16.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0868635 s, 12.1 MB/s , 0.077518 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.081241 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0935272 s, 11.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00811504 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00946299 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102885 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00888732 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00995283 s, 105 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0103153 s, 102 MB/s , 0.0128426 s, 81.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00976138 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100341 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113256 s, 92.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105631 s, 99.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011778 s, 89.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114363 s, 91.7 MB/s , 0.0117404 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115214 s, 91.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137571 s, 76.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93107 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72977 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg458-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.342547 s, 49.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110208 s, 95.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00798069 s, 131 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00691371 s, 152 MB/s 1048576 bytes (1.0 MB) copied, 0.0167761 s, 62.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0195365 s, 53.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0096711 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00928343 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00731633 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.173328 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.187369 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.17675 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.18417 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.374156 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.37291 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.368152 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.378106 s, 2.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0097386 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0091773 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105512 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00883121 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00957739 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127506 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.009644 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00947231 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107408 s, 97.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011409 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010928 s, 96.0 MB/s , 0.0110973 s, 94.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119659 s, 87.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115556 s, 90.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110768 s, 94.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121487 s, 86.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.78609 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73095 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.76293 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72923 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (355s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 19:33:41 (1736987621) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (3s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 19:33:43 (1736987623) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg458-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.57715 s, 293 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg458-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0168 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 19:34:33 (1736987673) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0911288 s, 184 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126883 s, 82.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166828 s, 62.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147167 s, 71.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281283 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353129 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346327 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0375566 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0334428 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322321 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0415983 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0377152 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304984 s, 34.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0397006 s, 26.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0400093 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353039 s, 29.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88013731d000.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff88013731d000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0406216 s, 25.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00791379 s, 132 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00878663 s, 119 MB/s , 0.0114225 s, 91.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105595 s, 99.3 MB/s 1048576 bytes (1.0 MB) copied, 0.00937724 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011726 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00862981 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00480878 s, 218 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968347 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00979683 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0095904 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107893 s, 97.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123036 s, 85.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109102 s, 96.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113663 s, 92.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106494 s, 98.5 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg458-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92451 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96654 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.143786 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00704502 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165955 s, 63.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146756 s, 71.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027412 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218637 s, 48.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0239841 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238789 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00768709 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0669366 s, 15.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0650696 s, 16.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.154752 s, 6.8 MB/s , 0.149944 s, 7.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.263893 s, 4.0 MB/s , 0.265596 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.355241 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.356079 s, 2.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292546 s, 35.8 MB/s , 0.0269889 s, 38.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0267097 s, 39.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275741 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030322 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282346 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288853 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0293151 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.029257 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.029628 s, 35.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300332 s, 34.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0317277 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300378 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307786 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0314941 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0320967 s, 32.7 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg458-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg458-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92861 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.9695 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (67s) == sanityn test 77o: Changing rank should not panic ====== 19:35:40 (1736987740) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (3s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 19:35:43 (1736987743) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (25s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 19:36:08 (1736987768) mds.MDS.mdt.nrs_policies=tbf oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg458-client: oleg458-server: ssh exited with exit code 36 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg458-client: oleg458-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (9s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 19:36:18 (1736987778) oleg458-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: Resource temporarily unavailable pdsh@oleg458-client: oleg458-server: ssh exited with exit code 11 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (2s) == sanityn test 79: xattr: intent error ================== 19:36:20 (1736987780) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505431 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-15 19:36:20.000000000 -0500 Modify: 2025-01-15 19:36:20.000000000 -0500 Change: 2025-01-15 19:36:21.000000000 -0500 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 19:36:32 (1736987792) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 19:36:33 (1736987793) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 19:36:34 (1736987794) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (0s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 19:36:34 (1736987794) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 19:36:35 (1736987795) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 19:36:36 (1736987796) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6873 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6873 PASS 82 (1s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 19:36:37 (1736987797) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 19:36:38 (1736987798) fail_loc=0x8000060b PASS 84 (7s) == sanityn test 90: open/create and unlink striped directory ========================================================== 19:36:45 (1736987805) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (0s) == sanityn test 91: chmod and unlink striped directory === 19:36:45 (1736987805) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 19:36:46 (1736987806) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 19:36:47 (1736987807) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2644 0xa54 0 1 2629 0xa45 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2630 0xa46 0 0 2645 0xa55 0 rc1=0 and rc2=0 PASS 93 (8s) == sanityn test 94: signal vs CP callback race =========== 19:36:55 (1736987815) osc.lustre-OST0000-osc-ffff8800aa144800.idle_timeout=0 osc.lustre-OST0000-osc-ffff88013731d000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800aa144800.idle_timeout=0 osc.lustre-OST0001-osc-ffff88013731d000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0299899 s, 273 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 3.00074 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 4633 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0113221 s, 362 kB/s osc.lustre-OST0000-osc-ffff8800aa144800.idle_timeout=debug osc.lustre-OST0000-osc-ffff88013731d000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aa144800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88013731d000.idle_timeout=debug PASS 94 (9s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 19:37:04 (1736987824) llite.lustre-ffff8800aa144800.fast_read=0 llite.lustre-ffff88013731d000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00139269 s, 8.8 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800aa144800.fast_read=1 llite.lustre-ffff88013731d000.fast_read=1 PASS 95b (7s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 19:37:12 (1736987832) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (0s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 19:37:12 (1736987832) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00140691 s, 2.9 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 19:37:14 (1736987834) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0192628 s, 109 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (1s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 19:37:15 (1736987835) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0159542 s, 131 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 19:37:17 (1736987837) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00126412 s, 9.5 MB/s PASS 100e (1s) == sanityn test 101a: Discard DoM data on unlink ========= 19:37:18 (1736987838) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000510733 s, 8.0 MB/s PASS 101a (2s) == sanityn test 101b: Discard DoM data on rename ========= 19:37:20 (1736987840) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000528393 s, 7.8 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 19:37:22 (1736987842) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000894024 s, 4.6 MB/s PASS 101c (3s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 19:37:25 (1736987845) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1736987846 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1736987848 Finishing test test23 at 1736987849 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 19:37:32 (1736987852) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 19:37:33 (1736987853) fail_loc=0x416 PASS 105 (47s) == sanityn test 106a: Verify the btime via statx() ======= 19:38:20 (1736987900) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 19:38:21 (1736987901) SKIP: sanityn test_106b statx() only test SKIP 106b (0s) == sanityn test 106c: Verify statx attributes mask ======= 19:38:22 (1736987902) SKIP: sanityn test_106c statx() only test SKIP 106c (0s) == sanityn test 107a: Basic grouplock conflict =========== 19:38:22 (1736987902) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0568409 s, 184 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2635 0xa4b 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6873 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6873 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 19:38:26 (1736987906) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.050303 s, 208 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2651 0xa5b 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6873 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) == sanityn test 108a: lseek: parallel updates ============ 19:38:34 (1736987914) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00185789 s, 5.4 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.29688 s, 2.0 MB/s PASS 108a (6s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 19:38:40 (1736987920) 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions oleg458-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 7 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions oleg458-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 8 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions oleg458-server: ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' oleg458-server: insmod: ERROR: could not insert module /home/green/git/lustre-release/lustre/../libcfs/libcfs/libcfs.ko: File exists pdsh@oleg458-client: oleg458-server: ssh exited with exit code 1 Iteration 17 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions oleg458-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 20 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions oleg458-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg458-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 28 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions oleg458-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 33 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions oleg458-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 34 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 192.168.204.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.204.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg458-server' oleg458-server: oleg458-server.virtnet: executing load_modules_local oleg458-server: Loading modules from /home/green/git/lustre-release/lustre oleg458-server: detected 4 online CPUs by sysfs oleg458-server: Force libcfs to create 2 CPU partitions oleg458-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre Starting client: oleg458-client.virtnet: -o user_xattr,flock oleg458-server@tcp:/lustre /mnt/lustre2 PASS 109 (417s) == sanityn test 110: do not grant another lock on resend ========================================================== 19:45:37 (1736988337) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-15 19:45:37.000000000 -0500 Modify: 2025-01-15 19:45:37.000000000 -0500 Change: 2025-01-15 19:45:37.000000000 -0500 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg458-client.virtnet oleg458-client.virtnet PASS 110 (36s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 19:46:13 (1736988373) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (0s) == sanityn test 112: update max-inherit in default LMV === 19:46:13 (1736988373) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 19:46:14 (1736988374) lustre-OST0000-osc-ffff88012b14f000 active. lustre-OST0001-osc-ffff88012b14f000 active. lustre-OST0000-osc-ffff88012b14f000 active. lustre-OST0001-osc-ffff88012b14f000 active. MGC192.168.204.158@tcp active. PASS 113 (2s) cleanup: ====================================================== == sanityn test complete, duration 4132 sec ============== 19:46:16 (1736988376) Stopping clients: oleg458-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg458-client.virtnet /mnt/lustre2 opts: