-----============= acceptance-small: sanityn ============----- Tue Jan 7 10:22:11 EST 2025 excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Started clients oleg307-client.virtnet: 192.168.203.107@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt) oleg307-client.virtnet: executing check_config_client /mnt/lustre oleg307-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg307-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800aa5a6800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b5944800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aa5a6800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b5944800.idle_timeout=debug disable quota as required oleg307-server: oleg307-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135971 s, 77.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6878] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 10:22:22 (1736263342) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (3s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 10:22:25 (1736263345) -rw-r--r-- 1 root root 0 Jan 7 10:22 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (2s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 10:22:27 (1736263347) -rw-r--r-- 1 root root 0 Jan 7 10:22 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (2s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 10:22:30 (1736263350) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (3s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 10:22:32 (1736263352) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (2s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 10:22:34 (1736263354) total 1 -rwxrwxrwx 1 root root 0 Jan 7 10:22 f2a -rwxrwxrwx 1 root root 0 Jan 7 10:22 f2b total 1 -rwxrwxrwx 1 root root 0 Jan 7 10:22 f2a -rwxrwxrwx 1 root root 0 Jan 7 10:22 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.0101037 s, 50.7 kB/s PASS 2e (3s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 10:22:37 (1736263357) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 10:22:38 (1736263358) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.174137 s, 12.0 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (3s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 10:22:41 (1736263361) PASS 3 (2s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 10:22:43 (1736263363) PASS 4 (3s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 10:22:47 (1736263367) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (2s) == sanityn test 6: remove of open file on other node ============================================================================ 10:22:49 (1736263369) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (3s) == sanityn test 7: remove of open directory on other node ======================================================================= 10:22:52 (1736263372) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (3s) == sanityn test 8: remove of open special file on other node ==================================================================== 10:22:55 (1736263375) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (2s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 10:22:58 (1736263378) PASS 9a (3s) == sanityn test 9b: append to striped sparse file ======== 10:23:00 (1736263380) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176317 s, 59.5 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0379061 s, 0.1 kB/s Data read (expecting 'foo'): foo PASS 9b (3s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 10:23:03 (1736263383) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00339954 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00225599 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00376209 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00210095 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00306569 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00403326 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0047806 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.003178 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00547969 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.004345 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00294525 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00608256 s, 0.2 kB/s PASS 10a (3s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 10:23:06 (1736263386) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00774117 s, 397 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.031999 s, 128 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000845928 s, 3.6 MB/s PASS 10b (2s) == sanityn test 11: execution of file opened for write should return error ============================================================== 10:23:08 (1736263388) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6878 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (3s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 10:23:11 (1736263391) warning: '-runtime' deprecated, use '-t runtime' instead - link 1017 (time 1736263402.26 total 10.00 last 101.69) - link 1853 (time 1736263412.26 total 20.00 last 83.60) - link 2638 (time 1736263422.26 total 30.01 last 78.44) - link 3508 (time 1736263432.27 total 40.01 last 87.00) - link 4384 (time 1736263442.28 total 50.02 last 87.49) - link 5416 (time 1736263452.29 total 60.03 last 103.11) - link 6504 (time 1736263462.29 total 70.03 last 108.77) - link 7614 (time 1736263472.29 total 80.03 last 111.00) - link 8805 (time 1736263482.29 total 90.03 last 119.08) - link 10000 (time 1736263490.64 total 98.39 last 143.08) total: 10305 link in 100.00 seconds: 103.05 ops/second using seed 2543878077 running for 100 seconds - stat 10000 (time 1736263397 ; total 4 ; last 4) - stat 20000 (time 1736263401 ; total 8 ; last 4) - stat 30000 (time 1736263406 ; total 13 ; last 5) - stat 40000 (time 1736263410 ; total 17 ; last 4) - stat 50000 (time 1736263415 ; total 22 ; last 5) - stat 60000 (time 1736263420 ; total 27 ; last 5) - stat 70000 (time 1736263425 ; total 32 ; last 5) - stat 80000 (time 1736263429 ; total 36 ; last 4) - stat 90000 (time 1736263434 ; total 41 ; last 5) - stat 100000 (time 1736263438 ; total 45 ; last 4) - stat 110000 (time 1736263443 ; total 50 ; last 5) - stat 117991 (time 1736263454 ; total 61 ; last 11) - stat 120000 (time 1736263456 ; total 63 ; last 2) - stat 128295 (time 1736263467 ; total 74 ; last 11) - stat 130000 (time 1736263469 ; total 76 ; last 2) - stat 138327 (time 1736263480 ; total 87 ; last 11) - stat 140000 (time 1736263481 ; total 88 ; last 1) - stat 149642 (time 1736263492 ; total 99 ; last 11) - stat 150000 (time 1736263492 ; total 99 ; last 0) total: 151493 stats in 100 seconds: 1514.930054 stats/second - unlinked 0 (time 1736263443 ; total 0 ; last 0) - unlinked 10000 (time 1736263520 ; total 77 ; last 77) unlink(/mnt/lustre2/lockdir/lockfile10305) error: No such file or directory total: 10304 unlinks in 78 seconds: 132.102570 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (20490) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (20492) - No such process PASS 12 (132s) == sanityn test 13: test directory page revocation ======= 10:25:23 (1736263523) f13 f13 PASS 13 (3s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 10:25:26 (1736263526) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6878 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4260: 21862 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 10:25:28 (1736263528) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4260: 22490 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (2s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 10:25:30 (1736263530) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 23118 Terminated $DIR1/$tdir/sleep 60 PASS 14b (2s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 10:25:32 (1736263532) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 23747 Terminated $DIR1/$tdir/sleep 60 PASS 14c (2s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 10:25:34 (1736263534) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 24376 Terminated $DIR1/$tdir/sleep 60 PASS 14d (3s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 10:25:37 (1736263537) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7523328 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7523328kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (12s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 10:25:50 (1736263550) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.2043 s, 10.3 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.121395 s, 17.3 MB/s Chance of close/open is 1 in 50 Seed set to 3551 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x4772b4 000002[0] 1736263551.637113 trunc from 00000000 to 0x4772b3 (0x4772b4 bytes) 000004[1] 1736263551.688967 mapread 0x23feba thru 0x241b11 (0x1c58 bytes) 000011[1] 1736263551.740283 mapwrite 0x4db635 thru 0x4e92e5 (0xdcb1 bytes) 000015[1] 1736263551.880482 write 0x4e6ef7 thru 0x4f1a0e (0xab18 bytes) 000018[0] 1736263551.899803 read 0x1a23dd thru 0x1a9e7d (0x7aa1 bytes) 000019[0] 1736263551.914653 write 0x7b6c7c thru 0x7b8abb (0x1e40 bytes) 000020[1] 1736263551.929320 trunc from 0x7b8abc to 0x37dfb8 (0x43ab03 bytes) 000021[0] 1736263551.954323 read 0x0e069e thru 0x0e1dc1 (0x1724 bytes) 000023[1] 1736263551.981608 mapwrite 0x43c903 thru 0x43f52f (0x2c2d bytes) 000026[1] 1736263552.044357 mapwrite 0x17d4d1 thru 0x17d821 (0x351 bytes) truncating to largest ever: 0x671444 000027[0] 1736263552.094139 trunc from 0x43f530 to 0x671443 (0x231f14 bytes) 000028[1] 1736263552.133426 read 0x4bc7e9 thru 0x4c4a30 (0x8248 bytes) 000030[0] 1736263552.149875 read 0x2b67b3 thru 0x2b8ba8 (0x23f6 bytes) 000032[1] 1736263552.191224 trunc from 0x671444 to 0x229e1a (0x447629 bytes) 000033[1] 1736263552.224617 trunc from 0x229e1b to 0x090fc7 (0x198e53 bytes) 000034[0] 1736263552.245428 read 0x04eb47 thru 0x04fff3 (0x14ad bytes) 000038[0] 1736263552.294487 mapread 0x0029cf thru 0x01066a (0xdc9c bytes) 000041[0] 1736263552.356441 read 0x001345 thru 0x004986 (0x3642 bytes) 000042[1] 1736263552.381519 trunc from 0x090fc8 to 0x4b62f0 (0x425329 bytes) 000047[0] 1736263552.435582 write 0x8e8c70 thru 0x8ec930 (0x3cc1 bytes) 000048[0] 1736263552.457241 read 0x511e51 thru 0x516f8d (0x513d bytes) 000049[0] 1736263552.491978 mapwrite 0x293195 thru 0x294f36 (0x1da2 bytes) 000051[1] 1736263552.545280 read 0x75e21e thru 0x76901a (0xadfd bytes) 000053[0] 1736263552.600559 read 0x54c6a8 thru 0x54f512 (0x2e6b bytes) 000055[1] 1736263552.622615 write 0x4ee6ef thru 0x4fb5e0 (0xcef2 bytes) 000059[1] 1736263552.696268 mapread 0x72f805 thru 0x73c1d9 (0xc9d5 bytes) 000061[0] 1736263552.765743 write 0x008301 thru 0x0151a2 (0xcea2 bytes) 000062[1] 1736263552.781192 mapread 0x430707 thru 0x43bf81 (0xb87b bytes) truncating to largest ever: 0x676ae6 000063[0] 1736263552.828068 trunc from 0x8ec931 to 0x676ae5 (0x275e4b bytes) 000066[1] 1736263552.886733 write 0x4dbba9 thru 0x4e74cd (0xb925 bytes) 000068[0] 1736263552.910245 trunc from 0x676ae6 to 0x0c0ea8 (0x5b5c3d bytes) truncating to largest ever: 0x8e8c11 000071[0] 1736263552.981557 trunc from 0x0c0ea9 to 0x8e8c10 (0x827d68 bytes) 000074[1] 1736263552.999366 read 0x1486b4 thru 0x14d64a (0x4f97 bytes) 000078[0] 1736263553.059706 read 0x421ff6 thru 0x42ee0c (0xce17 bytes) 000079[1] 1736263553.089675 mapread 0x64961a thru 0x653311 (0x9cf8 bytes) 000082[1] 1736263553.120523 read 0x718e74 thru 0x7260a8 (0xd235 bytes) 000083[1] 1736263553.128337 write 0x3e7275 thru 0x3ef348 (0x80d4 bytes) 000084[0] 1736263553.155899 mapread 0x242146 thru 0x249388 (0x7243 bytes) 000085[0] 1736263553.199456 mapwrite 0x9a0f3c thru 0x9ad73c (0xc801 bytes) 000086[1] 1736263553.307247 read 0x233c82 thru 0x2406a7 (0xca26 bytes) 000087[0] 1736263553.328975 mapwrite 0x429b48 thru 0x42cdc5 (0x327e bytes) 000090[0] 1736263553.401832 mapwrite 0x97dbac thru 0x9810d6 (0x352b bytes) 000091[0] 1736263553.430101 mapwrite 0x91bb4c thru 0x91efa5 (0x345a bytes) 000092[0] 1736263553.470101 write 0x0144eb thru 0x01e0cf (0x9be5 bytes) 000101[0] 1736263553.512970 mapread 0x36232b thru 0x363789 (0x145f bytes) 000103[1] 1736263553.527576 trunc from 0x9ad73d to 0x3ace42 (0x6008fa bytes) 000104[1] 1736263553.547055 mapread 0x2dea0e thru 0x2ed605 (0xebf8 bytes) 000105[0] 1736263553.603260 mapwrite 0x362d7a thru 0x369764 (0x69eb bytes) 000106[0] 1736263553.676869 write 0x730e34 thru 0x73a6a6 (0x9873 bytes) 000108[0] 1736263553.698215 mapread 0x2144ea thru 0x219206 (0x4d1d bytes) 000111[0] 1736263553.743644 mapread 0x624d4e thru 0x62a926 (0x5bd9 bytes) 000113[0] 1736263553.762876 trunc from 0x73a6a7 to 0x6de24d (0x5c459 bytes) 000116[0] 1736263553.784689 mapread 0x0bc34d thru 0x0be720 (0x23d4 bytes) 000118[0] 1736263553.803896 mapread 0x18d680 thru 0x197cec (0xa66d bytes) 000119[1] 1736263553.836392 read 0x65d9ab thru 0x668987 (0xafdd bytes) 000120[1] 1736263553.855296 mapwrite 0x4eb74e thru 0x4f58e8 (0xa19b bytes) 000121[1] 1736263553.987800 trunc from 0x6de24e to 0x849fb7 (0x16bd6a bytes) 000123[1] 1736263554.023338 read 0x5bb043 thru 0x5ca9be (0xf97c bytes) 000128[0] 1736263554.080740 mapread 0x5b1c00 thru 0x5b2e95 (0x1296 bytes) 000129[0] 1736263554.095800 trunc from 0x849fb8 to 0x391dd2 (0x4b81e5 bytes) 000130[0] 1736263554.134476 write 0x70c900 thru 0x7155d9 (0x8cda bytes) 000131[1] 1736263554.152478 write 0x5fa0fd thru 0x5fe0f8 (0x3ffc bytes) 000133[0] 1736263554.182550 trunc from 0x7155da to 0x2bf459 (0x456180 bytes) 000134[0] 1736263554.211360 mapread 0x02ebf7 thru 0x036337 (0x7741 bytes) 000136[1] 1736263554.242106 write 0x23f3bc thru 0x240583 (0x11c8 bytes) 000140[0] 1736263554.285702 mapwrite 0x975bfb thru 0x97a0c8 (0x44ce bytes) 000141[0] 1736263554.412734 read 0x46c74b thru 0x46e428 (0x1cde bytes) 000142[1] 1736263554.431318 mapread 0x6c39ac thru 0x6cdbbc (0xa211 bytes) 000143[0] 1736263554.503311 trunc from 0x97a0c9 to 0x32db41 (0x64c587 bytes) 000144[0] 1736263554.527768 write 0x8d4bd7 thru 0x8e275e (0xdb88 bytes) 000147[1] 1736263554.558686 mapread 0x88709c thru 0x88a456 (0x33bb bytes) 000148[1] 1736263554.586401 mapread 0x12f034 thru 0x131589 (0x2556 bytes) 000149[0] 1736263554.607223 mapread 0x891645 thru 0x8945b7 (0x2f73 bytes) 000150[0] 1736263554.619488 read 0x6c026e thru 0x6cc27c (0xc00f bytes) 000155[0] 1736263554.635708 write 0x182379 thru 0x188591 (0x6219 bytes) 000156[1] 1736263554.649955 trunc from 0x8e275f to 0x369e88 (0x5788d6 bytes) 000157[1] 1736263554.675161 read 0x078f8a thru 0x083734 (0xa7ab bytes) 000158[0] 1736263554.719629 trunc from 0x369e89 to 0x249d73 (0x120115 bytes) 000159[1] 1736263554.766749 mapwrite 0x29f244 thru 0x2ad697 (0xe454 bytes) 000163[0] 1736263554.890868 read 0x01e1f4 thru 0x02d49b (0xf2a8 bytes) 000165[1] 1736263554.928142 read 0x022895 thru 0x0282cc (0x5a38 bytes) 000168[1] 1736263554.959985 mapread 0x0e8827 thru 0x0ece27 (0x4601 bytes) 000169[0] 1736263554.961013 read 0x1936c5 thru 0x1992c4 (0x5c00 bytes) 000170[1] 1736263554.979565 mapwrite 0x8c775d thru 0x8cd4f3 (0x5d97 bytes) 000171[1] 1736263555.113338 write 0x05d8d6 thru 0x06262c (0x4d57 bytes) 000173[0] 1736263555.128429 mapwrite 0x2b8aec thru 0x2c2eff (0xa414 bytes) 000174[0] 1736263555.201355 write 0x24a2c1 thru 0x2504ea (0x622a bytes) 000175[0] 1736263555.211852 write 0x7afca6 thru 0x7bd928 (0xdc83 bytes) 000177[1] 1736263555.244049 read 0x5dfb5f thru 0x5ee6da (0xeb7c bytes) 000181[1] 1736263555.284919 mapwrite 0x8e8bd2 thru 0x8e9d63 (0x1192 bytes) 000182[0] 1736263555.396012 trunc from 0x8e9d64 to 0x03a2a0 (0x8afac3 bytes) 000185[0] 1736263555.435661 read 0x0073a7 thru 0x014e6d (0xdac7 bytes) 000186[1] 1736263555.458978 read 0x021be6 thru 0x02cb5e (0xaf79 bytes) truncating to largest ever: 0x9ed017 000187[1] 1736263555.476857 trunc from 0x03a2a1 to 0x9ed016 (0x9b2d76 bytes) 000189[0] 1736263555.514976 read 0x55e1a3 thru 0x564143 (0x5fa1 bytes) 000191[1] 1736263555.558547 trunc from 0x9ed017 to 0x6f8b2d (0x2f44e9 bytes) 000192[1] 1736263555.582739 trunc from 0x6f8b2e to 0x2386e2 (0x4c044b bytes) 000193[1] 1736263555.591647 trunc from 0x2386e3 to 0x7aa014 (0x571932 bytes) 000198[1] 1736263555.639096 write 0x129c2c thru 0x12c4eb (0x28c0 bytes) 000200[1] 1736263555.710924 mapwrite 0x4aa769 thru 0x4af65b (0x4ef3 bytes) 000202[0] 1736263555.760444 read 0x4b98ae thru 0x4c4080 (0xa7d3 bytes) 000203[0] 1736263555.785670 read 0x3dee12 thru 0x3eed2d (0xff1c bytes) 000206[1] 1736263555.842559 read 0x51cf6a thru 0x521821 (0x48b8 bytes) 000208[1] 1736263555.869548 read 0x46f2f8 thru 0x472ff1 (0x3cfa bytes) 000209[1] 1736263555.887426 mapread 0x789a7c thru 0x79690b (0xce90 bytes) 000210[1] 1736263555.888612 mapwrite 0x073b48 thru 0x07b74d (0x7c06 bytes) 000211[1] 1736263555.998554 mapread 0x393157 thru 0x3993c9 (0x6273 bytes) 000212[0] 1736263556.019986 trunc from 0x7aa015 to 0x67e387 (0x12bc8d bytes) 000213[1] 1736263556.051949 trunc from 0x67e388 to 0x107441 (0x576f46 bytes) 000214[1] 1736263556.068796 trunc from 0x107442 to 0x29568e (0x18e24d bytes) 000215[0] 1736263556.079744 trunc from 0x29568f to 0x04d49d (0x2481f1 bytes) 000216[0] 1736263556.124296 mapwrite 0x05618b thru 0x063d3e (0xdbb4 bytes) 000219[0] 1736263556.246389 mapread 0x01d53c thru 0x027a4c (0xa511 bytes) 000220[1] 1736263556.279488 read 0x05059e thru 0x052ac6 (0x2529 bytes) 000222[0] 1736263556.309436 mapread 0x0267fa thru 0x033fe7 (0xd7ee bytes) 000224[1] 1736263556.351690 read 0x054205 thru 0x058f8a (0x4d86 bytes) 000226[1] 1736263556.352950 write 0x0fa2a3 thru 0x0fddd5 (0x3b33 bytes) 000229[0] 1736263556.397969 write 0x965c4b thru 0x974b79 (0xef2f bytes) 000231[0] 1736263556.421940 read 0x772c37 thru 0x77a5eb (0x79b5 bytes) 000233[0] 1736263556.454969 mapwrite 0x37778a thru 0x383d0d (0xc584 bytes) 000234[1] 1736263556.528745 trunc from 0x974b7a to 0x4560cb (0x51eaae bytes) 000235[0] 1736263556.561233 mapwrite 0x055ec3 thru 0x0626da (0xc818 bytes) 000236[1] 1736263556.669824 mapread 0x40904a thru 0x4126ad (0x9664 bytes) 000238[1] 1736263556.726088 write 0x0d5626 thru 0x0e282d (0xd208 bytes) 000239[0] 1736263556.736686 write 0x5cb5c7 thru 0x5d6285 (0xacbf bytes) 000241[1] 1736263556.768665 mapwrite 0x1ea76a thru 0x1ef41d (0x4cb4 bytes) 000243[1] 1736263556.839135 write 0x4a39ad thru 0x4b1a87 (0xe0db bytes) 000244[0] 1736263556.843823 trunc from 0x5d6286 to 0x03a12d (0x59c158 bytes) 000245[1] 1736263556.874571 mapread 0x02aecf thru 0x03493a (0x9a6c bytes) 000247[1] 1736263556.931214 read 0x015434 thru 0x01cd29 (0x78f6 bytes) 000249[1] 1736263556.961301 mapread 0x00269e thru 0x0096a7 (0x700a bytes) 000252[0] 1736263557.007746 trunc from 0x03a12e to 0x20bfd6 (0x1d1ea9 bytes) 000253[0] 1736263557.032790 read 0x1482d8 thru 0x1516d7 (0x9400 bytes) 000256[0] 1736263557.063516 read 0x056105 thru 0x05f08d (0x8f89 bytes) 000258[1] 1736263557.107806 write 0x5e5e02 thru 0x5ebd88 (0x5f87 bytes) 000259[1] 1736263557.122066 read 0x21235f thru 0x21704b (0x4ced bytes) 000260[1] 1736263557.148687 mapread 0x0a647c thru 0x0b30c2 (0xcc47 bytes) 000261[0] 1736263557.180191 mapread 0x543919 thru 0x54c329 (0x8a11 bytes) 000262[1] 1736263557.227738 read 0x4ac5a0 thru 0x4b0e87 (0x48e8 bytes) 000264[1] 1736263557.228791 mapwrite 0x6b4856 thru 0x6c1c1d (0xd3c8 bytes) 000266[0] 1736263557.358739 mapwrite 0x07a48c thru 0x0898f0 (0xf465 bytes) 000267[1] 1736263557.447865 write 0x57f1c2 thru 0x589c0d (0xaa4c bytes) 000269[0] 1736263557.465603 mapread 0x3aaf43 thru 0x3ac728 (0x17e6 bytes) 000270[0] 1736263557.487289 trunc from 0x6c1c1e to 0x7f4959 (0x132d3c bytes) 000273[1] 1736263557.518792 read 0x41fb42 thru 0x421237 (0x16f6 bytes) 000277[0] 1736263557.547150 trunc from 0x7f495a to 0x9b474e (0x1bfdf5 bytes) 000278[1] 1736263557.562577 mapwrite 0x461a6f thru 0x46ae38 (0x93ca bytes) 000280[1] 1736263557.658965 trunc from 0x9b474f to 0x7baedb (0x1f9873 bytes) 000282[1] 1736263557.699447 trunc from 0x7baedc to 0x670336 (0x14aba5 bytes) 000283[1] 1736263557.708778 trunc from 0x670337 to 0x946c63 (0x2d692d bytes) 000284[1] 1736263557.716581 trunc from 0x946c64 to 0x66bb5a (0x2db109 bytes) 000285[0] 1736263557.727760 mapwrite 0x85d8bd thru 0x85f47e (0x1bc2 bytes) 000288[0] 1736263557.783413 write 0x161b04 thru 0x16c3b9 (0xa8b6 bytes) 000290[0] 1736263557.789870 mapwrite 0x921f6d thru 0x924191 (0x2225 bytes) 000292[0] 1736263557.841244 mapwrite 0x15956a thru 0x15a4c6 (0xf5d bytes) 000294[1] 1736263557.884823 read 0x491c98 thru 0x49f24c (0xd5b5 bytes) 000295[1] 1736263557.896735 read 0x300e41 thru 0x3032f9 (0x24b9 bytes) 000299[0] 1736263557.949154 mapread 0x651c5d thru 0x653975 (0x1d19 bytes) 000300[0] 1736263557.959825 mapwrite 0x523c09 thru 0x526138 (0x2530 bytes) 000301[0] 1736263558.023987 read 0x59e12e thru 0x5a4c5a (0x6b2d bytes) 000303[1] 1736263558.057655 read 0x28b83d thru 0x293037 (0x77fb bytes) 000304[0] 1736263558.070423 mapwrite 0x4fba5c thru 0x50ab63 (0xf108 bytes) 000306[1] 1736263558.171997 mapwrite 0x9ce6df thru 0x9cee8b (0x7ad bytes) 000307[1] 1736263558.290831 write 0x62d01f thru 0x63a99f (0xd981 bytes) 000308[1] 1736263558.293882 read 0x7bee95 thru 0x7c6220 (0x738c bytes) 000310[1] 1736263558.312100 mapwrite 0x3cb88e thru 0x3d91c5 (0xd938 bytes) 000311[0] 1736263558.355391 trunc from 0x9cee8c to 0x4acebe (0x521fcd bytes) 000312[0] 1736263558.391532 mapread 0x23d431 thru 0x23d695 (0x265 bytes) 000313[1] 1736263558.416602 mapread 0x07ec55 thru 0x088a9c (0x9e48 bytes) 000314[0] 1736263558.458528 trunc from 0x4acebf to 0x083c39 (0x429285 bytes) 000317[0] 1736263558.484398 mapread 0x00a798 thru 0x011894 (0x70fd bytes) 000318[0] 1736263558.500133 mapread 0x001ac4 thru 0x0089cf (0x6f0c bytes) 000320[1] 1736263558.527644 mapwrite 0x88480d thru 0x886b71 (0x2365 bytes) 000321[0] 1736263558.591580 trunc from 0x886b72 to 0x2c2533 (0x5c463e bytes) 000323[0] 1736263558.623304 read 0x1fb4d4 thru 0x1fbda0 (0x8cd bytes) 000325[1] 1736263558.649684 read 0x0e4488 thru 0x0e887c (0x43f5 bytes) 000326[0] 1736263558.659721 write 0x2aead8 thru 0x2b81bc (0x96e5 bytes) 000327[0] 1736263558.670340 write 0x620ce7 thru 0x62faab (0xedc5 bytes) 000328[1] 1736263558.671965 read 0x15c87c thru 0x167b78 (0xb2fd bytes) 000329[0] 1736263558.683376 mapread 0x5382d7 thru 0x53b7e2 (0x350c bytes) 000331[0] 1736263558.695781 write 0x4f89c4 thru 0x506cc2 (0xe2ff bytes) 000335[1] 1736263558.736762 write 0x8aa116 thru 0x8b6e9b (0xcd86 bytes) 000336[1] 1736263558.757799 mapread 0x39c1d9 thru 0x3a4e8f (0x8cb7 bytes) 000337[0] 1736263558.795488 mapwrite 0x5b8123 thru 0x5c27a7 (0xa685 bytes) 000338[0] 1736263559.004914 write 0x879468 thru 0x8865e9 (0xd182 bytes) 000339[0] 1736263559.031045 mapread 0x3118ff thru 0x31339e (0x1aa0 bytes) 000341[0] 1736263559.037432 mapread 0x2f417f thru 0x2fd467 (0x92e9 bytes) 000342[1] 1736263559.061063 trunc from 0x8b6e9c to 0x2e66d9 (0x5d07c2 bytes) 000347[1] 1736263559.090058 read 0x1d9a71 thru 0x1dc185 (0x2715 bytes) 000351[0] 1736263559.117654 read 0x15f6be thru 0x166f2a (0x786d bytes) 000352[0] 1736263559.133830 mapwrite 0x261624 thru 0x26b7a3 (0xa180 bytes) 000353[0] 1736263559.177920 write 0x67ee1e thru 0x68c514 (0xd6f7 bytes) 000355[1] 1736263559.179507 mapwrite 0x83704a thru 0x83abb8 (0x3b6f bytes) 000356[0] 1736263559.276516 trunc from 0x83abb9 to 0x254e8e (0x5e5d2a bytes) 000357[0] 1736263559.314499 write 0x3487a0 thru 0x3545b2 (0xbe13 bytes) 000359[0] 1736263559.318492 mapwrite 0x79c334 thru 0x7a4b55 (0x8822 bytes) 000361[0] 1736263559.412359 trunc from 0x7a4b56 to 0x1f326c (0x5b18e9 bytes) 000362[0] 1736263559.425538 write 0x8e1c1e thru 0x8ebc9c (0xa07f bytes) 000366[0] 1736263559.459609 mapwrite 0x4b0027 thru 0x4b1ba6 (0x1b80 bytes) 000367[0] 1736263559.499170 write 0x0e3553 thru 0x0e356b (0x19 bytes) 000368[0] 1736263559.501651 write 0x614e0f thru 0x61db43 (0x8d35 bytes) 000369[1] 1736263559.515611 write 0x50f894 thru 0x516dec (0x7559 bytes) 000370[0] 1736263559.524938 write 0x553205 thru 0x55e4e9 (0xb2e5 bytes) 000372[1] 1736263559.542068 mapwrite 0x4c418e thru 0x4d1c6d (0xdae0 bytes) 000373[1] 1736263559.633339 trunc from 0x8ebc9d to 0x2b2d0f (0x638f8d bytes) 000375[1] 1736263559.658793 trunc from 0x2b2d10 to 0x40dff1 (0x15b2e2 bytes) 000378[1] 1736263559.672232 mapread 0x26d63c thru 0x27d5da (0xff9f bytes) 000379[0] 1736263559.717200 write 0x55fb93 thru 0x56ed70 (0xf1de bytes) 000382[1] 1736263559.749535 trunc from 0x56ed71 to 0x015e9b (0x558ed5 bytes) 000383[1] 1736263559.772726 trunc from 0x015e9c to 0x0b913a (0xa329f bytes) truncating to largest ever: 0x9fabdb 000385[0] 1736263559.784193 trunc from 0x0b913b to 0x9fabda (0x941aa0 bytes) 000386[0] 1736263559.803476 trunc from 0x9fabdb to 0x49c334 (0x55e8a6 bytes) 000387[1] 1736263559.811621 mapwrite 0x21941d thru 0x221b5f (0x8743 bytes) 000388[0] 1736263559.892570 write 0x9d3fc8 thru 0x9df6e2 (0xb71b bytes) 000391[1] 1736263559.909858 mapread 0x89490c thru 0x895dcb (0x14c0 bytes) 000392[1] 1736263559.924935 write 0x4c4695 thru 0x4c8e11 (0x477d bytes) 000393[0] 1736263559.930427 trunc from 0x9df6e3 to 0x1c069f (0x81f043 bytes) 000394[1] 1736263559.944166 read 0x0095ec thru 0x00aa8c (0x14a1 bytes) 000395[0] 1736263559.963573 mapread 0x177a1b thru 0x17e138 (0x671e bytes) 000396[0] 1736263560.009012 mapread 0x10031f thru 0x10a9b4 (0xa696 bytes) 000398[1] 1736263560.038621 mapwrite 0x9a63a5 thru 0x9abec5 (0x5b21 bytes) 000399[1] 1736263560.138352 mapread 0x81933c thru 0x81c91c (0x35e1 bytes) 000400[0] 1736263560.143218 mapwrite 0x8dd103 thru 0x8e213a (0x5038 bytes) 000401[0] 1736263560.216775 mapread 0x93575c thru 0x93e8d3 (0x9178 bytes) 000406[1] 1736263560.274349 read 0x28bbe9 thru 0x290e78 (0x5290 bytes) 000408[1] 1736263560.312557 read 0x3e64d5 thru 0x3eb4fa (0x5026 bytes) 000409[1] 1736263560.317014 mapwrite 0x503602 thru 0x5076ac (0x40ab bytes) 000410[0] 1736263560.369761 mapread 0x3d3dbb thru 0x3db7fc (0x7a42 bytes) 000412[1] 1736263560.415521 mapwrite 0x1a05bf thru 0x1a0cad (0x6ef bytes) 000414[0] 1736263560.445738 trunc from 0x9abec6 to 0x0590de (0x952de7 bytes) 000416[1] 1736263560.466496 mapread 0x00e359 thru 0x013e9d (0x5b45 bytes) 000417[1] 1736263560.503331 trunc from 0x0590df to 0x4c9ec1 (0x470de3 bytes) 000418[1] 1736263560.520738 mapread 0x3a6ed6 thru 0x3a91a6 (0x22d1 bytes) 000419[0] 1736263560.540387 read 0x4a9fcc thru 0x4b0af4 (0x6b29 bytes) 000422[1] 1736263560.574760 read 0x3c9f63 thru 0x3cf4ac (0x554a bytes) 000423[1] 1736263560.591972 write 0x36626f thru 0x3706d2 (0xa464 bytes) 000424[1] 1736263560.602363 trunc from 0x4c9ec2 to 0x210787 (0x2b973a bytes) 000425[1] 1736263560.623174 trunc from 0x210788 to 0x99f1c1 (0x78ea3a bytes) 000427[0] 1736263560.637364 read 0x26427d thru 0x2667f9 (0x257d bytes) 000429[1] 1736263560.653974 write 0x2eafb5 thru 0x2efc65 (0x4cb1 bytes) 000430[0] 1736263560.698953 mapwrite 0x5294f6 thru 0x538830 (0xf33b bytes) 000432[0] 1736263560.830610 read 0x3ca8a7 thru 0x3d0537 (0x5c91 bytes) 000433[0] 1736263560.841075 mapwrite 0x9c8106 thru 0x9cc03a (0x3f35 bytes) 000435[0] 1736263560.884583 mapwrite 0x5531e6 thru 0x556820 (0x363b bytes) 000436[0] 1736263560.921016 trunc from 0x9cc03b to 0x4541ba (0x577e80 bytes) 000438[1] 1736263560.945895 read 0x3d7bd5 thru 0x3dabf0 (0x301c bytes) 000441[1] 1736263560.973532 trunc from 0x4541bb to 0x3fde08 (0x563b2 bytes) 000443[1] 1736263560.990835 read 0x077160 thru 0x085c80 (0xeb21 bytes) 000444[1] 1736263561.025479 read 0x02b455 thru 0x02bc84 (0x830 bytes) 000446[1] 1736263561.044113 mapwrite 0x0c497d thru 0x0cc047 (0x76cb bytes) 000447[1] 1736263561.073353 mapread 0x2f1af7 thru 0x2fe343 (0xc84d bytes) 000448[1] 1736263561.073949 mapwrite 0x8c5b47 thru 0x8cd412 (0x78cc bytes) 000451[1] 1736263561.163603 mapwrite 0x271da4 thru 0x276c52 (0x4eaf bytes) 000453[0] 1736263561.196935 write 0x69726f thru 0x69d627 (0x63b9 bytes) 000455[1] 1736263561.230773 trunc from 0x8cd413 to 0x37983a (0x553bd8 bytes) 000456[0] 1736263561.259615 write 0x47b22c thru 0x48917b (0xdf50 bytes) 000458[1] 1736263561.280126 mapwrite 0x641eeb thru 0x64ad4a (0x8e60 bytes) 000460[0] 1736263561.385214 read 0x11daa8 thru 0x12b704 (0xdc5d bytes) 000462[1] 1736263561.410636 mapwrite 0x6a5df0 thru 0x6af13c (0x934d bytes) 000463[1] 1736263561.491074 trunc from 0x6af13d to 0x5e85bc (0xc6b80 bytes) 000464[1] 1736263561.518054 trunc from 0x5e85bd to 0x8838ad (0x29b2f1 bytes) 000469[1] 1736263561.557414 read 0x5c16e2 thru 0x5c6d2a (0x5649 bytes) 000478[1] 1736263561.625113 trunc from 0x8838ae to 0x85c46b (0x27442 bytes) 000479[1] 1736263561.643199 mapread 0x245e76 thru 0x25196a (0xbaf5 bytes) 000482[0] 1736263561.692808 mapread 0x7ba70a thru 0x7c0bf3 (0x64ea bytes) 000485[0] 1736263561.744363 mapwrite 0x5d5478 thru 0x5e2223 (0xcdac bytes) 000486[0] 1736263561.846998 trunc from 0x85c46c to 0x3e9911 (0x472b5a bytes) 000487[1] 1736263561.879574 write 0x7b40b0 thru 0x7b78ae (0x37ff bytes) 000488[0] 1736263561.894994 write 0x12ddec thru 0x134ae0 (0x6cf5 bytes) 000489[1] 1736263561.950425 trunc from 0x7b78af to 0x686350 (0x13155e bytes) 000490[1] 1736263561.995116 mapwrite 0x895d4d thru 0x89b8a7 (0x5b5b bytes) 000491[1] 1736263562.071775 write 0x7c2762 thru 0x7caece (0x876d bytes) 000494[0] 1736263562.081333 mapread 0x420622 thru 0x4234f3 (0x2ed2 bytes) 000495[0] 1736263562.117713 read 0x3497a0 thru 0x34b7e6 (0x2047 bytes) 000497[0] 1736263562.189015 mapread 0x0ed0f0 thru 0x0fc07b (0xef8c bytes) 000498[0] 1736263562.231227 write 0x2a1bc9 thru 0x2a4464 (0x289c bytes) 000499[0] 1736263562.246608 mapread 0x1eedf8 thru 0x1f88eb (0x9af4 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 3562 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000004[0] 1736263562.629456 write 0x61a000 thru 0x62679b (0xc79c bytes) truncating to largest ever: 0x58c31d 000009[0] 1736263562.646968 trunc from 0x62679c to 0x58c31c (0x9a47f bytes) 000012[0] 1736263562.679806 read 0x1f3000 thru 0x1fd3b7 (0xa3b8 bytes) 000013[1] 1736263562.699297 trunc from 0x58c31d to 0x1508ec (0x43ba30 bytes) 000014[1] 1736263562.711465 read 0x023000 thru 0x0308f9 (0xd8fa bytes) 000016[0] 1736263562.728509 write 0x540000 thru 0x54d38b (0xd38c bytes) 000017[0] 1736263562.738664 write 0x94b000 thru 0x9549d4 (0x99d5 bytes) 000018[0] 1736263562.748500 mapwrite 0x5b8000 thru 0x5b9af2 (0x1af3 bytes) 000020[0] 1736263562.811888 mapread 0x5d2000 thru 0x5d4f1d (0x2f1e bytes) 000024[0] 1736263562.838057 read 0x5be000 thru 0x5c7251 (0x9252 bytes) truncating to largest ever: 0x5f45e6 000025[0] 1736263562.863855 trunc from 0x9549d5 to 0x5f45e5 (0x3603ef bytes) 000026[1] 1736263562.890718 mapread 0x1f2000 thru 0x1fb658 (0x9659 bytes) 000031[0] 1736263562.913880 trunc from 0x5f45e6 to 0x4e5340 (0x10f2a5 bytes) truncating to largest ever: 0x806068 000034[0] 1736263562.964197 trunc from 0x4e5341 to 0x806067 (0x320d27 bytes) 000035[0] 1736263562.977642 mapread 0x39a000 thru 0x3a3395 (0x9396 bytes) 000036[1] 1736263563.030481 write 0x50f000 thru 0x515d6f (0x6d70 bytes) 000038[1] 1736263563.057723 mapwrite 0x6da000 thru 0x6e347d (0x947e bytes) 000040[1] 1736263563.121574 mapwrite 0x64b000 thru 0x64d3f5 (0x23f6 bytes) 000044[0] 1736263563.168175 mapread 0x3d8000 thru 0x3e36d0 (0xb6d1 bytes) 000045[1] 1736263563.220201 write 0x177000 thru 0x17e33f (0x7340 bytes) 000046[1] 1736263563.242884 mapwrite 0x925000 thru 0x9326cc (0xd6cd bytes) 000048[0] 1736263563.331060 read 0x680000 thru 0x68e35e (0xe35f bytes) 000049[1] 1736263563.356590 trunc from 0x9326cd to 0x59a664 (0x398068 bytes) 000050[1] 1736263563.391434 mapread 0x38d000 thru 0x390251 (0x3252 bytes) truncating to largest ever: 0x9e4c6b 000052[1] 1736263563.421500 trunc from 0x59a665 to 0x9e4c6a (0x44a606 bytes) 000053[0] 1736263563.447852 mapwrite 0x835000 thru 0x842a0d (0xda0e bytes) 000054[1] 1736263563.544839 mapwrite 0x0da000 thru 0x0db91e (0x191f bytes) 000055[1] 1736263563.607928 mapread 0x8ba000 thru 0x8c8aa5 (0xeaa6 bytes) 000057[0] 1736263563.625722 mapwrite 0x576000 thru 0x583d2c (0xdd2d bytes) 000058[0] 1736263563.681051 trunc from 0x9e4c6b to 0x347acf (0x69d19b bytes) 000060[0] 1736263563.688402 mapread 0x03a000 thru 0x03ac04 (0xc05 bytes) 000064[1] 1736263563.702805 mapread 0x0c0000 thru 0x0c273b (0x273c bytes) 000068[1] 1736263563.721844 mapwrite 0x6b9000 thru 0x6c1fc8 (0x8fc9 bytes) 000070[0] 1736263563.785051 write 0x8bc000 thru 0x8bc8e7 (0x8e8 bytes) 000071[0] 1736263563.791617 mapwrite 0x071000 thru 0x07f80e (0xe80f bytes) 000072[1] 1736263563.876127 trunc from 0x8bc8e8 to 0x1aa826 (0x7120c1 bytes) 000073[0] 1736263563.899698 mapwrite 0x253000 thru 0x257b6e (0x4b6f bytes) 000074[1] 1736263563.959213 read 0x14f000 thru 0x150281 (0x1282 bytes) 000077[1] 1736263563.973727 read 0x117000 thru 0x11e35b (0x735c bytes) 000079[1] 1736263563.981872 mapwrite 0x339000 thru 0x346080 (0xd081 bytes) 000080[1] 1736263564.039578 mapwrite 0x379000 thru 0x37cb9b (0x3b9c bytes) 000081[0] 1736263564.091276 write 0x65f000 thru 0x662466 (0x3467 bytes) 000082[0] 1736263564.104632 mapwrite 0x275000 thru 0x276a42 (0x1a43 bytes) 000083[0] 1736263564.141861 mapwrite 0x35b000 thru 0x361673 (0x6674 bytes) 000084[0] 1736263564.202998 mapread 0x5ca000 thru 0x5d0f79 (0x6f7a bytes) 000086[1] 1736263564.230555 mapread 0x0af000 thru 0x0b6290 (0x7291 bytes) 000090[1] 1736263564.267595 mapwrite 0x8b2000 thru 0x8be1cc (0xc1cd bytes) 000093[0] 1736263564.351852 write 0x8aa000 thru 0x8b8690 (0xe691 bytes) 000095[0] 1736263564.375374 mapwrite 0x5b4000 thru 0x5bb9cb (0x79cc bytes) 000098[0] 1736263564.457187 trunc from 0x8be1cd to 0x0bc9c3 (0x801809 bytes) 000100[1] 1736263564.475050 trunc from 0x0bc9c4 to 0x6a898d (0x5ebfca bytes) 000101[0] 1736263564.491625 trunc from 0x6a898e to 0x6781af (0x307de bytes) 000102[0] 1736263564.510000 mapread 0x0c0000 thru 0x0c1e16 (0x1e17 bytes) 000103[0] 1736263564.514250 write 0x27b000 thru 0x2835b3 (0x85b4 bytes) 000105[0] 1736263564.529837 trunc from 0x6781b0 to 0x5bc803 (0xbb9ac bytes) 000106[0] 1736263564.535853 mapread 0x426000 thru 0x42bcf0 (0x5cf1 bytes) 000107[0] 1736263564.559189 mapwrite 0x41f000 thru 0x420711 (0x1712 bytes) 000109[1] 1736263564.603961 write 0x2bc000 thru 0x2c09bb (0x49bc bytes) 000110[1] 1736263564.615473 mapread 0x4a0000 thru 0x4a02c1 (0x2c2 bytes) 000111[1] 1736263564.623140 trunc from 0x5bc804 to 0x28ba74 (0x330d8f bytes) 000112[1] 1736263564.639306 mapread 0x0a8000 thru 0x0ab8f9 (0x38fa bytes) 000114[0] 1736263564.648748 mapread 0x088000 thru 0x0959f1 (0xd9f2 bytes) 000115[0] 1736263564.692147 mapread 0x244000 thru 0x24fc2c (0xbc2d bytes) 000119[1] 1736263564.728281 read 0x010000 thru 0x017923 (0x7924 bytes) 000123[1] 1736263564.757418 write 0x623000 thru 0x62da6b (0xaa6c bytes) 000127[1] 1736263564.795146 trunc from 0x62da6c to 0x5b5d4b (0x77d20 bytes) 000128[1] 1736263564.810194 read 0x3f8000 thru 0x40772d (0xf72e bytes) 000131[1] 1736263564.846077 write 0x4ac000 thru 0x4b686b (0xa86c bytes) 000132[0] 1736263564.856987 trunc from 0x5b5d4c to 0x94958b (0x393840 bytes) 000134[0] 1736263564.890854 trunc from 0x94958c to 0x338f9b (0x6105f0 bytes) 000136[0] 1736263564.897463 write 0x102000 thru 0x10f10c (0xd10d bytes) 000138[0] 1736263564.914169 write 0x7db000 thru 0x7db063 (0x64 bytes) 000139[1] 1736263564.920621 mapread 0x5de000 thru 0x5e6e5c (0x8e5d bytes) 000140[1] 1736263564.944569 read 0x632000 thru 0x64013a (0xe13b bytes) 000141[0] 1736263564.946231 trunc from 0x7db064 to 0x2a0f69 (0x53a0fa bytes) 000142[1] 1736263564.973922 write 0x799000 thru 0x7a37b3 (0xa7b4 bytes) 000143[1] 1736263564.992210 mapwrite 0x0ac000 thru 0x0b6337 (0xa338 bytes) 000144[0] 1736263565.069488 read 0x077000 thru 0x07f5c7 (0x85c8 bytes) 000145[0] 1736263565.115874 read 0x4a4000 thru 0x4b26be (0xe6bf bytes) 000147[0] 1736263565.128255 read 0x2b9000 thru 0x2c62e4 (0xd2e5 bytes) 000148[1] 1736263565.128542 mapread 0x208000 thru 0x21543c (0xd43d bytes) 000151[1] 1736263565.155333 mapread 0x55f000 thru 0x5681ee (0x91ef bytes) 000152[0] 1736263565.179952 mapread 0x18f000 thru 0x195459 (0x645a bytes) 000154[0] 1736263565.180864 read 0x10c000 thru 0x116217 (0xa218 bytes) 000155[0] 1736263565.181120 mapread 0x1b6000 thru 0x1bd940 (0x7941 bytes) 000157[1] 1736263565.181564 trunc from 0x7a37b4 to 0x66bb13 (0x137ca0 bytes) 000159[0] 1736263565.199597 read 0x4ec000 thru 0x4f417c (0x817d bytes) 000160[1] 1736263565.216250 mapread 0x518000 thru 0x51c6d0 (0x46d1 bytes) 000161[0] 1736263565.231332 write 0x4ae000 thru 0x4b9944 (0xb945 bytes) 000162[0] 1736263565.253004 read 0x414000 thru 0x4160d0 (0x20d1 bytes) 000163[1] 1736263565.278138 mapread 0x616000 thru 0x61b977 (0x5978 bytes) 000164[0] 1736263565.311581 mapread 0x131000 thru 0x135eba (0x4ebb bytes) 000167[1] 1736263565.334779 write 0x45d000 thru 0x464229 (0x722a bytes) 000170[1] 1736263565.366679 trunc from 0x66bb14 to 0x4dd75b (0x18e3b8 bytes) 000172[1] 1736263565.378953 read 0x0e5000 thru 0x0f0603 (0xb604 bytes) 000173[0] 1736263565.405702 trunc from 0x4dd75c to 0x92b8b4 (0x44e159 bytes) 000174[0] 1736263565.435264 write 0x9c1000 thru 0x9cd93f (0xc940 bytes) 000175[1] 1736263565.441461 mapwrite 0x1cc000 thru 0x1d1b62 (0x5b63 bytes) 000177[1] 1736263565.532764 mapwrite 0x01d000 thru 0x028e58 (0xbe59 bytes) 000178[1] 1736263565.592990 mapread 0x498000 thru 0x49fb44 (0x7b45 bytes) 000180[0] 1736263565.624542 mapwrite 0x009000 thru 0x00ecc3 (0x5cc4 bytes) 000182[1] 1736263565.707907 write 0x35a000 thru 0x35a7d5 (0x7d6 bytes) 000184[1] 1736263565.745109 mapwrite 0x0ce000 thru 0x0da372 (0xc373 bytes) 000186[1] 1736263565.836095 read 0x776000 thru 0x781d14 (0xbd15 bytes) 000187[0] 1736263565.856900 read 0x7fc000 thru 0x802b94 (0x6b95 bytes) 000188[0] 1736263565.880219 read 0x056000 thru 0x061b96 (0xbb97 bytes) 000190[1] 1736263565.920155 mapwrite 0x345000 thru 0x3454c2 (0x4c3 bytes) 000191[0] 1736263565.972160 write 0x8b5000 thru 0x8b8b94 (0x3b95 bytes) 000193[1] 1736263565.986739 read 0x65e000 thru 0x66c57b (0xe57c bytes) 000195[0] 1736263566.017595 mapwrite 0x6f7000 thru 0x6fcb4d (0x5b4e bytes) 000197[1] 1736263566.083374 read 0x7b8000 thru 0x7ba90c (0x290d bytes) 000199[0] 1736263566.102248 mapread 0x77d000 thru 0x788ee5 (0xbee6 bytes) 000202[0] 1736263566.132379 trunc from 0x9cd940 to 0x220d29 (0x7acc16 bytes) 000203[0] 1736263566.156391 trunc from 0x220d2a to 0x25f86e (0x3eb45 bytes) 000213[1] 1736263566.225659 write 0x2ca000 thru 0x2cd103 (0x3104 bytes) 000214[0] 1736263566.230431 write 0x552000 thru 0x561957 (0xf958 bytes) 000215[0] 1736263566.243554 write 0x4cc000 thru 0x4cf478 (0x3479 bytes) 000216[1] 1736263566.246218 trunc from 0x561958 to 0x411826 (0x150131 bytes) 000217[0] 1736263566.270566 mapwrite 0x6ff000 thru 0x707544 (0x8545 bytes) 000218[1] 1736263566.328924 mapwrite 0x679000 thru 0x67e61c (0x561d bytes) 000221[0] 1736263566.387727 read 0x5f0000 thru 0x5f3068 (0x3069 bytes) 000224[1] 1736263566.404402 trunc from 0x707545 to 0x68967f (0x7dec5 bytes) 000225[1] 1736263566.430702 write 0x4a1000 thru 0x4a13f9 (0x3fa bytes) 000226[1] 1736263566.440728 mapwrite 0x5f0000 thru 0x5fd870 (0xd871 bytes) 000227[1] 1736263566.519122 trunc from 0x689680 to 0x395457 (0x2f4228 bytes) 000230[0] 1736263566.539394 read 0x101000 thru 0x10da2f (0xca30 bytes) 000233[1] 1736263566.566313 mapread 0x07f000 thru 0x0853f8 (0x63f9 bytes) 000234[1] 1736263566.585554 mapread 0x16d000 thru 0x174738 (0x7739 bytes) 000235[0] 1736263566.605687 trunc from 0x395458 to 0x9db5d6 (0x64617f bytes) 000238[1] 1736263566.627605 write 0x856000 thru 0x8574f5 (0x14f6 bytes) 000240[1] 1736263566.649374 trunc from 0x9db5d7 to 0x994be1 (0x469f5 bytes) 000241[0] 1736263566.666919 read 0x2bb000 thru 0x2c1be0 (0x6be1 bytes) 000242[0] 1736263566.713008 mapread 0x284000 thru 0x2925db (0xe5dc bytes) 000244[1] 1736263566.743553 write 0x3b3000 thru 0x3b6b3f (0x3b40 bytes) 000246[0] 1736263566.760715 write 0x306000 thru 0x30a05a (0x405b bytes) 000249[0] 1736263566.780236 mapread 0x3d7000 thru 0x3dbb57 (0x4b58 bytes) 000252[1] 1736263566.815913 trunc from 0x994be2 to 0x119933 (0x87b2ae bytes) 000253[0] 1736263566.837800 write 0x308000 thru 0x30af78 (0x2f79 bytes) 000254[0] 1736263566.846628 mapread 0x24b000 thru 0x25387b (0x887c bytes) 000255[0] 1736263566.873296 mapwrite 0x14c000 thru 0x14e04e (0x204f bytes) 000258[0] 1736263566.913640 mapwrite 0x11d000 thru 0x12c4f3 (0xf4f4 bytes) 000264[0] 1736263566.987296 write 0x28c000 thru 0x291165 (0x5166 bytes) 000265[0] 1736263566.989537 write 0x002000 thru 0x002695 (0x696 bytes) 000267[1] 1736263566.991704 read 0x1e1000 thru 0x1ec0bb (0xb0bc bytes) 000270[1] 1736263567.013884 read 0x046000 thru 0x04ee42 (0x8e43 bytes) 000271[0] 1736263567.029707 mapwrite 0x943000 thru 0x94a6c2 (0x76c3 bytes) 000273[1] 1736263567.114928 mapread 0x3a5000 thru 0x3b120b (0xc20c bytes) 000274[1] 1736263567.155976 trunc from 0x94a6c3 to 0x9e3dce (0x9970c bytes) 000276[1] 1736263567.193807 read 0x246000 thru 0x24a089 (0x408a bytes) 000279[1] 1736263567.247979 mapread 0x326000 thru 0x328bd1 (0x2bd2 bytes) 000281[0] 1736263567.256704 trunc from 0x9e3dcf to 0x0424af (0x9a191f bytes) 000282[0] 1736263567.281101 mapwrite 0x5f1000 thru 0x5fd1e9 (0xc1ea bytes) 000284[0] 1736263567.390740 read 0x4d3000 thru 0x4e1688 (0xe689 bytes) 000285[1] 1736263567.404643 mapwrite 0x58b000 thru 0x58d336 (0x2337 bytes) 000287[1] 1736263567.447915 mapwrite 0x079000 thru 0x0815ae (0x85af bytes) 000288[0] 1736263567.523962 write 0x28f000 thru 0x29af0a (0xbf0b bytes) 000290[1] 1736263567.547988 read 0x02e000 thru 0x03800a (0xa00b bytes) 000291[1] 1736263567.594654 write 0x730000 thru 0x73a4a5 (0xa4a6 bytes) 000292[0] 1736263567.607125 mapread 0x44f000 thru 0x453500 (0x4501 bytes) 000296[0] 1736263567.654521 read 0x68a000 thru 0x6988b4 (0xe8b5 bytes) 000299[0] 1736263567.655544 trunc from 0x73a4a6 to 0x45d62d (0x2dce78 bytes) 000301[1] 1736263567.690711 mapread 0x2f1000 thru 0x2fe129 (0xd12a bytes) 000303[1] 1736263567.750220 read 0x457000 thru 0x459cc2 (0x2cc3 bytes) 000304[0] 1736263567.753991 read 0x3f7000 thru 0x403f93 (0xcf94 bytes) 000306[1] 1736263567.761919 trunc from 0x45d62e to 0x3e6bce (0x76a5f bytes) 000307[0] 1736263567.777896 mapread 0x3e4000 thru 0x3e6389 (0x238a bytes) truncating to largest ever: 0x9ec067 000308[0] 1736263567.791636 trunc from 0x3e6bcf to 0x9ec066 (0x605498 bytes) 000311[0] 1736263567.818003 read 0x533000 thru 0x53a860 (0x7861 bytes) 000312[0] 1736263567.847513 trunc from 0x9ec067 to 0x90a0d3 (0xe1f93 bytes) 000313[1] 1736263567.863719 read 0x456000 thru 0x46165b (0xb65c bytes) 000314[1] 1736263567.903820 mapwrite 0x156000 thru 0x16501c (0xf01d bytes) 000318[0] 1736263568.040526 mapwrite 0x6e5000 thru 0x6e5b5f (0xb60 bytes) 000320[0] 1736263568.095470 mapwrite 0x1dc000 thru 0x1e6ddc (0xaddd bytes) 000322[1] 1736263568.178087 trunc from 0x90a0d4 to 0x018635 (0x8f1a9e bytes) 000323[0] 1736263568.188902 trunc from 0x018636 to 0x2b371b (0x29b0e6 bytes) 000324[1] 1736263568.200357 write 0x3ca000 thru 0x3d8f4a (0xef4b bytes) 000326[0] 1736263568.214408 mapwrite 0x50e000 thru 0x516d2b (0x8d2c bytes) 000328[0] 1736263568.321915 mapread 0x38a000 thru 0x395a36 (0xba37 bytes) 000330[0] 1736263568.349878 mapwrite 0x9f0000 thru 0x9fcc48 (0xcc49 bytes) 000331[0] 1736263568.418396 mapread 0x25b000 thru 0x269b9e (0xeb9f bytes) 000332[0] 1736263568.472288 write 0x545000 thru 0x548a43 (0x3a44 bytes) 000335[0] 1736263568.488669 mapwrite 0x8ac000 thru 0x8add8c (0x1d8d bytes) 000337[1] 1736263568.543751 mapwrite 0x23e000 thru 0x247d7f (0x9d80 bytes) 000338[1] 1736263568.629444 mapread 0x9de000 thru 0x9e37ef (0x57f0 bytes) 000339[1] 1736263568.636802 mapwrite 0x2e7000 thru 0x2e7879 (0x87a bytes) 000342[0] 1736263568.681367 write 0x958000 thru 0x95fd4c (0x7d4d bytes) 000343[1] 1736263568.689572 mapread 0x8d8000 thru 0x8e091a (0x891b bytes) 000344[0] 1736263568.726893 write 0x02a000 thru 0x0389d2 (0xe9d3 bytes) 000345[1] 1736263568.741102 write 0x01a000 thru 0x023156 (0x9157 bytes) 000346[1] 1736263568.751409 mapwrite 0x146000 thru 0x14cb11 (0x6b12 bytes) 000349[1] 1736263568.823727 mapwrite 0x4ee000 thru 0x4f228d (0x428e bytes) 000350[1] 1736263568.880852 read 0x16c000 thru 0x175fbc (0x9fbd bytes) 000351[1] 1736263568.917276 mapread 0x115000 thru 0x1244ab (0xf4ac bytes) 000352[1] 1736263568.961053 read 0x691000 thru 0x69327e (0x227f bytes) 000353[0] 1736263568.968786 write 0x121000 thru 0x128901 (0x7902 bytes) 000356[0] 1736263568.999387 mapread 0x483000 thru 0x48ac05 (0x7c06 bytes) 000357[0] 1736263569.018221 mapwrite 0x44a000 thru 0x45310f (0x9110 bytes) 000358[0] 1736263569.075515 mapread 0x2ac000 thru 0x2b77a1 (0xb7a2 bytes) 000359[1] 1736263569.100331 mapwrite 0x114000 thru 0x1237f6 (0xf7f7 bytes) 000360[1] 1736263569.182506 write 0x4bd000 thru 0x4c8eab (0xbeac bytes) 000362[0] 1736263569.203175 trunc from 0x9fcc49 to 0x9b8b70 (0x440d8 bytes) 000363[0] 1736263569.239233 read 0x5b5000 thru 0x5b72fc (0x22fd bytes) 000364[0] 1736263569.263462 write 0x881000 thru 0x890039 (0xf03a bytes) 000365[0] 1736263569.264804 mapwrite 0x4d4000 thru 0x4d868e (0x468f bytes) 000366[1] 1736263569.362695 mapread 0x5df000 thru 0x5e13cb (0x23cc bytes) 000367[0] 1736263569.385615 read 0x2ff000 thru 0x2ff4d7 (0x4d8 bytes) 000368[1] 1736263569.417986 mapwrite 0x4f8000 thru 0x4fbfe3 (0x3fe4 bytes) 000369[1] 1736263569.481440 read 0x56f000 thru 0x57727b (0x827c bytes) 000370[0] 1736263569.504089 mapwrite 0x4ae000 thru 0x4bcbb5 (0xebb6 bytes) 000371[0] 1736263569.605514 mapread 0x83a000 thru 0x843802 (0x9803 bytes) 000373[1] 1736263569.628191 write 0x776000 thru 0x782d2e (0xcd2f bytes) 000377[0] 1736263569.684445 mapwrite 0x3e5000 thru 0x3e6c80 (0x1c81 bytes) 000379[0] 1736263569.740586 mapread 0x11e000 thru 0x12d6db (0xf6dc bytes) 000381[0] 1736263569.782252 read 0x57d000 thru 0x580ec1 (0x3ec2 bytes) 000382[1] 1736263569.801457 trunc from 0x9b8b71 to 0x269a99 (0x74f0d7 bytes) 000383[0] 1736263569.831513 trunc from 0x269a9a to 0x6d7315 (0x46d87c bytes) 000385[0] 1736263569.851416 trunc from 0x6d7316 to 0x7bf0e0 (0xe7dcb bytes) 000386[0] 1736263569.864203 mapread 0x4be000 thru 0x4cdcba (0xfcbb bytes) 000387[0] 1736263569.919511 mapread 0x75f000 thru 0x7657c8 (0x67c9 bytes) 000392[1] 1736263569.946695 read 0x1f4000 thru 0x201138 (0xd139 bytes) 000393[1] 1736263569.983577 mapwrite 0x7c2000 thru 0x7cd497 (0xb498 bytes) 000394[0] 1736263570.049869 write 0x2d1000 thru 0x2d8fca (0x7fcb bytes) 000396[0] 1736263570.067534 trunc from 0x7cd498 to 0x955c1e (0x188787 bytes) 000397[1] 1736263570.081364 mapwrite 0x1fe000 thru 0x20be03 (0xde04 bytes) 000402[0] 1736263570.182802 write 0x2f6000 thru 0x2f7f96 (0x1f97 bytes) 000403[0] 1736263570.190524 trunc from 0x955c1f to 0x98485d (0x2ec3f bytes) 000405[0] 1736263570.212279 trunc from 0x98485e to 0x1d40ed (0x7b0770 bytes) 000406[0] 1736263570.230281 mapwrite 0x498000 thru 0x49cbda (0x4bdb bytes) 000408[1] 1736263570.297655 trunc from 0x49cbdb to 0x6284d6 (0x18b8fc bytes) 000410[0] 1736263570.315667 read 0x061000 thru 0x066083 (0x5084 bytes) 000413[1] 1736263570.348005 mapread 0x049000 thru 0x04ffcf (0x6fd0 bytes) 000414[1] 1736263570.360302 write 0x269000 thru 0x26d13c (0x413d bytes) 000416[1] 1736263570.385658 write 0x1f7000 thru 0x1fd3ea (0x63eb bytes) 000420[1] 1736263570.395095 trunc from 0x6284d7 to 0x52dc41 (0xfa895 bytes) 000421[1] 1736263570.408335 mapread 0x47f000 thru 0x48a109 (0xb10a bytes) 000422[1] 1736263570.436116 write 0x73b000 thru 0x73b5ca (0x5cb bytes) 000427[0] 1736263570.464821 mapread 0x484000 thru 0x491fc8 (0xdfc9 bytes) 000429[1] 1736263570.524755 trunc from 0x73b5cb to 0x9d5f48 (0x29a97e bytes) 000436[1] 1736263570.566755 mapwrite 0x7d0000 thru 0x7dab92 (0xab93 bytes) 000438[1] 1736263570.630224 read 0x65e000 thru 0x66c919 (0xe91a bytes) 000439[1] 1736263570.658293 write 0x8dd000 thru 0x8e9c5d (0xcc5e bytes) 000440[0] 1736263570.659053 mapwrite 0x69c000 thru 0x6a53be (0x93bf bytes) 000441[0] 1736263570.754294 mapread 0x97f000 thru 0x985d15 (0x6d16 bytes) 000442[0] 1736263570.768878 write 0x564000 thru 0x566214 (0x2215 bytes) 000443[0] 1736263570.773326 trunc from 0x9d5f49 to 0x73f21d (0x296d2b bytes) 000445[1] 1736263570.787008 trunc from 0x73f21e to 0x5ba629 (0x184bf4 bytes) 000448[1] 1736263570.825783 write 0x96d000 thru 0x971c2e (0x4c2f bytes) 000450[1] 1736263570.835976 trunc from 0x971c2f to 0x5b26b7 (0x3bf577 bytes) 000451[0] 1736263570.850403 mapwrite 0x9f4000 thru 0x9f6824 (0x2825 bytes) 000453[1] 1736263570.924602 mapread 0x4ac000 thru 0x4b16f8 (0x56f9 bytes) 000455[0] 1736263570.950086 read 0x6bc000 thru 0x6c361b (0x761c bytes) 000457[1] 1736263570.974588 write 0x0e8000 thru 0x0ea153 (0x2154 bytes) 000459[0] 1736263570.990682 trunc from 0x9f6825 to 0x06c24b (0x98a5d9 bytes) 000464[1] 1736263571.059849 trunc from 0x06c24c to 0x1e9942 (0x17d6f7 bytes) 000467[1] 1736263571.096094 write 0x83e000 thru 0x84c98b (0xe98c bytes) 000469[0] 1736263571.109297 read 0x107000 thru 0x113ec2 (0xcec3 bytes) 000471[0] 1736263571.153571 write 0x698000 thru 0x6a26ec (0xa6ed bytes) 000473[1] 1736263571.179936 write 0x705000 thru 0x707939 (0x293a bytes) 000474[1] 1736263571.192338 write 0x088000 thru 0x08c06f (0x4070 bytes) 000476[1] 1736263571.202809 trunc from 0x84c98c to 0x99accb (0x14e340 bytes) 000481[0] 1736263571.212934 trunc from 0x99accc to 0x856f76 (0x143d55 bytes) 000482[1] 1736263571.231541 trunc from 0x856f77 to 0x958cae (0x101d38 bytes) 000483[1] 1736263571.246490 mapwrite 0x9e9000 thru 0x9ec850 (0x3851 bytes) 000484[0] 1736263571.296176 mapread 0x0bc000 thru 0x0c415f (0x8160 bytes) 000485[1] 1736263571.336006 write 0x620000 thru 0x62f185 (0xf186 bytes) 000486[0] 1736263571.346618 write 0x46c000 thru 0x46e747 (0x2748 bytes) 000488[1] 1736263571.361526 mapwrite 0x46c000 thru 0x474220 (0x8221 bytes) 000490[1] 1736263571.417346 trunc from 0x9ec851 to 0x338b85 (0x6b3ccb bytes) 000491[0] 1736263571.438688 write 0x255000 thru 0x26378c (0xe78d bytes) 000492[0] 1736263571.456645 mapread 0x116000 thru 0x119b97 (0x3b98 bytes) 000493[1] 1736263571.472704 mapwrite 0x9c4000 thru 0x9d1735 (0xd736 bytes) 000494[1] 1736263571.574165 mapread 0x007000 thru 0x00e521 (0x7522 bytes) 000495[0] 1736263571.596503 mapread 0x4b9000 thru 0x4c7bb2 (0xebb3 bytes) 000497[0] 1736263571.647136 read 0x2e6000 thru 0x2e9a74 (0x3a75 bytes) 000499[1] 1736263571.685559 write 0x76f000 thru 0x7765ac (0x75ad bytes) All operations completed A-OK! PASS 16a (24s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 10:26:13 (1736263573) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.142764 s, 14.7 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0975024 s, 21.5 MB/s Chance of close/open is 1 in 50 Seed set to 3573 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[1] 1736263574.255013 mapwrite 0x41110f thru 0x412e1c (0x1d0e bytes) 000002[0] 1736263574.321922 mapread 0x2d66c4 thru 0x2d710d (0xa4a bytes) truncating to largest ever: 0x5cc850 000003[1] 1736263574.343065 trunc from 0x412e1d to 0x5cc84f (0x1b9a33 bytes) 000005[1] 1736263574.367304 read 0x58baef thru 0x58bf18 (0x42a bytes) truncating to largest ever: 0x5cd97b 000006[0] 1736263574.376310 trunc from 0x5cc850 to 0x5cd97a (0x112b bytes) 000007[0] 1736263574.391982 mapread 0x2a9548 thru 0x2aaa75 (0x152e bytes) 000009[0] 1736263574.406125 mapread 0x451140 thru 0x452789 (0x164a bytes) 000011[1] 1736263574.425460 read 0x3974b4 thru 0x398f2d (0x1a7a bytes) 000012[0] 1736263574.440302 write 0x200694 thru 0x201770 (0x10dd bytes) 000013[1] 1736263574.454391 read 0x231191 thru 0x232a89 (0x18f9 bytes) 000015[0] 1736263574.488340 mapwrite 0x8019dd thru 0x802d0a (0x132e bytes) 000018[0] 1736263574.560743 trunc from 0x802d0b to 0x3fc684 (0x406686 bytes) 000019[0] 1736263574.572336 mapread 0x352a86 thru 0x354960 (0x1edb bytes) 000022[0] 1736263574.590895 mapwrite 0x5c3b28 thru 0x5c443b (0x914 bytes) 000024[1] 1736263574.647696 trunc from 0x5c443c to 0x372670 (0x251dcb bytes) 000025[1] 1736263574.679910 write 0x27646a thru 0x277a35 (0x15cc bytes) truncating to largest ever: 0x74bc9d 000026[1] 1736263574.687150 trunc from 0x372671 to 0x74bc9c (0x3d962c bytes) 000027[1] 1736263574.698066 mapwrite 0x3d55e8 thru 0x3d5e3d (0x856 bytes) 000030[0] 1736263574.746651 trunc from 0x74bc9d to 0x4b211b (0x299b81 bytes) 000031[1] 1736263574.768906 trunc from 0x4b211c to 0x53f653 (0x8d538 bytes) 000032[1] 1736263574.793834 mapwrite 0x9b0daf thru 0x9b25aa (0x17fc bytes) 000033[0] 1736263574.862821 write 0x8583d8 thru 0x858d70 (0x999 bytes) 000035[0] 1736263574.886947 write 0x2f3c48 thru 0x2f583a (0x1bf3 bytes) 000036[0] 1736263574.898179 mapwrite 0x7be404 thru 0x7c025e (0x1e5b bytes) 000037[1] 1736263574.959068 mapwrite 0x28417b thru 0x284b20 (0x9a6 bytes) 000040[1] 1736263575.016421 read 0x95524f thru 0x955ea6 (0xc58 bytes) 000042[1] 1736263575.039685 mapwrite 0x3d3fe5 thru 0x3d43eb (0x407 bytes) 000043[1] 1736263575.099243 mapread 0x28771c thru 0x289131 (0x1a16 bytes) 000045[1] 1736263575.103746 mapread 0x11c470 thru 0x11c8ae (0x43f bytes) truncating to largest ever: 0x9baefc 000046[0] 1736263575.109723 trunc from 0x9b25ab to 0x9baefb (0x8951 bytes) 000049[0] 1736263575.142572 mapwrite 0x1b30c6 thru 0x1b408a (0xfc5 bytes) 000050[0] 1736263575.185784 mapwrite 0x7fbf5d thru 0x7fc669 (0x70d bytes) 000051[1] 1736263575.231731 read 0x7188dc thru 0x719085 (0x7aa bytes) 000054[1] 1736263575.251100 trunc from 0x9baefc to 0x391d6e (0x62918d bytes) 000059[1] 1736263575.300398 write 0x6015fc thru 0x602c53 (0x1658 bytes) 000061[1] 1736263575.301781 mapread 0x4efe44 thru 0x4f0905 (0xac2 bytes) 000063[0] 1736263575.328526 mapwrite 0x5e1d5a thru 0x5e237a (0x621 bytes) 000066[0] 1736263575.431915 read 0x4372f4 thru 0x437de7 (0xaf4 bytes) 000084[0] 1736263575.512518 read 0x065f5c thru 0x067a6c (0x1b11 bytes) 000085[0] 1736263575.559612 read 0x06cac5 thru 0x06de04 (0x1340 bytes) 000087[0] 1736263575.581907 mapread 0x189eb0 thru 0x18bbc5 (0x1d16 bytes) 000089[1] 1736263575.588700 mapwrite 0x1396c0 thru 0x13b5f5 (0x1f36 bytes) 000093[1] 1736263575.656259 read 0x48e4a0 thru 0x48f76f (0x12d0 bytes) 000094[1] 1736263575.673693 trunc from 0x602c54 to 0x897ee0 (0x29528d bytes) 000095[0] 1736263575.689534 read 0x265a3b thru 0x267688 (0x1c4e bytes) 000096[0] 1736263575.713009 mapread 0x55e08d thru 0x560039 (0x1fad bytes) 000097[1] 1736263575.746791 mapread 0x6907c6 thru 0x6923bf (0x1bfa bytes) 000099[0] 1736263575.770621 mapread 0x12c15b thru 0x12caa1 (0x947 bytes) 000100[0] 1736263575.774322 read 0x618b0f thru 0x61950b (0x9fd bytes) 000101[0] 1736263575.775381 trunc from 0x897ee1 to 0x4a0f24 (0x3f6fbc bytes) 000102[0] 1736263575.799655 mapwrite 0x22c6fb thru 0x22cc2d (0x533 bytes) 000103[1] 1736263575.837927 mapwrite 0x97fe87 thru 0x981cb4 (0x1e2e bytes) 000106[0] 1736263575.902045 write 0x15e47f thru 0x1602ab (0x1e2d bytes) 000107[0] 1736263575.922298 mapwrite 0x1b636c thru 0x1b68a9 (0x53e bytes) 000108[1] 1736263575.966978 trunc from 0x981cb5 to 0x8fa0a8 (0x87c0c bytes) 000109[0] 1736263575.987591 read 0x8c5136 thru 0x8c6607 (0x14d2 bytes) 000110[1] 1736263576.000792 mapwrite 0x9a2bcc thru 0x9a2cdc (0x111 bytes) 000111[1] 1736263576.051009 mapwrite 0x2a52e4 thru 0x2a5679 (0x396 bytes) 000114[1] 1736263576.078461 trunc from 0x9a2cdd to 0x96ead2 (0x3420a bytes) 000115[1] 1736263576.087615 mapread 0x59bb33 thru 0x59c9c5 (0xe93 bytes) 000119[1] 1736263576.119927 trunc from 0x96ead3 to 0x1e12cc (0x78d806 bytes) 000121[1] 1736263576.137661 write 0x28d57b thru 0x28efaf (0x1a35 bytes) 000122[1] 1736263576.144914 mapread 0x024f0c thru 0x02641a (0x150f bytes) 000123[0] 1736263576.160594 read 0x014eb0 thru 0x015c55 (0xda6 bytes) 000124[0] 1736263576.180664 mapread 0x076f7b thru 0x0778bc (0x942 bytes) 000126[1] 1736263576.188184 write 0x597f81 thru 0x599598 (0x1618 bytes) 000127[1] 1736263576.195106 read 0x068f61 thru 0x069b06 (0xba6 bytes) 000128[1] 1736263576.236181 mapwrite 0x410d43 thru 0x412014 (0x12d2 bytes) 000130[1] 1736263576.321489 mapwrite 0x49a5f8 thru 0x49b949 (0x1352 bytes) 000133[1] 1736263576.373165 write 0x0bd520 thru 0x0bdbaf (0x690 bytes) 000135[1] 1736263576.382392 trunc from 0x599599 to 0x0ac271 (0x4ed327 bytes) 000136[0] 1736263576.392393 trunc from 0x0ac272 to 0x7052ea (0x659079 bytes) 000137[0] 1736263576.405860 read 0x322d25 thru 0x32331b (0x5f7 bytes) 000138[1] 1736263576.434121 write 0x9ff8d5 thru 0x9fffff (0x72b bytes) 000139[0] 1736263576.459087 mapread 0x139875 thru 0x13b361 (0x1aed bytes) 000141[1] 1736263576.503531 write 0x7005eb thru 0x7006a3 (0xb9 bytes) 000142[0] 1736263576.518620 read 0x34d8e3 thru 0x34d97f (0x9d bytes) 000143[1] 1736263576.552849 read 0x5abc20 thru 0x5ac4ad (0x88e bytes) 000148[0] 1736263576.590630 mapread 0x917c05 thru 0x9199cc (0x1dc8 bytes) 000152[0] 1736263576.598255 mapwrite 0x573f09 thru 0x573f32 (0x2a bytes) 000154[1] 1736263576.637642 write 0x252e02 thru 0x253afa (0xcf9 bytes) 000156[1] 1736263576.675605 mapread 0x15acf6 thru 0x15ca82 (0x1d8d bytes) 000158[0] 1736263576.705721 trunc from 0xa00000 to 0x43bdb9 (0x5c4246 bytes) 000160[0] 1736263576.742984 read 0x3f2e2c thru 0x3f3fda (0x11af bytes) 000162[0] 1736263576.756808 mapread 0x123c1d thru 0x12486b (0xc4f bytes) 000164[1] 1736263576.773086 read 0x158871 thru 0x159571 (0xd01 bytes) 000165[1] 1736263576.800245 read 0x3bb3fe thru 0x3bcdda (0x19dd bytes) 000167[1] 1736263576.815328 write 0x3dd5dc thru 0x3df30a (0x1d2f bytes) 000169[0] 1736263576.839077 trunc from 0x43bdba to 0x984bb9 (0x548e00 bytes) 000171[1] 1736263576.858784 read 0x878e03 thru 0x87a29f (0x149d bytes) 000172[1] 1736263576.881003 mapwrite 0x01cac2 thru 0x01de22 (0x1361 bytes) 000173[0] 1736263576.925110 read 0x599f75 thru 0x59b1c1 (0x124d bytes) 000174[0] 1736263576.939231 mapread 0x4dc32b thru 0x4dcc66 (0x93c bytes) 000175[1] 1736263576.977444 mapread 0x32bde7 thru 0x32c6d9 (0x8f3 bytes) 000179[1] 1736263577.000291 mapread 0x5c6836 thru 0x5c7aeb (0x12b6 bytes) 000180[1] 1736263577.007307 write 0x0a66a7 thru 0x0a84bc (0x1e16 bytes) 000181[0] 1736263577.021199 write 0x1af06c thru 0x1b0b29 (0x1abe bytes) 000182[1] 1736263577.042481 mapread 0x763f2e thru 0x764948 (0xa1b bytes) 000184[1] 1736263577.058659 read 0x3ef9b1 thru 0x3f19a0 (0x1ff0 bytes) 000186[1] 1736263577.096071 read 0x5a9811 thru 0x5a9cca (0x4ba bytes) 000188[0] 1736263577.102069 mapwrite 0x520b16 thru 0x520fca (0x4b5 bytes) 000189[1] 1736263577.154558 trunc from 0x984bba to 0x4aad93 (0x4d9e26 bytes) 000190[0] 1736263577.172510 trunc from 0x4aad94 to 0x35f043 (0x14bd50 bytes) 000191[0] 1736263577.206606 trunc from 0x35f044 to 0x414a22 (0xb59df bytes) 000198[0] 1736263577.260717 write 0x85d89d thru 0x85e2b7 (0xa1b bytes) 000199[1] 1736263577.265200 write 0x9abc3d thru 0x9ad895 (0x1c59 bytes) 000200[1] 1736263577.277999 mapwrite 0x7ea603 thru 0x7eafab (0x9a9 bytes) 000201[0] 1736263577.325602 mapwrite 0x1c71ee thru 0x1c7b94 (0x9a7 bytes) 000203[0] 1736263577.369141 mapread 0x407670 thru 0x408553 (0xee4 bytes) 000204[1] 1736263577.374090 trunc from 0x9ad896 to 0x0d088f (0x8dd006 bytes) 000205[1] 1736263577.402389 trunc from 0x0d0890 to 0x68d070 (0x5bc7e1 bytes) 000209[1] 1736263577.431401 write 0x8c857e thru 0x8ca039 (0x1abc bytes) 000211[0] 1736263577.439223 mapread 0x4fb482 thru 0x4fd082 (0x1c01 bytes) 000214[0] 1736263577.477285 read 0x46e18a thru 0x46eeb1 (0xd28 bytes) 000215[0] 1736263577.493580 trunc from 0x8ca03a to 0x75bc15 (0x16e424 bytes) 000216[1] 1736263577.509628 write 0x781f28 thru 0x782f91 (0x106a bytes) 000218[0] 1736263577.527571 mapwrite 0x8c63a6 thru 0x8c6460 (0xbb bytes) 000219[0] 1736263577.600295 read 0x71dbf3 thru 0x71f804 (0x1c12 bytes) 000220[0] 1736263577.604488 read 0x2d653a thru 0x2d7d2f (0x17f6 bytes) 000221[0] 1736263577.621950 mapread 0x505d41 thru 0x506476 (0x736 bytes) 000222[1] 1736263577.628017 mapread 0x68688b thru 0x688693 (0x1e09 bytes) 000224[1] 1736263577.659441 mapread 0x75627b thru 0x756b72 (0x8f8 bytes) 000226[1] 1736263577.679519 mapwrite 0x868c0e thru 0x86a966 (0x1d59 bytes) 000228[1] 1736263577.719731 read 0x063643 thru 0x064897 (0x1255 bytes) 000229[1] 1736263577.761908 mapwrite 0x0d0123 thru 0x0d01e1 (0xbf bytes) 000230[1] 1736263577.794044 mapread 0x5309bc thru 0x5318da (0xf1f bytes) 000231[0] 1736263577.799562 trunc from 0x8c6461 to 0x6e5f4d (0x1e0513 bytes) 000232[0] 1736263577.836800 mapread 0x0636b4 thru 0x064c40 (0x158d bytes) 000233[0] 1736263577.849595 mapwrite 0x4a01b5 thru 0x4a14fc (0x1348 bytes) 000234[1] 1736263577.875456 read 0x2be481 thru 0x2bfaab (0x162b bytes) 000236[0] 1736263577.890712 mapread 0x099dbc thru 0x09b8dd (0x1b22 bytes) 000240[0] 1736263577.924396 mapwrite 0x4bceed thru 0x4be13f (0x1253 bytes) 000241[1] 1736263577.963836 mapread 0x0d3216 thru 0x0d49a6 (0x1791 bytes) 000242[1] 1736263577.988445 trunc from 0x6e5f4e to 0x97a711 (0x2947c4 bytes) 000243[0] 1736263578.001472 trunc from 0x97a712 to 0x03774d (0x942fc4 bytes) 000245[1] 1736263578.020295 trunc from 0x03774e to 0x48ede4 (0x457697 bytes) 000246[0] 1736263578.051113 trunc from 0x48ede5 to 0x808550 (0x37976c bytes) 000247[0] 1736263578.068505 mapread 0x7294ac thru 0x72a569 (0x10be bytes) 000249[0] 1736263578.085008 write 0x616dd1 thru 0x618242 (0x1472 bytes) 000250[1] 1736263578.088307 mapread 0x5eac67 thru 0x5eaeb1 (0x24b bytes) 000252[0] 1736263578.116069 trunc from 0x808551 to 0x11cfa2 (0x6eb5ae bytes) 000253[0] 1736263578.129388 write 0x5eddee thru 0x5ee4dc (0x6ef bytes) 000254[1] 1736263578.130817 mapread 0x32a756 thru 0x32c02d (0x18d8 bytes) 000255[1] 1736263578.159857 mapread 0x4721c4 thru 0x473233 (0x1070 bytes) 000256[0] 1736263578.181529 trunc from 0x5ee4dd to 0x97da8a (0x38f5ae bytes) 000257[1] 1736263578.200711 mapread 0x4e66cc thru 0x4e6774 (0xa9 bytes) 000258[1] 1736263578.211622 mapread 0x8fb51b thru 0x8fd34d (0x1e33 bytes) 000259[1] 1736263578.223304 read 0x26659e thru 0x267eac (0x190f bytes) 000261[1] 1736263578.231001 read 0x2f5d22 thru 0x2f6d27 (0x1006 bytes) 000262[1] 1736263578.231436 write 0x15f2f0 thru 0x16099d (0x16ae bytes) 000265[1] 1736263578.248249 read 0x0a812e thru 0x0a96ed (0x15c0 bytes) 000266[0] 1736263578.296469 read 0x654285 thru 0x6547f0 (0x56c bytes) 000270[1] 1736263578.315756 trunc from 0x97da8b to 0x52d815 (0x450275 bytes) 000272[1] 1736263578.346010 trunc from 0x52d816 to 0x86e0d2 (0x3408bd bytes) 000273[1] 1736263578.355294 mapwrite 0x125608 thru 0x125c3f (0x638 bytes) 000278[0] 1736263578.400959 trunc from 0x86e0d3 to 0x98bf26 (0x11de54 bytes) 000279[0] 1736263578.428932 mapwrite 0x013df0 thru 0x014b0f (0xd20 bytes) 000280[0] 1736263578.489413 mapwrite 0x2fe8a4 thru 0x3003c1 (0x1b1e bytes) 000281[0] 1736263578.547090 mapwrite 0x00af79 thru 0x00bc7a (0xd02 bytes) 000283[1] 1736263578.587682 mapread 0x974061 thru 0x9754e7 (0x1487 bytes) 000284[0] 1736263578.606890 mapread 0x1074ea thru 0x10757f (0x96 bytes) 000285[0] 1736263578.616769 trunc from 0x98bf27 to 0x02d17a (0x95edac bytes) 000287[1] 1736263578.629500 mapread 0x00ecd9 thru 0x010533 (0x185b bytes) 000291[1] 1736263578.664586 mapwrite 0x0306bd thru 0x030fda (0x91e bytes) 000292[1] 1736263578.707868 mapread 0x018c77 thru 0x019090 (0x41a bytes) 000294[1] 1736263578.725742 mapread 0x02de42 thru 0x02e0f4 (0x2b3 bytes) 000296[1] 1736263578.746092 write 0x6366c9 thru 0x636c1d (0x555 bytes) 000299[0] 1736263578.753364 read 0x2dd053 thru 0x2dee55 (0x1e03 bytes) 000302[1] 1736263578.772614 trunc from 0x636c1e to 0x0cc336 (0x56a8e7 bytes) 000303[1] 1736263578.790396 mapwrite 0x5bca77 thru 0x5bd0f6 (0x680 bytes) 000306[0] 1736263578.844193 mapwrite 0x563852 thru 0x563b01 (0x2b0 bytes) 000308[0] 1736263578.909638 read 0x4d1a1a thru 0x4d32ab (0x1892 bytes) 000309[0] 1736263578.926202 read 0x3283d0 thru 0x3294fc (0x112d bytes) 000310[1] 1736263578.937959 mapread 0x05c601 thru 0x05d0e6 (0xae6 bytes) 000312[1] 1736263578.987756 trunc from 0x5bd0f7 to 0x6a1a70 (0xe497a bytes) 000315[0] 1736263579.039126 write 0x1d538f thru 0x1d5823 (0x495 bytes) 000317[1] 1736263579.061411 write 0x7ae38e thru 0x7ae692 (0x305 bytes) 000318[0] 1736263579.077409 mapwrite 0x510a1d thru 0x511a64 (0x1048 bytes) 000320[0] 1736263579.147593 mapread 0x787783 thru 0x787b77 (0x3f5 bytes) 000321[1] 1736263579.165429 trunc from 0x7ae693 to 0x3abaff (0x402b93 bytes) 000323[1] 1736263579.190413 mapwrite 0x07e4bc thru 0x080128 (0x1c6d bytes) 000324[0] 1736263579.222483 write 0x56b0be thru 0x56c018 (0xf5b bytes) 000325[0] 1736263579.234793 trunc from 0x56c019 to 0x8b759a (0x34b582 bytes) 000328[0] 1736263579.254763 trunc from 0x8b759b to 0x57b0f5 (0x33c4a5 bytes) 000329[1] 1736263579.261754 read 0x34f762 thru 0x34ffc4 (0x863 bytes) 000331[0] 1736263579.276459 trunc from 0x57b0f6 to 0x62f39c (0xb42a7 bytes) 000332[0] 1736263579.301561 write 0x982856 thru 0x984499 (0x1c44 bytes) 000336[0] 1736263579.326863 write 0x07c667 thru 0x07e629 (0x1fc3 bytes) 000337[0] 1736263579.339129 mapread 0x68844e thru 0x689245 (0xdf8 bytes) 000338[1] 1736263579.344730 write 0x0eced1 thru 0x0eceeb (0x1b bytes) 000340[1] 1736263579.367361 trunc from 0x98449a to 0x7d86a8 (0x1abdf1 bytes) 000341[0] 1736263579.387927 mapread 0x1bc8b6 thru 0x1be827 (0x1f72 bytes) 000344[1] 1736263579.428340 mapread 0x2b153a thru 0x2b2647 (0x110e bytes) 000345[0] 1736263579.445452 mapread 0x430552 thru 0x430675 (0x124 bytes) 000347[1] 1736263579.460207 mapwrite 0x813199 thru 0x813ea4 (0xd0c bytes) 000348[1] 1736263579.517182 mapread 0x7b4714 thru 0x7b4f23 (0x810 bytes) 000349[1] 1736263579.521929 mapread 0x09c8f6 thru 0x09d1fa (0x905 bytes) 000351[1] 1736263579.537348 write 0x8cec2c thru 0x8cf0a4 (0x479 bytes) 000353[0] 1736263579.548829 write 0x94b8ff thru 0x94c3bb (0xabd bytes) 000354[1] 1736263579.555978 trunc from 0x94c3bc to 0x7498aa (0x202b11 bytes) 000358[0] 1736263579.587645 mapread 0x58ad97 thru 0x58b1b8 (0x422 bytes) 000359[1] 1736263579.605327 mapwrite 0x81ef12 thru 0x81f13a (0x229 bytes) 000360[1] 1736263579.649450 mapwrite 0x3b0f9c thru 0x3b2260 (0x12c5 bytes) 000364[1] 1736263579.704289 mapwrite 0x420014 thru 0x4210ea (0x10d7 bytes) 000369[1] 1736263579.743741 write 0x891a2d thru 0x89272c (0xd00 bytes) 000371[1] 1736263579.754745 write 0x7d5e77 thru 0x7d673e (0x8c8 bytes) 000374[0] 1736263579.782000 trunc from 0x89272d to 0x3b73d5 (0x4db357 bytes) 000375[1] 1736263579.811231 trunc from 0x3b73d6 to 0x66ed7b (0x2b79a6 bytes) 000376[1] 1736263579.821315 trunc from 0x66ed7c to 0x0e80f3 (0x586c88 bytes) 000380[1] 1736263579.840194 write 0x5438f9 thru 0x544515 (0xc1d bytes) 000382[1] 1736263579.853587 read 0x1e7a8b thru 0x1e8c80 (0x11f6 bytes) 000383[1] 1736263579.873276 read 0x44389a thru 0x4445ba (0xd21 bytes) 000385[0] 1736263579.874850 write 0x3345b0 thru 0x335730 (0x1181 bytes) 000387[1] 1736263579.901343 write 0x89ba75 thru 0x89d77d (0x1d09 bytes) 000388[1] 1736263579.908565 read 0x1824bb thru 0x183b7c (0x16c2 bytes) 000391[1] 1736263579.951608 mapwrite 0x5cf5a6 thru 0x5d07af (0x120a bytes) 000393[1] 1736263580.015058 read 0x0bc888 thru 0x0be83d (0x1fb6 bytes) 000394[1] 1736263580.026872 mapwrite 0x72cd21 thru 0x72e1e7 (0x14c7 bytes) 000395[1] 1736263580.060279 write 0x76f3f8 thru 0x7708f9 (0x1502 bytes) 000396[0] 1736263580.060903 trunc from 0x89d77e to 0x51c743 (0x38103a bytes) 000397[1] 1736263580.095661 write 0x1e34ef thru 0x1e5046 (0x1b58 bytes) 000398[0] 1736263580.111999 mapread 0x4a327a thru 0x4a33c9 (0x150 bytes) 000399[1] 1736263580.124228 write 0x7ab55d thru 0x7abb32 (0x5d6 bytes) 000402[1] 1736263580.153990 read 0x01fefc thru 0x021138 (0x123d bytes) 000403[1] 1736263580.207904 mapread 0x2361b4 thru 0x236bc1 (0xa0e bytes) 000404[0] 1736263580.218114 write 0x6b9128 thru 0x6bad14 (0x1bed bytes) 000408[0] 1736263580.264946 mapwrite 0x82297f thru 0x822a8e (0x110 bytes) 000409[1] 1736263580.323239 trunc from 0x822a8f to 0x69368e (0x18f400 bytes) 000411[0] 1736263580.353285 write 0x580d0c thru 0x581d8f (0x1084 bytes) 000412[1] 1736263580.370053 mapread 0x1ca8fb thru 0x1cc584 (0x1c8a bytes) 000414[0] 1736263580.404463 trunc from 0x69368f to 0x6128c5 (0x80dc9 bytes) 000417[1] 1736263580.450209 mapread 0x41cce5 thru 0x41e978 (0x1c94 bytes) 000418[1] 1736263580.469494 read 0x3c6e2d thru 0x3c799d (0xb71 bytes) 000419[0] 1736263580.496823 read 0x0c9dca thru 0x0cacd9 (0xf10 bytes) 000420[1] 1736263580.525317 read 0x58b06a thru 0x58bcb7 (0xc4e bytes) 000422[1] 1736263580.525821 write 0x692613 thru 0x6945a8 (0x1f96 bytes) 000423[1] 1736263580.544363 mapwrite 0x2983a7 thru 0x2988fa (0x554 bytes) 000425[1] 1736263580.582823 write 0x1c682c thru 0x1c85b0 (0x1d85 bytes) 000426[0] 1736263580.601951 write 0x503bd5 thru 0x505495 (0x18c1 bytes) 000429[1] 1736263580.637956 read 0x1d483f thru 0x1d5414 (0xbd6 bytes) 000430[0] 1736263580.664607 read 0x1b9a48 thru 0x1bb613 (0x1bcc bytes) 000431[1] 1736263580.681148 write 0x406ce4 thru 0x4089f6 (0x1d13 bytes) 000432[0] 1736263580.681679 trunc from 0x6945a9 to 0x36766c (0x32cf3c bytes) 000434[1] 1736263580.714703 read 0x070b37 thru 0x071f05 (0x13cf bytes) 000435[1] 1736263580.735429 mapread 0x19442c thru 0x196341 (0x1f16 bytes) 000437[0] 1736263580.761710 mapread 0x16430d thru 0x1655d3 (0x12c7 bytes) 000439[1] 1736263580.775122 write 0x13e450 thru 0x13ebc6 (0x777 bytes) 000440[1] 1736263580.789241 trunc from 0x36766d to 0x89a448 (0x532ddc bytes) 000441[0] 1736263580.815449 read 0x86d2da thru 0x86d894 (0x5bb bytes) 000442[0] 1736263580.832467 trunc from 0x89a449 to 0x1002de (0x79a16a bytes) 000443[1] 1736263580.857333 mapread 0x0a01d7 thru 0x0a1ca6 (0x1ad0 bytes) 000444[1] 1736263580.882758 write 0x96314b thru 0x9646ba (0x1570 bytes) 000448[0] 1736263580.904905 mapwrite 0x23b174 thru 0x23bcfb (0xb88 bytes) 000449[0] 1736263580.956643 write 0x8339a5 thru 0x833d3d (0x399 bytes) 000451[1] 1736263580.972829 mapread 0x07624f thru 0x0772f8 (0x10aa bytes) 000452[0] 1736263580.991935 read 0x3ef649 thru 0x3f1212 (0x1bca bytes) 000453[0] 1736263581.030834 mapread 0x168c89 thru 0x169d07 (0x107f bytes) 000455[0] 1736263581.038881 read 0x3ba0f6 thru 0x3ba3b3 (0x2be bytes) 000456[1] 1736263581.046587 mapwrite 0x433cc5 thru 0x435212 (0x154e bytes) 000457[1] 1736263581.096604 mapread 0x0ca5f9 thru 0x0cb19b (0xba3 bytes) 000458[1] 1736263581.101867 write 0x5eda48 thru 0x5edd9e (0x357 bytes) 000461[1] 1736263581.119197 read 0x012a03 thru 0x01454a (0x1b48 bytes) 000462[0] 1736263581.160653 read 0x4f429d thru 0x4f5899 (0x15fd bytes) 000468[1] 1736263581.198282 trunc from 0x9646bb to 0x2c0088 (0x6a4632 bytes) 000470[0] 1736263581.214170 write 0x277fc4 thru 0x278198 (0x1d5 bytes) 000471[0] 1736263581.235232 trunc from 0x2c0089 to 0x668124 (0x3a809c bytes) 000472[0] 1736263581.256252 mapwrite 0x297afe thru 0x299394 (0x1897 bytes) 000473[0] 1736263581.294503 mapread 0x6574af thru 0x6580b9 (0xc0b bytes) 000475[0] 1736263581.311986 mapwrite 0x7e35bf thru 0x7e429b (0xcdd bytes) 000476[1] 1736263581.348842 write 0x05f470 thru 0x060dd4 (0x1965 bytes) 000477[0] 1736263581.374446 mapread 0x04c632 thru 0x04c950 (0x31f bytes) 000479[0] 1736263581.393783 mapwrite 0x0d436b thru 0x0d494d (0x5e3 bytes) 000480[0] 1736263581.432163 read 0x1ed354 thru 0x1ed4cc (0x179 bytes) 000483[0] 1736263581.464374 write 0x200f1c thru 0x2029dd (0x1ac2 bytes) 000485[1] 1736263581.465073 trunc from 0x7e429c to 0x2a12ba (0x542fe1 bytes) 000486[1] 1736263581.504167 write 0x11a4e7 thru 0x11b1ec (0xd06 bytes) 000488[0] 1736263581.520186 read 0x0921ed thru 0x092fb1 (0xdc5 bytes) 000489[0] 1736263581.534861 mapwrite 0x02aa7d thru 0x02b4b3 (0xa37 bytes) 000490[0] 1736263581.577285 mapread 0x079d9a thru 0x07b656 (0x18bd bytes) 000491[0] 1736263581.591620 write 0x3d1c66 thru 0x3d3142 (0x14dd bytes) 000492[0] 1736263581.601473 trunc from 0x3d3143 to 0x90bd6d (0x538c2b bytes) 000493[0] 1736263581.618577 write 0x0a3989 thru 0x0a4f32 (0x15aa bytes) 000494[1] 1736263581.659573 trunc from 0x90bd6e to 0x880430 (0x8b93d bytes) 000495[1] 1736263581.696175 mapread 0x64148f thru 0x641b79 (0x6eb bytes) 000500[1] 1736263581.719296 mapwrite 0x6dfb8d thru 0x6e0bbd (0x1031 bytes) All operations completed A-OK! PASS 16b (10s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 10:26:23 (1736263583) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 10:26:24 (1736263584) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.142758 s, 14.7 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.110787 s, 18.9 MB/s ldlm.namespaces.MGC192.168.203.107@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aa5a6800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b5944800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800aa5a6800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b5944800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800aa5a6800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b5944800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.83545 s, 21.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.15775 s, 90.6 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.777366 s, 135 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.9067 s, 55.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.806 s, 37.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.919183 s, 114 MB/s PASS 16d (16s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 10:26:40 (1736263600) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0761964 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0542226 s, 19.3 MB/s PASS 16e (3s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 10:26:43 (1736263603) TIMEOUT PASS 16f (22s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 10:27:05 (1736263625) TIMEOUT PASS 16g (21s) == sanityn test 16i: read after truncate file ============ 10:27:26 (1736263646) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (2s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 10:27:28 (1736263648) fail_loc=0x8000030a PASS 17 (3s) == sanityn test 18: mmap sanity check =========================================================================================== 10:27:32 (1736263652) mmap test1: basic mmap operation (PASS, 0.018026s) mmap test2: MAP_PRIVATE not write back (PASS, 0.011715s) mmap test3: concurrent mmap ops on two nodes (PASS, 4.3373s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.0906s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.032542s) mmap test6: check mmap write/read content on two nodes (PASS, 0.041139s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0178s) PASS 18 (10s) == sanityn test 19: test concurrent uncached read races ========================================================================= 10:27:42 (1736263662) oleg307-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg307-client: oleg307-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 10:27:43 (1736263663) PASS 20 (1s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 10:27:44 (1736263664) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) == sanityn test 23: others should see updated atime while another read============================================================== 10:27:46 (1736263666) now is 1736263666 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6878 new atime is 1736263727 PASS 23 (63s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 10:28:49 (1736263729) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 10240 3759104 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 16384 7522304 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 10240 3759104 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 16384 7522304 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.1K 376 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.0K 369 229.7K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.1K 376 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.0K 369 229.7K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 10.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 16.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117912 376 117536 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235569 369 235200 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 10240 3759104 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 16384 7522304 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.1K 376 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.0K 369 229.7K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 117912 376 117536 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre[OST:1] filesystem_summary: 235569 369 235200 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117912 376 117536 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235569 369 235200 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 10240 3759104 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 16384 7522304 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 10240 3759104 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 16384 7522304 1% /mnt/lustre2 PASS 24a (1s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 10:28:50 (1736263730) PASS 24b (2s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 10:28:52 (1736263732) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (1s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 10:28:53 (1736263733) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 10:28:54 (1736263734) utime: good mknod times 1736263733 <= 1736263734 <= 1736263734 for /mnt/lustre/f26a utime: good mknod times 1736263733 <= 1736263734 <= 1736263734 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (3s) == sanityn test 26b: sync mtime between ost and mds ====== 10:28:57 (1736263737) PASS 26b (3s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 10:29:01 (1736263741) 0 PASS 30 (4s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 10:29:05 (1736263745) fail_loc=0x314 PASS 31a (2s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 10:29:07 (1736263747) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg307-server mds-ost sync done. sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.203.107@tcp:/lustre 7542784 17408 7521280 1% /mnt/lustre2 PASS 31b (20s) == sanityn test 31r: open-rename(replace) race =========== 10:29:27 (1736263767) -rw-r--r-- 1 root root 0 Jan 7 10:29 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (4s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 10:29:32 (1736263772) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (0s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 10:29:32 (1736263772) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 10:29:33 (1736263773) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 10:29:34 (1736263774) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 10:29:35 (1736263775) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg307-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa5a6800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b5944800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800aa5a6800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8800b5944800.ost_server_uuid in FULL state after 0 sec oleg307-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa5a6800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b5944800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800aa5a6800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800b5944800.ost_server_uuid in FULL state after 0 sec oleg307-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa5a6800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b5944800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800aa5a6800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8800b5944800.ost_server_uuid in IDLE state after 0 sec oleg307-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa5a6800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b5944800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800aa5a6800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800b5944800.ost_server_uuid in IDLE state after 0 sec oleg307-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa5a6800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b5944800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800aa5a6800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8800b5944800.ost_server_uuid in IDLE state after 0 sec oleg307-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa5a6800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b5944800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800aa5a6800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800b5944800.ost_server_uuid in IDLE state after 0 sec PASS 34 (71s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 10:30:46 (1736263846) fail_loc=0x80000317 Race attempt 0 Wait for 15639 15779 for 60 sec... - open/close 3365 (time 1736263858.64 total 10.00 last 336.47) total: 4000 open/close in 11.87 seconds: 336.94 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.203.107@tcp:/lustre 7.2G 18M 7.2G 1% /mnt/lustre 192.168.203.107@tcp:/lustre 7.2G 18M 7.2G 1% /mnt/lustre2 PASS 35 (64s) == sanityn test 36: handle ESTALE/open-unlink correctly == 10:31:50 (1736263910) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg307-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.314173 s, 167 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6878 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7520256) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.32311 s, 162 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6878 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7520256) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.307746 s, 170 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6878 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7520256) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.310892 s, 169 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6878 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7520256) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.303491 s, 173 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6878 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7520256) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.272182 s, 193 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6878 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7520256) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.274948 s, 191 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6878 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7520256) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.274355 s, 191 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6878 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7520256) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.271379 s, 193 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6878 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7520256) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.30479 s, 172 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6878 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7520256) after_dd(7469056) after(7520256) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.28621 s, 183 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6878 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7520256) after_dd(7469056) after(7520256) PASS 36 (226s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 10:35:36 (1736264136) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6878 total: 10000 create in 8.87 seconds: 1126.77 ops/second PASS 37 (11s) == sanityn test 39a: test from 11063 ============================================================================================ 10:35:47 (1736264147) repeat after cancel_lru_locks PASS 39a (2s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 10:35:49 (1736264149) repeat after cancel_lru_locks PASS 39b (2s) == sanityn test 39c: check truncate mtime update ================================================================================ 10:35:51 (1736264151) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 10:35:54 (1736264154) fail_loc=0x411 fail_loc=0 PASS 39d (1s) == sanityn test 40a: pdirops: create vs others ======================================================================== 10:35:55 (1736264155) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (7s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 10:36:02 (1736264162) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (7s) == sanityn test 40c: pdirops: link and others ======================================================================== 10:36:09 (1736264169) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (7s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 10:36:16 (1736264176) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (6s) == sanityn test 40e: pdirops: rename and others ======================================================================== 10:36:22 (1736264182) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (7s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 10:36:29 (1736264189) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (4s) == sanityn test 41b: pdirops: create vs create ======================================================================== 10:36:33 (1736264193) open: File exists Conflict PASS 41b (4s) == sanityn test 41c: pdirops: create vs link ======================================================================== 10:36:37 (1736264197) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (4s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 10:36:41 (1736264201) Conflict PASS 41d (4s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 10:36:45 (1736264205) Conflict PASS 41e (4s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 10:36:49 (1736264209) Conflict PASS 41f (5s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 10:36:54 (1736264214) Conflict PASS 41g (4s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 10:36:58 (1736264218) Conflict PASS 41h (4s) == sanityn test 41i: reint_open: create vs create ======== 10:37:02 (1736264222) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (647s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 10:47:49 (1736264869) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (4s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 10:47:53 (1736264873) open: File exists Conflict PASS 42b (4s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 10:47:57 (1736264877) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (5s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 10:48:02 (1736264882) Conflict PASS 42d (4s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 10:48:06 (1736264886) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (4s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 10:48:10 (1736264890) Conflict PASS 42f (5s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 10:48:15 (1736264895) Conflict PASS 42g (4s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 10:48:19 (1736264899) Conflict PASS 42h (4s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 10:48:23 (1736264903) PASS 43a (10s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 10:48:33 (1736264913) Conflict PASS 43b (4s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 10:48:37 (1736264917) Conflict PASS 43c (4s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 10:48:41 (1736264921) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (4s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 10:48:45 (1736264925) Conflict PASS 43e (4s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 10:48:49 (1736264929) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (5s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 10:48:54 (1736264934) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (4s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 10:48:58 (1736264938) Conflict PASS 43h (4s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 10:49:02 (1736264942) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 10:49:03 (1736264943) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (32s) == sanityn test 43k: unlink vs create ==================== 10:49:35 (1736264975) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (353s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 10:55:28 (1736265328) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Tue Jan 7 10:55:31 EST 2025 PASS 44a (4s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 10:55:32 (1736265332) open: File exists Conflict PASS 44b (4s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 10:55:36 (1736265336) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (4s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 10:55:40 (1736265340) Conflict PASS 44d (4s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 10:55:44 (1736265344) Conflict PASS 44e (5s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 10:55:49 (1736265349) Conflict PASS 44f (4s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 10:55:53 (1736265353) Conflict PASS 44g (4s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 10:55:57 (1736265357) Conflict PASS 44h (4s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 10:56:01 (1736265361) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 10:56:02 (1736265362) PASS 45a (18s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 10:56:20 (1736265380) Conflict PASS 45b (4s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 10:56:24 (1736265384) Conflict PASS 45c (5s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 10:56:29 (1736265389) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (4s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 10:56:33 (1736265393) Conflict PASS 45e (4s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 10:56:37 (1736265397) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (4s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 10:56:41 (1736265401) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (5s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 10:56:46 (1736265406) Conflict PASS 45h (3s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 10:56:49 (1736265409) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 10:56:50 (1736265410) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (362s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 11:02:52 (1736265772) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (4s) == sanityn test 46b: pdirops: link vs create ======================================================================== 11:02:56 (1736265776) open: File exists Conflict PASS 46b (4s) == sanityn test 46c: pdirops: link vs link ======================================================================== 11:03:00 (1736265780) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (4s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 11:03:04 (1736265784) Conflict PASS 46d (4s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 11:03:08 (1736265788) Conflict PASS 46e (5s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 11:03:13 (1736265793) Conflict PASS 46f (4s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 11:03:17 (1736265797) Conflict PASS 46g (4s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 11:03:21 (1736265801) Conflict PASS 46h (4s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 11:03:25 (1736265805) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 11:03:26 (1736265806) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 11:03:27 (1736265807) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 11:03:28 (1736265808) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 11:03:29 (1736265809) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (0s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 11:03:29 (1736265809) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 11:03:30 (1736265810) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 11:03:31 (1736265811) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 11:03:32 (1736265812) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.0039621 s, 2.6 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (6s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 11:03:38 (1736265818) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00332869 s, 47.8 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (24526) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 11:03:42 (1736265822) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00205834 s, 497 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128308 s, 81.7 MB/s 3145728 PASS 51b (13s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 11:03:55 (1736265835) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00326216 s, 314 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 11:04:01 (1736265841) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104799 s, 100 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (3s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 11:04:04 (1736265844) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2432 0x980 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2432 0x980 0 read lease(1) has applied. PASS 51e (3s) == sanityn test 54: rename locking ======================= 11:04:07 (1736265847) llite.lustre-ffff8800aa5a6800.xattr_cache=1 llite.lustre-ffff8800b5944800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504963 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 11:04:08.000000000 -0500 Modify: 2025-01-07 11:04:08.000000000 -0500 Change: 2025-01-07 11:04:08.000000000 -0500 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504966 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 11:04:13.000000000 -0500 Modify: 2025-01-07 11:04:13.000000000 -0500 Change: 2025-01-07 11:04:13.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504969 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 11:04:19.000000000 -0500 Modify: 2025-01-07 11:04:19.000000000 -0500 Change: 2025-01-07 11:04:19.000000000 -0500 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 11:04:19.000000000 -0500 Modify: 2025-01-07 11:04:19.000000000 -0500 Change: 2025-01-07 11:04:19.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 11:04:24.000000000 -0500 Modify: 2025-01-07 11:04:24.000000000 -0500 Change: 2025-01-07 11:04:24.000000000 -0500 Birth: - File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504972 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 11:04:24.000000000 -0500 Modify: 2025-01-07 11:04:24.000000000 -0500 Change: 2025-01-07 11:04:24.000000000 -0500 Birth: - PASS 54 (23s) == sanityn test 55a: rename vs unlink target dir ========= 11:04:31 (1736265871) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (6s) == sanityn test 55b: rename vs unlink source dir ========= 11:04:37 (1736265877) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 11:04:44 (1736265884) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 11:04:56 (1736265896) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) == sanityn test 60: Verify data_version behaviour ======== 11:05:05 (1736265905) PASS 60 (2s) == sanityn test 70a: cd directory && rm directory ======== 11:05:07 (1736265907) PASS 70a (1s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 11:05:08 (1736265908) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 11:05:10 (1736265910) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00242008 s, 27.1 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 11:05:11 (1736265911) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.0032132 s, 20.4 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (0s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 11:05:11 (1736265911) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 11:05:12 (1736265912) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00305018 s, 21.5 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 11:05:13 (1736265913) llite.lustre-ffff8800aa5a6800.xattr_cache=1 llite.lustre-ffff8800b5944800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (1s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 11:05:14 (1736265914) llite.lustre-ffff8800aa5a6800.xattr_cache=1 llite.lustre-ffff8800b5944800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (2s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 11:05:16 (1736265916) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00209899 s, 488 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0044858 s, 228 kB/s 3178: get lock2 3179: get lock1 3179: done 3179: get lock2 3178: done 3178: get lock1 fcntl cmd 7 failed: Resource deadlock avoided 3178: cannot get lock1: Resource deadlock avoided fcntl cmd 7 failed: Resource deadlock avoided 3179: cannot get lock2: Resource deadlock avoided 3179: exit rc=0 3178: done 3178: exit rc=0 PASS 74 (5s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 11:05:21 (1736265921) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0160947 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00911183 s, 115 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 3813 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 11:05:28 (1736265928) opening files: ........ closing files: ........ PASS 76 (13s) == sanityn test 77a: check FIFO NRS policy =============== 11:05:41 (1736265941) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0984675 s, 170 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111293 s, 94.2 MB/s 1048576 bytes (1.0 MB) copied, 0.00995805 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232597 s, 45.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237525 s, 44.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251364 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0224833 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162957 s, 64.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378303 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300068 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0358443 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245471 s, 42.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316695 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0349462 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272336 s, 38.5 MB/s , 0.0203781 s, 51.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127302 s, 82.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00836951 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00962505 s, 109 MB/s , 0.0076022 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00172877 s, 607 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00441371 s, 238 MB/s , 0.0125968 s, 83.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108856 s, 96.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102927 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100222 s, 105 MB/s , 0.00943202 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117939 s, 88.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.011151 s, 94.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0113695 s, 92.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0120408 s, 87.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123449 s, 84.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00239291 s, 438 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b5944800.lock_unused_count=1 removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) == sanityn test 77b: check CRR-N NRS policy ============== 11:05:43 (1736265943) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0887483 s, 189 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00931475 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118511 s, 88.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0244165 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196601 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178147 s, 58.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283869 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260761 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338743 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175288 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251144 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.035408 s, 29.6 MB/s , 0.0314261 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182474 s, 57.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225874 s, 46.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026599 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322815 s, 32.5 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b5944800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00885562 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00190714 s, 550 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00744929 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00651559 s, 161 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b5944800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00469628 s, 223 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0069909 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00153001 s, 685 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00632321 s, 166 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00970415 s, 108 MB/s 1048576 bytes (1.0 MB) copied, 0.00942828 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101386 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0103243 s, 102 MB/s , 0.0104434 s, 100 MB/s 1048576 bytes (1.0 MB) copied, 0.0114146 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115379 s, 90.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125251 s, 83.7 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.105954 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00768641 s, 136 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101524 s, 103 MB/s 1048576 bytes (1.0 MB) copied, 0.0166966 s, 62.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146803 s, 71.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192131 s, 54.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247246 s, 42.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0232338 s, 45.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180674 s, 58.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023068 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198607 s, 52.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0342652 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.024387 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213694 s, 49.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196874 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191421 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233021 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00952213 s, 110 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0109781 s, 95.5 MB/s 1048576 bytes (1.0 MB) copied, 0.00804443 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00902804 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00500576 s, 209 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010094 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0091221 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00906225 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116087 s, 90.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00619471 s, 169 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00743665 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00964655 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103248 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104263 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144859 s, 72.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125845 s, 83.3 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b5944800.lock_unused_count=1 removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) == sanityn test 77c: check ORR NRS policy ================ 11:05:46 (1736265946) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0942123 s, 178 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111254 s, 94.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117252 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318988 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02883 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372536 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343595 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449895 s, 23.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b5944800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff8800b5944800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0336122 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0437836 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0507328 s, 20.7 MB/s , 0.0457395 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0462798 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0529138 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0568049 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0476218 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.065427 s, 16.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b5944800.lock_unused_count=3 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00888304 s, 118 MB/s , 0.0107273 s, 97.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131994 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162359 s, 64.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126255 s, 83.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108834 s, 96.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130306 s, 80.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143917 s, 72.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153417 s, 68.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169551 s, 61.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133497 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170283 s, 61.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0158105 s, 66.3 MB/s , 0.0157446 s, 66.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165913 s, 63.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167198 s, 62.7 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.131199 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00897443 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171766 s, 61.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0069639 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02185 s, 48.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249434 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0299531 s, 35.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215815 s, 48.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0302595 s, 34.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0282169 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0344547 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338057 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449477 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279796 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0385931 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155852 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245732 s, 42.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124211 s, 84.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00524284 s, 200 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109296 s, 95.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00642298 s, 163 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00786428 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00696991 s, 150 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0124208 s, 84.4 MB/s , 0.0147556 s, 71.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00923007 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102519 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103877 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123851 s, 84.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124099 s, 84.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126388 s, 83.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127915 s, 82.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143825 s, 72.9 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b5944800.lock_unused_count=2 removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0961601 s, 174 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121378 s, 86.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149805 s, 70.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0091295 s, 115 MB/s , 0.0174971 s, 59.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00927041 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317127 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0302261 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268053 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0358712 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253183 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174743 s, 60.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193358 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0410716 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0473797 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016831 s, 62.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b5944800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103265 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00866895 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00619195 s, 169 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00913636 s, 115 MB/s , 0.00125268 s, 837 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106101 s, 98.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00911283 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00512463 s, 205 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00782509 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00122433 s, 856 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00134979 s, 777 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00354133 s, 296 MB/s , 0.00739943 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0104034 s, 101 MB/s 1048576 bytes (1.0 MB) copied, 0.00152772 s, 686 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b5944800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123452 s, 84.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00406833 s, 258 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (4s) == sanityn test 77d: check TRR nrs policy ================ 11:05:50 (1736265950) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.114428 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00837799 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188202 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0257339 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161536 s, 64.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0318579 s, 32.9 MB/s , 0.0211918 s, 49.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222179 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250429 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289506 s, 36.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b5944800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247724 s, 42.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.040758 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0368519 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0461462 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331823 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447328 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0421543 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00913654 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00791405 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00829597 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00786042 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00320574 s, 327 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00408869 s, 256 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00811839 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00869893 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00932849 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00945247 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972624 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105697 s, 99.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111569 s, 94.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111156 s, 94.3 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0123501 s, 84.9 MB/s 1048576 bytes (1.0 MB) copiedldlm.namespaces.lustre-OST0000-osc-ffff8800b5944800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b5944800.lock_unused_count=1 , 0.0118345 s, 88.6 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0809376 s, 207 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00970321 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00914637 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147442 s, 71.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102692 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164891 s, 63.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175106 s, 59.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135026 s, 77.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0245456 s, 42.7 MB/s , 0.00981713 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248773 s, 42.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196032 s, 53.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01813 s, 57.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233646 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342387 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259805 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254263 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00666877 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00854256 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00314469 s, 333 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00794024 s, 132 MB/s , 0.00657799 s, 159 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00727152 s, 144 MB/s , 0.0083279 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100019 s, 105 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00946622 s, 111 MB/s 1048576 bytes (1.0 MB) copied, 0.00861103 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00890991 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00933252 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00956751 s, 110 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00996684 s, 105 MB/s 1048576 bytes (1.0 MB) copied, 0.00947792 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00967134 s, 108 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.13459 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00830154 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00839437 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195953 s, 53.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283013 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291296 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272978 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280105 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0244239 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162467 s, 64.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284331 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023953 s, 43.8 MB/s , 0.0125703 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254532 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0311485 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282363 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158139 s, 66.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b5944800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b5944800.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00954455 s, 110 MB/s , 0.00643079 s, 163 MB/s , 0.00634758 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00274978 s, 381 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00811282 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00133105 s, 788 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00163905 s, 640 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0101087 s, 104 MB/s 1048576 bytes (1.0 MB) copied, 0.0103521 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948497 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00861951 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109041 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125086 s, 83.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010877 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00321769 s, 326 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123516 s, 84.9 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b5944800.lock_unused_count=1 removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (3s) == sanityn test 77e: check TBF NID nrs policy ============ 11:05:53 (1736265953) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.203.7@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.142265 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00964661 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00833488 s, 126 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208587 s, 50.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025037 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282801 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276858 s, 37.9 MB/s 1048576 bytes (1.0 MB) copied, 0.018103 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283182 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0325375 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304474 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237459 s, 44.2 MB/s , 0.0213061 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144653 s, 72.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0238432 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0313025 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00557226 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00662308 s, 158 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104425 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00885255 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00971519 s, 108 MB/s 1048576 bytes (1.0 MB) copied, 0.00952069 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00820812 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101845 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00837474 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00912938 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00925606 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101941 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105153 s, 99.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105779 s, 99.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105331 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108753 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115134 s, 91.1 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0965436 s, 174 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00868086 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140489 s, 74.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013424 s, 78.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127119 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226991 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276015 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194399 s, 53.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319249 s, 32.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.026148 s, 40.1 MB/s , 0.0378474 s, 27.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0375675 s, 27.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0445921 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0387515 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0325828 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364952 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0542525 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00689 s, 152 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00592253 s, 177 MB/s , 0.00567556 s, 185 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0062173 s, 169 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111743 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102815 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00772913 s, 136 MB/s 1048576 bytes (1.0 MB) copied, 0.00778901 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00946013 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00923877 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948525 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00978274 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0107274 s, 97.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0102315 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00963955 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0097952 s, 107 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0870201 s, 193 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00804018 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158848 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016689 s, 62.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319569 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288742 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300512 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386069 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0298412 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0434604 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.033357 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343732 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457586 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449659 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353205 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345964 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0500509 s, 21.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b5944800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff8800b5944800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00821635 s, 128 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108084 s, 97.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00846108 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00991405 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0078982 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00413456 s, 254 MB/s 1048576 bytes (1.0 MB) copied, 0.0098331 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00878759 s, 119 MB/s , 0.00870887 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00967725 s, 108 MB/s 1048576 bytes (1.0 MB) copied, 0.0107061 s, 97.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00926014 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109032 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102047 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00960961 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102008 s, 103 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.120298 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118408 s, 88.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146307 s, 71.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194881 s, 53.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187124 s, 56.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227241 s, 46.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0254656 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0347293 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01673 s, 62.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239937 s, 43.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0217272 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388052 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0224961 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326509 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290656 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034184 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219501 s, 47.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b5944800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00590119 s, 178 MB/s 1048576 bytes (1.0 MB) copied, 0.00537789 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00665007 s, 158 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0080746 s, 130 MB/s 1048576 bytes (1.0 MB) copied, 0.00740356 s, 142 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00922933 s, 114 MB/s , 0.010345 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00476296 s, 220 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00581362 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00952058 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00906373 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00960127 s, 109 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00947598 s, 111 MB/s 1048576 bytes (1.0 MB) copied, 0.00984699 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00982749 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113313 s, 92.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (7s) == sanityn test 77f: check TBF JobID nrs policy ========== 11:06:00 (1736265960) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.127446 s, 132 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00764142 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00919 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00781521 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293195 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232545 s, 45.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268694 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245883 s, 42.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249757 s, 42.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335526 s, 31.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0406341 s, 25.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0420387 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331411 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345753 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.042878 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449037 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0355395 s, 29.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103278 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00978452 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00882524 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112906 s, 92.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010897 s, 96.2 MB/s , 0.0128706 s, 81.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102455 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103237 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115264 s, 91.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117853 s, 89.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118722 s, 88.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118947 s, 88.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013199 s, 79.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.013205 s, 79.4 MB/s , 0.0112058 s, 93.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119934 s, 87.4 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.108189 s, 155 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00600146 s, 175 MB/s , 0.00699016 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141913 s, 73.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246969 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037005 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256711 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282481 s, 37.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318106 s, 33.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0322942 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276002 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0351285 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0406682 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0408723 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0478884 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457211 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0419638 s, 25.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00658938 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00666319 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00722812 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00719003 s, 146 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.010205 s, 103 MB/s , 0.00425953 s, 246 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120809 s, 86.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00914949 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101574 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102132 s, 103 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00947648 s, 111 MB/s , 0.0108028 s, 97.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00953922 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00940082 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107188 s, 97.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124028 s, 84.5 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.098876 s, 170 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010794 s, 97.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117352 s, 89.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0199425 s, 52.6 MB/s , 0.0259737 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270223 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208756 s, 50.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327265 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341441 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022902 s, 45.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343006 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027536 s, 38.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0242754 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352422 s, 29.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b5944800.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff8800b5944800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341658 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333418 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0367987 s, 28.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00214735 s, 488 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0078443 s, 134 MB/s , 0.00772794 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102499 s, 102 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114244 s, 91.8 MB/s , 0.00983998 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00937064 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00167838 s, 625 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b5944800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b5944800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104422 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101602 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115355 s, 90.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118037 s, 88.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00941743 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.012406 s, 84.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0123473 s, 84.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131759 s, 79.6 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.114168 s, 147 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112514 s, 93.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124412 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101276 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0344189 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297041 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.039194 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0426604 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0355109 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0415956 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388043 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0415958 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0512679 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0504196 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0515151 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0472981 s, 22.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b5944800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff8800b5944800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0545003 s, 19.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00866235 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00723897 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00600138 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00665897 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00165991 s, 632 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00133005 s, 788 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00909084 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00880369 s, 119 MB/s 1048576 bytes (1.0 MB) copied, 0.0101319 s, 103 MB/s 1048576 bytes (1.0 MB) copiedldlm.namespaces.lustre-OST0001-osc-ffff8800b5944800.lock_unused_count=1 , 0.00274274 s, 382 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112689 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010599 s, 98.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108562 s, 96.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107293 s, 97.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00606589 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00118869 s, 882 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (5s) == sanityn test 77g: Change TBF type directly ============ 11:06:05 (1736265965) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (3s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 11:06:08 (1736265968) oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg307-client: oleg307-server: ssh exited with exit code 2 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg307-client: oleg307-server: ssh exited with exit code 12 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg307-client: oleg307-server: ssh exited with exit code 12 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg307-client: oleg307-server: ssh exited with exit code 12 PASS 77h (3s) == sanityn test 77i: Change rank of TBF rule ============= 11:06:11 (1736265971) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (6s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 11:06:17 (1736265977) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0872059 s, 192 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00737214 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117377 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129753 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196674 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142253 s, 73.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225434 s, 46.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114886 s, 91.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172247 s, 60.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154984 s, 67.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201863 s, 51.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0323534 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231178 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207537 s, 50.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127701 s, 82.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00637305 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100794 s, 104 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112244 s, 93.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113352 s, 92.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211411 s, 49.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120773 s, 86.8 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159417 s, 65.8 MB/s , 0.0208967 s, 50.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127448 s, 82.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0129184 s, 81.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144785 s, 72.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0168151 s, 62.4 MB/s , 0.0147788 s, 71.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147245 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158055 s, 66.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157036 s, 66.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162963 s, 64.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017486 s, 60.0 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94003 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4345 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (51s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 11:07:08 (1736266028) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.649763 s, 25.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.164842 s, 6.4 MB/s , 0.163308 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169944 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.167567 s, 6.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.180182 s, 5.8 MB/s , 0.165745 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.183505 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.172871 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.363216 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.366971 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.376742 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.371149 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.388468 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.38435 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.394965 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.379197 s, 2.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147618 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.144865 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145874 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143504 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14389 s, 7.3 MB/s , 0.146602 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.144658 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146069 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146019 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14647 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147009 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147409 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147288 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14769 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148497 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148863 s, 7.0 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8425 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9396 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.659411 s, 25.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145748 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142069 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147453 s, 7.1 MB/s , 0.13828 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.162941 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148495 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145257 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166093 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.345489 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.359689 s, 2.9 MB/s 1048576 bytes (1.0 MB) copied, 0.365145 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.352151 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.363426 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.381085 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.386049 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.361245 s, 2.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.128128 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1306 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.129084 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.128984 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.130725 s, 8.0 MB/s 1048576 bytes (1.0 MB) copied, 0.12976 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.130325 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.129653 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.132445 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.131209 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.132028 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.135592 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13357 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.134177 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.135986 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13824 s, 7.6 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8455 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.943 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (132s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 11:09:20 (1736266160) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.208832 s, 80.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131051 s, 80.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201659 s, 52.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283321 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290528 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267344 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263091 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274255 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319261 s, 32.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0645704 s, 16.2 MB/s , 0.0627804 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0701086 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0708466 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0738847 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0816911 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0818007 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0924031 s, 11.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0073592 s, 142 MB/s , 0.00665863 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110589 s, 94.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00763859 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00824255 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00846553 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00867357 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00904109 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100651 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100855 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0106956 s, 98.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0106484 s, 98.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113551 s, 92.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109908 s, 95.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114755 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137523 s, 76.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92888 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73624 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.7@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.7@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.146488 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00904642 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100133 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192013 s, 54.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0325899 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0186556 s, 56.2 MB/s , 0.0200363 s, 52.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103018 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194297 s, 54.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0579953 s, 18.1 MB/s , 0.0644478 s, 16.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.149248 s, 7.0 MB/s , 0.15643 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.257525 s, 4.1 MB/s , 0.264606 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.350742 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.358588 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00634566 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00140595 s, 746 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00134666 s, 779 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00140948 s, 744 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00161241 s, 650 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013279 s, 79.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00143354 s, 731 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00994673 s, 105 MB/s , 0.0155696 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117788 s, 89.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00150675 s, 696 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173797 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158677 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0076769 s, 137 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848717 s, 124 MB/s 1048576 bytes (1.0 MB) copied, 0.014718 s, 71.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92995 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.75003 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.7@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.168291 s, 99.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00880852 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200126 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00728285 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247609 s, 42.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273478 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293769 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0206223 s, 50.8 MB/s 1048576 bytes (1.0 MB) copied, 0.007886 s, 133 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0502759 s, 20.9 MB/s , 0.0526609 s, 19.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0587938 s, 17.8 MB/s , 0.0487373 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0600136 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0491456 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0532306 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.065071 s, 16.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102685 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103141 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134249 s, 78.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012769 s, 82.1 MB/s , 0.0103838 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130979 s, 80.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111724 s, 93.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132704 s, 79.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141352 s, 74.2 MB/s , 0.0131938 s, 79.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142029 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144305 s, 72.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131998 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140364 s, 74.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136519 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151269 s, 69.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93918 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.88337 s, 21.5 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.283935 s, 59.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259305 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285594 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261863 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0402033 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392648 s, 26.7 MB/s , 0.0381918 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207266 s, 50.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266129 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.133072 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.130521 s, 8.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.283195 s, 3.7 MB/s , 0.29116 s, 3.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.331787 s, 3.2 MB/s , 0.324661 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.321785 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.334859 s, 3.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00747166 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00804317 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121323 s, 86.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120979 s, 86.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114424 s, 91.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0114863 s, 91.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122191 s, 85.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127559 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117256 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127878 s, 82.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012519 s, 83.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125767 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127305 s, 82.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140024 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147831 s, 70.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140601 s, 74.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.79494 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74335 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96208 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7521 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.465915 s, 36.0 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00822926 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0224913 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180827 s, 58.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166196 s, 63.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00821835 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00678955 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228595 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0059784 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.349294 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.349827 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.353685 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.351102 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.364449 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.359445 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.36742 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.346996 s, 3.0 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137296 s, 76.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123311 s, 85.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00976189 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131505 s, 79.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162825 s, 64.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144522 s, 72.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156438 s, 67.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134265 s, 78.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127988 s, 81.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143821 s, 72.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145746 s, 71.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148868 s, 70.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149985 s, 69.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0170313 s, 61.6 MB/s , 0.0175644 s, 59.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202328 s, 51.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.6741 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4374 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.159134 s, 105 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140366 s, 74.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123483 s, 84.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282847 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0391711 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0374021 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390106 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263627 s, 39.8 MB/s , 0.0312483 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446388 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0617794 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0614846 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0576787 s, 18.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0786424 s, 13.3 MB/s , 0.0636162 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0757211 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0791154 s, 13.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140215 s, 74.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0138297 s, 75.8 MB/s , 0.0147494 s, 71.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174668 s, 60.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0159967 s, 65.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165448 s, 63.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209719 s, 50.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149025 s, 70.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186629 s, 56.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162392 s, 64.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018787 s, 55.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017291 s, 60.6 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171834 s, 61.0 MB/s , 0.0160098 s, 65.5 MB/s , 0.0178236 s, 58.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195487 s, 53.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95077 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74293 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.241558 s, 69.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00765042 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146019 s, 71.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216897 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126457 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200441 s, 52.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149306 s, 70.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022648 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226045 s, 46.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.168583 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.171801 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.182319 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.186014 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.314712 s, 3.3 MB/s , 0.314069 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.36631 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.366618 s, 2.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123153 s, 85.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113692 s, 92.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118179 s, 88.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00934721 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00379603 s, 276 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118106 s, 88.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107785 s, 97.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141307 s, 74.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134407 s, 78.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0129605 s, 80.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0119231 s, 87.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157816 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134861 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133636 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136609 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154141 s, 68.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77631 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73937 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.79666 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73504 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (355s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 11:15:15 (1736266515) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (3s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 11:15:18 (1736266518) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg307-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.44288 s, 305 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg307-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0111 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 11:16:08 (1736266568) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0894338 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00969222 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0127656 s, 82.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0195309 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237458 s, 44.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017818 s, 58.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225597 s, 46.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0257116 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326926 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0348244 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.031671 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251562 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0330951 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322262 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0286966 s, 36.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0218834 s, 47.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241062 s, 43.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b5944800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff8800b5944800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00974359 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00877017 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0013279 s, 790 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109428 s, 95.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00410503 s, 255 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00874784 s, 120 MB/s 1048576 bytes (1.0 MB) copied, 0.00304135 s, 345 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00876137 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00880828 s, 119 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00844618 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109707 s, 95.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122786 s, 85.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115972 s, 90.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0107554 s, 97.5 MB/s 1048576 bytes (1.0 MB) copied, 0.014054 s, 74.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00114021 s, 920 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94349 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97763 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.196595 s, 85.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102134 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172193 s, 60.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308717 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0349472 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233198 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0334323 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280993 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269454 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0596428 s, 17.6 MB/s , 0.0572211 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.065492 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.065502 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.070544 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0808307 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146096 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.16168 s, 6.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168804 s, 62.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126531 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191213 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180381 s, 58.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0160444 s, 65.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0175371 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173508 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164987 s, 63.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102253 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163149 s, 64.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016091 s, 65.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019928 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173488 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184083 s, 57.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160586 s, 65.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234334 s, 44.7 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93602 s, 21.2 MB/s Write runtime is 5 s, speed is 20.000000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97519 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (66s) == sanityn test 77o: Changing rank should not panic ====== 11:17:14 (1736266634) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (3s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 11:17:17 (1736266637) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (25s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 11:17:42 (1736266662) mds.MDS.mdt.nrs_policies=tbf oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg307-client: oleg307-server: ssh exited with exit code 36 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (12s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 11:17:54 (1736266674) oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: Resource temporarily unavailable pdsh@oleg307-client: oleg307-server: ssh exited with exit code 11 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (3s) == sanityn test 79: xattr: intent error ================== 11:17:57 (1736266677) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505431 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 11:17:57.000000000 -0500 Modify: 2025-01-07 11:17:57.000000000 -0500 Change: 2025-01-07 11:17:57.000000000 -0500 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 11:18:09 (1736266689) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 11:18:10 (1736266690) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (0s) == sanityn test 81a: rename and stat under striped directory ========================================================== 11:18:10 (1736266690) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 11:18:11 (1736266691) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 11:18:12 (1736266692) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 11:18:13 (1736266693) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6878 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6878 PASS 82 (2s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 11:18:15 (1736266695) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 11:18:16 (1736266696) fail_loc=0x8000060b PASS 84 (7s) == sanityn test 90: open/create and unlink striped directory ========================================================== 11:18:23 (1736266703) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 11:18:24 (1736266704) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 11:18:25 (1736266705) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (0s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 11:18:26 (1736266706) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2645 0xa55 0 1 2628 0xa44 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2646 0xa56 0 1 2629 0xa45 0 rc1=0 and rc2=0 PASS 93 (8s) == sanityn test 94: signal vs CP callback race =========== 11:18:34 (1736266714) osc.lustre-OST0000-osc-ffff8800aa5a6800.idle_timeout=0 osc.lustre-OST0000-osc-ffff8800b5944800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800aa5a6800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b5944800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0335787 s, 244 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 2.99715 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 4419 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0112411 s, 364 kB/s osc.lustre-OST0000-osc-ffff8800aa5a6800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b5944800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aa5a6800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b5944800.idle_timeout=debug PASS 94 (10s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 11:18:44 (1736266724) llite.lustre-ffff8800aa5a6800.fast_read=0 llite.lustre-ffff8800b5944800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00152097 s, 8.1 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800aa5a6800.fast_read=1 llite.lustre-ffff8800b5944800.fast_read=1 PASS 95b (7s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 11:18:51 (1736266731) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 11:18:52 (1736266732) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00154394 s, 2.7 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 11:18:54 (1736266734) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0211425 s, 99.2 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (1s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 11:18:55 (1736266735) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0190288 s, 110 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 11:18:57 (1736266737) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00144787 s, 8.3 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 11:18:59 (1736266739) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000548625 s, 7.5 MB/s PASS 101a (1s) == sanityn test 101b: Discard DoM data on rename ========= 11:19:00 (1736266740) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000644422 s, 6.4 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 11:19:02 (1736266742) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000500914 s, 8.2 MB/s PASS 101c (3s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 11:19:06 (1736266746) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1736266746 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1736266749 Finishing test test23 at 1736266749 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 11:19:12 (1736266752) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 11:19:13 (1736266753) fail_loc=0x416 PASS 105 (48s) == sanityn test 106a: Verify the btime via statx() ======= 11:20:01 (1736266801) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 11:20:02 (1736266802) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) == sanityn test 106c: Verify statx attributes mask ======= 11:20:03 (1736266803) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) == sanityn test 107a: Basic grouplock conflict =========== 11:20:04 (1736266804) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.084498 s, 124 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2653 0xa5d 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6878 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6878 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 11:20:08 (1736266808) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.069476 s, 151 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2633 0xa49 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6878 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (9s) == sanityn test 108a: lseek: parallel updates ============ 11:20:17 (1736266817) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00155738 s, 6.4 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.2669 s, 2.0 MB/s PASS 108a (6s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 11:20:23 (1736266823) 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 6 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 14 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 19 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 20 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 23 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 24 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 27 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 30 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 37 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 40 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 46 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 PASS 109 (438s) == sanityn test 110: do not grant another lock on resend ========================================================== 11:27:41 (1736267261) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 11:27:42.000000000 -0500 Modify: 2025-01-07 11:27:42.000000000 -0500 Change: 2025-01-07 11:27:42.000000000 -0500 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg307-client.virtnet oleg307-client.virtnet PASS 110 (37s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 11:28:18 (1736267298) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 11:28:19 (1736267299) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 11:28:20 (1736267300) lustre-OST0000-osc-ffff8800ab57a800 active. lustre-OST0001-osc-ffff8800ab57a800 active. lustre-OST0000-osc-ffff8800ab57a800 active. lustre-OST0001-osc-ffff8800ab57a800 active. MGC192.168.203.107@tcp active. PASS 113 (2s) cleanup: ====================================================== == sanityn test complete, duration 3972 sec ============== 11:28:23 (1736267303) Stopping clients: oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 opts: