-----============= acceptance-small: sanityn ============----- Tue Jan 7 10:22:02 EST 2025 excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 Started clients oleg340-client.virtnet: 192.168.203.140@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt) oleg340-client.virtnet: executing check_config_client /mnt/lustre oleg340-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg340-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800b3eee800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b652f000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b3eee800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b652f000.idle_timeout=debug disable quota as required oleg340-server: oleg340-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114482 s, 91.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6871] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 10:22:13 (1736263333) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (1s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 10:22:14 (1736263334) -rw-r--r-- 1 root root 0 Jan 7 10:22 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (2s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 10:22:16 (1736263336) -rw-r--r-- 1 root root 0 Jan 7 10:22 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (2s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 10:22:18 (1736263338) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (3s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 10:22:21 (1736263341) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (1s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 10:22:23 (1736263343) total 1 -rwxrwxrwx 1 root root 0 Jan 7 10:22 f2a -rwxrwxrwx 1 root root 0 Jan 7 10:22 f2b total 1 -rwxrwxrwx 1 root root 0 Jan 7 10:22 f2a -rwxrwxrwx 1 root root 0 Jan 7 10:22 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00829797 s, 61.7 kB/s PASS 2e (2s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 10:22:25 (1736263345) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 10:22:26 (1736263346) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.20484 s, 10.2 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (3s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 10:22:29 (1736263349) PASS 3 (2s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 10:22:31 (1736263351) PASS 4 (4s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 10:22:35 (1736263355) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (2s) == sanityn test 6: remove of open file on other node ============================================================================ 10:22:37 (1736263357) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (3s) == sanityn test 7: remove of open directory on other node ======================================================================= 10:22:40 (1736263360) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (3s) == sanityn test 8: remove of open special file on other node ==================================================================== 10:22:43 (1736263363) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (2s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 10:22:45 (1736263365) PASS 9a (3s) == sanityn test 9b: append to striped sparse file ======== 10:22:48 (1736263368) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0301725 s, 34.8 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0319374 s, 0.1 kB/s Data read (expecting 'foo'): foo PASS 9b (3s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 10:22:51 (1736263371) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0125744 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00497782 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00249909 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00214818 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00314665 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00461198 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00263473 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00771158 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00319607 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00332895 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00419387 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00278434 s, 0.4 kB/s PASS 10a (3s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 10:22:55 (1736263375) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00972411 s, 316 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0207531 s, 197 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000245015 s, 12.5 MB/s PASS 10b (3s) == sanityn test 11: execution of file opened for write should return error ============================================================== 10:22:57 (1736263377) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6871 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (3s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 10:23:00 (1736263380) warning: '-runtime' deprecated, use '-t runtime' instead - link 817 (time 1736263391.45 total 10.00 last 81.69) - link 1659 (time 1736263401.45 total 20.00 last 84.17) - link 2392 (time 1736263411.45 total 30.00 last 73.30) - link 3056 (time 1736263421.45 total 40.01 last 66.39) - link 3774 (time 1736263431.47 total 50.02 last 71.72) - link 4482 (time 1736263441.48 total 60.03 last 70.69) - link 5358 (time 1736263451.49 total 70.04 last 87.53) - link 6320 (time 1736263461.49 total 80.04 last 96.16) - link 7307 (time 1736263471.50 total 90.05 last 98.66) total: 8314 link in 100.01 seconds: 83.14 ops/second using seed 1716725561 running for 100 seconds - stat 10000 (time 1736263386 ; total 4 ; last 4) - stat 20000 (time 1736263391 ; total 9 ; last 5) - stat 30000 (time 1736263395 ; total 13 ; last 4) - stat 40000 (time 1736263399 ; total 17 ; last 4) - stat 50000 (time 1736263403 ; total 21 ; last 4) - stat 60000 (time 1736263408 ; total 26 ; last 5) - stat 70000 (time 1736263413 ; total 31 ; last 5) - stat 80000 (time 1736263417 ; total 35 ; last 4) - stat 90000 (time 1736263422 ; total 40 ; last 5) - stat 100000 (time 1736263427 ; total 45 ; last 5) - stat 110000 (time 1736263431 ; total 49 ; last 4) - stat 117514 (time 1736263442 ; total 60 ; last 11) - stat 120000 (time 1736263445 ; total 63 ; last 3) - stat 128466 (time 1736263456 ; total 74 ; last 11) - stat 130000 (time 1736263457 ; total 75 ; last 1) - stat 137718 (time 1736263468 ; total 86 ; last 11) - stat 140000 (time 1736263470 ; total 88 ; last 2) - stat 148200 (time 1736263481 ; total 99 ; last 11) total: 149452 stats in 100 seconds: 1494.520020 stats/second - unlinked 0 (time 1736263432 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile8314) error: No such file or directory total: 8313 unlinks in 75 seconds: 110.839996 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (20481) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (20484) - No such process PASS 12 (128s) == sanityn test 13: test directory page revocation ======= 10:25:08 (1736263508) f13 f13 PASS 13 (3s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 10:25:11 (1736263511) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6871 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4260: 21849 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 10:25:13 (1736263513) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4260: 22477 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (3s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 10:25:16 (1736263516) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 23104 Terminated $DIR1/$tdir/sleep 60 PASS 14b (2s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 10:25:18 (1736263518) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 23734 Terminated $DIR1/$tdir/sleep 60 PASS 14c (2s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 10:25:20 (1736263520) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 24363 Terminated $DIR1/$tdir/sleep 60 PASS 14d (3s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 10:25:23 (1736263523) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7517184 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7517184kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (12s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 10:25:35 (1736263535) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.177029 s, 11.8 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.149009 s, 14.1 MB/s Chance of close/open is 1 in 50 Seed set to 3536 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x773a8a 000001[0] 1736263537.010594 trunc from 00000000 to 0x773a89 (0x773a8a bytes) 000003[0] 1736263537.053330 trunc from 0x773a8a to 0x69c302 (0xd7787 bytes) 000004[0] 1736263537.084766 mapread 0x32e2af thru 0x32ee66 (0xbb8 bytes) 000007[1] 1736263537.094140 mapwrite 0x8fe516 thru 0x901801 (0x32ec bytes) 000008[1] 1736263537.207169 trunc from 0x901802 to 0x76c19c (0x195665 bytes) 000009[0] 1736263537.218435 mapwrite 0x0e8966 thru 0x0ef85a (0x6ef5 bytes) 000012[0] 1736263537.307700 write 0x6c4d94 thru 0x6c7b77 (0x2de4 bytes) 000013[1] 1736263537.322778 mapwrite 0x3e1cb3 thru 0x3ef792 (0xdae0 bytes) 000014[0] 1736263537.390990 mapwrite 0x681958 thru 0x68f7dc (0xde85 bytes) 000015[0] 1736263537.463529 write 0x7a20d0 thru 0x7b13db (0xf30c bytes) 000017[1] 1736263537.471103 mapread 0x69bd53 thru 0x69d280 (0x152e bytes) 000018[0] 1736263537.497584 trunc from 0x7b13dc to 0x499858 (0x317b83 bytes) 000021[1] 1736263537.531251 trunc from 0x499859 to 0x4c6845 (0x2cfed bytes) 000022[1] 1736263537.553010 write 0x7b3df0 thru 0x7bc4dc (0x86ed bytes) 000023[1] 1736263537.554509 mapwrite 0x4594c3 thru 0x45c6b7 (0x31f5 bytes) 000026[0] 1736263537.629217 mapread 0x170727 thru 0x17be87 (0xb761 bytes) 000027[1] 1736263537.683956 mapwrite 0x437de7 thru 0x43fb5a (0x7d74 bytes) 000028[1] 1736263537.735174 write 0x4006c4 thru 0x40c1fe (0xbb3b bytes) 000029[1] 1736263537.746066 read 0x64c441 thru 0x64d13e (0xcfe bytes) 000030[0] 1736263537.751609 mapread 0x0ea4f0 thru 0x0f2b50 (0x8661 bytes) 000033[0] 1736263537.784455 trunc from 0x7bc4dd to 0x317952 (0x4a4b8a bytes) 000035[0] 1736263537.841340 write 0x797827 thru 0x7a444e (0xcc28 bytes) 000037[1] 1736263537.866820 write 0x422980 thru 0x430bbb (0xe23c bytes) 000039[1] 1736263537.895744 mapread 0x1497a9 thru 0x152858 (0x90b0 bytes) 000043[1] 1736263537.959923 trunc from 0x7a444f to 0x405043 (0x39f40b bytes) 000049[1] 1736263538.037688 trunc from 0x405044 to 0x25dea0 (0x1a71a3 bytes) 000050[0] 1736263538.055117 mapwrite 0x20e4b5 thru 0x20e904 (0x450 bytes) 000055[0] 1736263538.169008 trunc from 0x25dea1 to 0x1ce524 (0x8f97c bytes) 000056[0] 1736263538.179570 mapread 0x00a7f8 thru 0x010018 (0x5821 bytes) 000058[0] 1736263538.214696 read 0x0912a0 thru 0x095407 (0x4168 bytes) 000060[0] 1736263538.223108 mapread 0x18e049 thru 0x191524 (0x34dc bytes) 000062[1] 1736263538.235157 mapread 0x13ca59 thru 0x14be8c (0xf434 bytes) 000065[0] 1736263538.261386 mapread 0x153547 thru 0x15d07a (0x9b34 bytes) 000066[0] 1736263538.271641 write 0x27e138 thru 0x2848f4 (0x67bd bytes) 000068[1] 1736263538.272777 write 0x352eae thru 0x35ee5e (0xbfb1 bytes) 000070[1] 1736263538.295331 read 0x19da2b thru 0x19f13c (0x1712 bytes) 000073[1] 1736263538.335147 read 0x2e5e8d thru 0x2f0c40 (0xadb4 bytes) 000074[1] 1736263538.350586 mapread 0x31c9d5 thru 0x323fdc (0x7608 bytes) 000075[1] 1736263538.351269 read 0x1a179a thru 0x1a631b (0x4b82 bytes) 000076[0] 1736263538.351535 read 0x359a45 thru 0x35ee5e (0x541a bytes) 000078[0] 1736263538.388818 mapread 0x15c1e4 thru 0x15e868 (0x2685 bytes) 000079[1] 1736263538.401072 mapread 0x0dd8d2 thru 0x0e399a (0x60c9 bytes) 000080[1] 1736263538.421510 trunc from 0x35ee5f to 0x299c55 (0xc5209 bytes) 000081[1] 1736263538.458469 mapwrite 0x3c2ace thru 0x3d239a (0xf8cd bytes) 000082[0] 1736263538.602553 trunc from 0x3d239b to 0x3b3b4c (0x1e84e bytes) 000084[1] 1736263538.652197 trunc from 0x3b3b4d to 0x4f6277 (0x14272b bytes) 000087[0] 1736263538.731016 mapwrite 0x227ee0 thru 0x22e243 (0x6364 bytes) 000089[0] 1736263538.839909 read 0x38d1f2 thru 0x3970e5 (0x9ef4 bytes) 000091[0] 1736263538.888687 mapwrite 0x01fc4a thru 0x0243d6 (0x478d bytes) 000094[0] 1736263538.966999 trunc from 0x4f6278 to 0x6deda5 (0x1e8b2e bytes) 000095[0] 1736263539.018358 mapwrite 0x63450b thru 0x642be1 (0xe6d7 bytes) 000097[1] 1736263539.119291 read 0x5b297f thru 0x5b780d (0x4e8f bytes) 000098[0] 1736263539.145332 trunc from 0x6deda6 to 0x67c41d (0x62988 bytes) 000099[0] 1736263539.171113 mapread 0x275341 thru 0x2817a9 (0xc469 bytes) 000100[1] 1736263539.221218 read 0x01401e thru 0x019d37 (0x5d1a bytes) 000101[1] 1736263539.254610 write 0x1a36f6 thru 0x1ad322 (0x9c2d bytes) 000102[1] 1736263539.316407 mapread 0x646c3c thru 0x64a84b (0x3c10 bytes) 000103[0] 1736263539.346524 read 0x46d46b thru 0x475d32 (0x88c8 bytes) 000104[0] 1736263539.357945 write 0x171194 thru 0x180e9b (0xfd08 bytes) 000106[1] 1736263539.408319 mapwrite 0x20f152 thru 0x2154ca (0x6379 bytes) 000107[0] 1736263539.522791 write 0x6f568a thru 0x6f83ee (0x2d65 bytes) 000108[1] 1736263539.544206 mapread 0x22c31d thru 0x232eca (0x6bae bytes) 000114[0] 1736263539.617446 write 0x462962 thru 0x4653da (0x2a79 bytes) 000115[1] 1736263539.642075 mapread 0x57060c thru 0x578d34 (0x8729 bytes) 000116[0] 1736263539.700194 write 0x557edb thru 0x558e6f (0xf95 bytes) 000117[1] 1736263539.722656 mapread 0x6c8bcc thru 0x6d824c (0xf681 bytes) 000118[1] 1736263539.802158 write 0x36e9b1 thru 0x37bc50 (0xd2a0 bytes) 000119[1] 1736263539.820430 read 0x3b3d95 thru 0x3b5a05 (0x1c71 bytes) 000123[0] 1736263539.860640 trunc from 0x6f83ef to 0x6aebf0 (0x497fe bytes) 000124[0] 1736263539.909776 write 0x10d520 thru 0x11570f (0x81f0 bytes) 000125[0] 1736263539.916840 mapwrite 0x2603bb thru 0x26f088 (0xecce bytes) 000126[1] 1736263540.014191 trunc from 0x6aebf1 to 0x746a60 (0x97e70 bytes) 000127[0] 1736263540.041848 read 0x6ec179 thru 0x6f68f5 (0xa77d bytes) 000128[1] 1736263540.052770 read 0x1d9c31 thru 0x1dea3a (0x4e0a bytes) 000129[1] 1736263540.096999 read 0x65960b thru 0x65a05b (0xa51 bytes) 000130[0] 1736263540.107493 write 0x7a5cac thru 0x7a7124 (0x1479 bytes) 000131[1] 1736263540.119287 mapwrite 0x880525 thru 0x8860cb (0x5ba7 bytes) 000132[0] 1736263540.215652 mapwrite 0x4d1f05 thru 0x4d3730 (0x182c bytes) 000135[0] 1736263540.301940 mapread 0x47a00d thru 0x47faaa (0x5a9e bytes) 000136[0] 1736263540.329732 trunc from 0x8860cc to 0x39bb8d (0x4ea53e bytes) 000139[1] 1736263540.366619 write 0x01b0bf thru 0x023689 (0x85cb bytes) 000143[0] 1736263540.413047 write 0x9a39cd thru 0x9af77f (0xbdb3 bytes) 000144[0] 1736263540.422444 read 0x95a460 thru 0x9654f2 (0xb093 bytes) 000146[1] 1736263540.454189 trunc from 0x9af780 to 0x3e89ca (0x5c6db5 bytes) 000147[1] 1736263540.472308 write 0x16818a thru 0x16f363 (0x71da bytes) 000149[1] 1736263540.479207 mapread 0x0a74c0 thru 0x0a93fc (0x1f3d bytes) 000150[1] 1736263540.496348 mapwrite 0x3d765a thru 0x3e3bc2 (0xc569 bytes) 000153[1] 1736263540.585718 write 0x9e4e21 thru 0x9e70fa (0x22da bytes) 000154[1] 1736263540.601168 mapwrite 0x8f7384 thru 0x901012 (0x9c8f bytes) 000158[0] 1736263540.717444 mapwrite 0x38701f thru 0x38963b (0x261d bytes) 000159[1] 1736263540.791002 write 0x170e98 thru 0x1795aa (0x8713 bytes) 000162[1] 1736263540.830615 mapwrite 0x4c2770 thru 0x4c38f3 (0x1184 bytes) 000164[1] 1736263540.876501 trunc from 0x9e70fb to 0x669639 (0x37dac1 bytes) 000166[0] 1736263540.907555 read 0x3b2898 thru 0x3bad76 (0x84df bytes) 000169[1] 1736263540.952220 mapwrite 0x86595c thru 0x8719b0 (0xc055 bytes) 000170[0] 1736263541.037202 write 0x5d2efa thru 0x5d3c7d (0xd84 bytes) 000171[0] 1736263541.058727 mapwrite 0x939172 thru 0x946c98 (0xdb27 bytes) 000175[0] 1736263541.157094 write 0x680f7b thru 0x687fcc (0x7052 bytes) 000180[0] 1736263541.158725 mapread 0x8343f3 thru 0x8376c0 (0x32ce bytes) 000182[0] 1736263541.174391 trunc from 0x946c99 to 0x0f8ff5 (0x84dca3 bytes) truncating to largest ever: 0x9c59e5 000183[0] 1736263541.200967 trunc from 0x0f8ff6 to 0x9c59e4 (0x8cc9ef bytes) 000184[1] 1736263541.218462 mapread 0x9601b9 thru 0x965363 (0x51ab bytes) 000186[1] 1736263541.246062 mapread 0x725700 thru 0x734811 (0xf112 bytes) 000189[1] 1736263541.269776 trunc from 0x9c59e5 to 0x5acbb1 (0x418e33 bytes) 000193[0] 1736263541.318006 mapwrite 0x560969 thru 0x56f8f7 (0xef8f bytes) 000194[0] 1736263541.404912 write 0x226fc5 thru 0x2313da (0xa416 bytes) 000195[1] 1736263541.428150 write 0x1abdce thru 0x1af003 (0x3236 bytes) 000198[0] 1736263541.479815 mapread 0x037586 thru 0x0435d7 (0xc052 bytes) 000199[0] 1736263541.524866 trunc from 0x5acbb2 to 0x87f5e9 (0x2d2a38 bytes) 000200[0] 1736263541.559467 read 0x50c21d thru 0x512bf8 (0x69dc bytes) 000203[1] 1736263541.638306 mapwrite 0x4b7eb3 thru 0x4bf411 (0x755f bytes) 000204[0] 1736263541.720406 trunc from 0x87f5ea to 0x8e4efe (0x65915 bytes) 000205[0] 1736263541.744449 mapwrite 0x6dbf77 thru 0x6e3629 (0x76b3 bytes) 000206[0] 1736263541.825181 mapwrite 0x22588e thru 0x22d3a7 (0x7b1a bytes) 000208[1] 1736263541.890860 trunc from 0x8e4eff to 0x51f4e5 (0x3c5a19 bytes) 000210[0] 1736263541.939500 write 0x83cee8 thru 0x8477ae (0xa8c7 bytes) 000213[1] 1736263541.992334 read 0x561155 thru 0x569ff9 (0x8ea5 bytes) 000216[1] 1736263542.046700 write 0x698274 thru 0x6a772f (0xf4bc bytes) 000217[0] 1736263542.057545 mapwrite 0x8f7862 thru 0x90185e (0x9ffd bytes) 000220[0] 1736263542.142590 write 0x0e15af thru 0x0e4d09 (0x375b bytes) 000222[0] 1736263542.158801 trunc from 0x90185f to 0x2f7cde (0x609b80 bytes) 000225[0] 1736263542.182598 read 0x1011a8 thru 0x10a825 (0x967e bytes) 000226[1] 1736263542.198695 read 0x2aa8ce thru 0x2ac747 (0x1e7a bytes) 000231[1] 1736263542.244564 mapwrite 0x1c9a03 thru 0x1cac3b (0x1239 bytes) 000232[0] 1736263542.293637 mapread 0x011aee thru 0x01de48 (0xc35b bytes) 000233[1] 1736263542.339617 mapwrite 0x4dd102 thru 0x4e7cdf (0xabde bytes) 000234[1] 1736263542.426989 trunc from 0x4e7ce0 to 0x297b64 (0x25017b bytes) 000235[1] 1736263542.439313 read 0x1e7aa7 thru 0x1ec3fd (0x4957 bytes) 000237[0] 1736263542.446942 mapwrite 0x1c8469 thru 0x1d5b75 (0xd70d bytes) 000241[1] 1736263542.597665 write 0x4dc81e thru 0x4e497d (0x8160 bytes) 000242[1] 1736263542.610027 mapwrite 0x83c319 thru 0x83fddd (0x3ac5 bytes) 000244[0] 1736263542.724461 trunc from 0x83fdde to 0x50612b (0x339cb2 bytes) 000245[0] 1736263542.770795 read 0x009f33 thru 0x00be9d (0x1f6b bytes) 000248[1] 1736263542.829710 read 0x2be55d thru 0x2c2cf8 (0x479c bytes) 000249[1] 1736263542.850475 write 0x71bf68 thru 0x729792 (0xd82b bytes) 000250[1] 1736263542.863139 trunc from 0x729793 to 0x5dbf44 (0x14d84e bytes) 000251[1] 1736263542.893894 read 0x39e7c7 thru 0x3ac5a1 (0xdddb bytes) 000253[1] 1736263542.940595 write 0x42dc7c thru 0x438aa8 (0xae2d bytes) 000257[0] 1736263542.961623 mapwrite 0x560c2b thru 0x56dd62 (0xd138 bytes) 000260[1] 1736263543.064802 read 0x1b7701 thru 0x1be607 (0x6f07 bytes) 000262[1] 1736263543.126142 mapwrite 0x09f1e9 thru 0x0a2068 (0x2e80 bytes) 000263[0] 1736263543.198755 trunc from 0x5dbf45 to 0x8e3959 (0x307a15 bytes) 000264[0] 1736263543.228794 mapread 0x1d4f79 thru 0x1de962 (0x99ea bytes) 000265[1] 1736263543.272128 mapread 0x058a78 thru 0x064026 (0xb5af bytes) 000266[1] 1736263543.325925 mapwrite 0x537ffc thru 0x541d1c (0x9d21 bytes) 000268[1] 1736263543.475427 trunc from 0x8e395a to 0x146596 (0x79d3c3 bytes) 000270[0] 1736263543.515277 write 0x959076 thru 0x95c82b (0x37b6 bytes) 000272[0] 1736263543.550598 mapwrite 0x0e9408 thru 0x0f2b20 (0x9719 bytes) 000274[1] 1736263543.650033 trunc from 0x95c82c to 0x11df2f (0x83e8fc bytes) 000275[0] 1736263543.687461 read 0x08553b thru 0x086581 (0x1047 bytes) 000276[0] 1736263543.726676 mapread 0x0e5cb6 thru 0x0ee362 (0x86ad bytes) 000279[0] 1736263543.755166 mapwrite 0x1d4bf3 thru 0x1d51c5 (0x5d3 bytes) 000280[0] 1736263543.857289 mapwrite 0x10f428 thru 0x11d165 (0xdd3e bytes) 000281[0] 1736263543.992045 trunc from 0x1d51c6 to 0x44e51c (0x279357 bytes) 000282[0] 1736263544.021953 read 0x0b0d04 thru 0x0bee3b (0xe138 bytes) 000286[1] 1736263544.084868 write 0x61a850 thru 0x629dca (0xf57b bytes) 000288[1] 1736263544.116806 write 0x0747e8 thru 0x074a16 (0x22f bytes) 000289[0] 1736263544.132924 mapwrite 0x36daa8 thru 0x36f74a (0x1ca3 bytes) 000291[0] 1736263544.199051 trunc from 0x629dcb to 0x65d8ab (0x33ae1 bytes) 000292[1] 1736263544.226043 mapwrite 0x9a6358 thru 0x9aee4b (0x8af4 bytes) 000295[1] 1736263544.343594 mapwrite 0x39a867 thru 0x3a30af (0x8849 bytes) 000296[1] 1736263544.444386 write 0x03604e thru 0x03ae61 (0x4e14 bytes) 000297[1] 1736263544.458297 mapwrite 0x9af5b5 thru 0x9b4c89 (0x56d5 bytes) 000298[0] 1736263544.560017 trunc from 0x9b4c8a to 0x7a9a6d (0x20b21c bytes) 000299[0] 1736263544.593907 write 0x2377df thru 0x23ebd4 (0x73f6 bytes) 000301[0] 1736263544.620417 mapwrite 0x4271b0 thru 0x42d96c (0x67bd bytes) 000304[0] 1736263544.707199 read 0x6293f7 thru 0x632e7c (0x9a86 bytes) 000306[1] 1736263544.731985 trunc from 0x7a9a6e to 0x25a4dc (0x54f591 bytes) 000307[0] 1736263544.769836 read 0x24993b thru 0x254d5d (0xb423 bytes) 000308[0] 1736263544.785539 read 0x20a8ff thru 0x20e67b (0x3d7d bytes) 000309[0] 1736263544.819316 trunc from 0x25a4dd to 0x7a5572 (0x54b096 bytes) 000310[0] 1736263544.867593 write 0x58abc5 thru 0x58f9ae (0x4dea bytes) 000311[1] 1736263544.918141 write 0x7811ef thru 0x788819 (0x762b bytes) 000315[0] 1736263544.981015 read 0x60d4ab thru 0x60fea2 (0x29f8 bytes) 000316[1] 1736263544.990684 write 0x120636 thru 0x1253fe (0x4dc9 bytes) 000318[1] 1736263545.009534 trunc from 0x7a5573 to 0x2317ba (0x573db8 bytes) 000320[0] 1736263545.043337 mapwrite 0x38a36e thru 0x38e82c (0x44bf bytes) 000321[0] 1736263545.136460 trunc from 0x38e82d to 0x15274d (0x23c0df bytes) 000322[0] 1736263545.157081 trunc from 0x15274e to 0x719d99 (0x5c764c bytes) 000324[1] 1736263545.176548 trunc from 0x719d9a to 0x101fa3 (0x617df6 bytes) 000325[0] 1736263545.205194 write 0x5ee58b thru 0x5ef209 (0xc7f bytes) 000326[1] 1736263545.220353 trunc from 0x5ef20a to 0x3586f1 (0x296b18 bytes) 000327[0] 1736263545.239811 read 0x1b2cd7 thru 0x1b4f08 (0x2232 bytes) 000329[0] 1736263545.261771 trunc from 0x3586f2 to 0x3b9850 (0x6115f bytes) 000330[0] 1736263545.290529 mapread 0x0ebb45 thru 0x0f7aa7 (0xbf63 bytes) 000331[0] 1736263545.351671 write 0x4c7ec0 thru 0x4caeab (0x2fec bytes) 000332[1] 1736263545.364487 trunc from 0x4caeac to 0x0f7642 (0x3d3869 bytes) 000334[1] 1736263545.412249 read 0x08fe26 thru 0x095cce (0x5ea9 bytes) 000335[1] 1736263545.436067 write 0x363e06 thru 0x370b64 (0xcd5f bytes) 000336[1] 1736263545.438125 mapread 0x2d5016 thru 0x2d8602 (0x35ed bytes) 000337[0] 1736263545.456093 write 0x33ecbf thru 0x33f1f8 (0x53a bytes) 000338[1] 1736263545.486748 trunc from 0x370b65 to 0x0d3bff (0x29cf65 bytes) 000342[1] 1736263545.549182 write 0x2972f9 thru 0x2a10fe (0x9e06 bytes) 000344[1] 1736263545.552513 trunc from 0x2a10ff to 0x8f1e13 (0x650d15 bytes) 000345[0] 1736263545.573493 mapwrite 0x35f823 thru 0x36f102 (0xf8e0 bytes) 000346[1] 1736263545.694195 write 0x79386b thru 0x7968ea (0x3080 bytes) 000348[0] 1736263545.720672 trunc from 0x8f1e14 to 0x06101f (0x890df4 bytes) 000350[0] 1736263545.791516 mapread 0x012e7d thru 0x0179d6 (0x4b5a bytes) 000353[0] 1736263545.823734 mapread 0x029663 thru 0x0315f3 (0x7f91 bytes) 000354[1] 1736263545.851001 mapwrite 0x28bc72 thru 0x293b67 (0x7ef6 bytes) 000356[1] 1736263545.946586 mapwrite 0x4be141 thru 0x4c5e6f (0x7d2f bytes) 000357[0] 1736263546.032791 read 0x23ba71 thru 0x241c3c (0x61cc bytes) 000358[1] 1736263546.060290 read 0x2f3813 thru 0x300672 (0xce60 bytes) 000359[1] 1736263546.090017 mapwrite 0x59f6d1 thru 0x5a3ec8 (0x47f8 bytes) 000361[0] 1736263546.156122 read 0x39cbcd thru 0x3ab0a8 (0xe4dc bytes) 000364[0] 1736263546.194179 read 0x525f04 thru 0x528dc2 (0x2ebf bytes) 000365[1] 1736263546.213637 mapread 0x2c5b7f thru 0x2c6257 (0x6d9 bytes) 000366[1] 1736263546.220461 mapread 0x13d7fd thru 0x13e7e3 (0xfe7 bytes) 000369[1] 1736263546.243750 mapread 0x337ffe thru 0x33d242 (0x5245 bytes) 000370[1] 1736263546.269705 trunc from 0x5a3ec9 to 0x8c192b (0x31da63 bytes) 000371[1] 1736263546.295493 mapread 0x4a1e1c thru 0x4a2d8a (0xf6f bytes) 000372[1] 1736263546.307153 mapwrite 0x15d6e1 thru 0x15feaa (0x27ca bytes) 000374[0] 1736263546.355158 trunc from 0x8c192c to 0x9a5601 (0xe3cd6 bytes) 000375[1] 1736263546.373366 read 0x3dd164 thru 0x3e5f7f (0x8e1c bytes) 000378[0] 1736263546.402980 mapread 0x7a9089 thru 0x7b33ed (0xa365 bytes) 000380[1] 1736263546.468751 read 0x4e9c71 thru 0x4f9842 (0xfbd2 bytes) 000381[0] 1736263546.516648 mapread 0x451cd5 thru 0x454b3b (0x2e67 bytes) 000382[0] 1736263546.532828 write 0x061800 thru 0x06867e (0x6e7f bytes) 000383[1] 1736263546.546370 mapread 0x22be99 thru 0x230b27 (0x4c8f bytes) 000386[0] 1736263546.583951 trunc from 0x9a5602 to 0x69f372 (0x30628f bytes) 000389[1] 1736263546.624167 trunc from 0x69f373 to 0x84f571 (0x1b01ff bytes) 000392[0] 1736263546.673266 mapwrite 0x0d33c1 thru 0x0e24ee (0xf12e bytes) 000393[0] 1736263546.800853 mapread 0x5d707c thru 0x5db98e (0x4913 bytes) 000395[1] 1736263546.826815 trunc from 0x84f572 to 0x9bd3a4 (0x16de33 bytes) 000396[0] 1736263546.864695 mapwrite 0x28f698 thru 0x29cf1a (0xd883 bytes) 000397[0] 1736263546.980493 read 0x53907c thru 0x53a11e (0x10a3 bytes) truncating to largest ever: 0x9ea1cf 000398[1] 1736263547.046758 trunc from 0x9bd3a5 to 0x9ea1ce (0x2ce2a bytes) 000399[0] 1736263547.082737 trunc from 0x9ea1cf to 0x13deab (0x8ac323 bytes) 000402[0] 1736263547.128627 mapread 0x039e5f thru 0x0458b0 (0xba52 bytes) 000403[0] 1736263547.178919 trunc from 0x13deac to 0x0a9631 (0x9487a bytes) 000405[1] 1736263547.200513 trunc from 0x0a9632 to 0x58f7fd (0x4e61cc bytes) 000406[1] 1736263547.242562 write 0x35912e thru 0x35d0f6 (0x3fc9 bytes) 000411[1] 1736263547.289749 trunc from 0x58f7fe to 0x87c7d7 (0x2ecfda bytes) 000412[0] 1736263547.304734 mapread 0x4a7473 thru 0x4b5f5b (0xeae9 bytes) 000414[1] 1736263547.372846 write 0x55d4dd thru 0x566738 (0x925c bytes) 000415[0] 1736263547.382094 write 0x2c0654 thru 0x2ca8a0 (0xa24d bytes) 000416[0] 1736263547.404297 read 0x567baa thru 0x572f64 (0xb3bb bytes) 000419[0] 1736263547.461241 write 0x3c39db thru 0x3c40fc (0x722 bytes) 000420[1] 1736263547.475628 trunc from 0x87c7d8 to 0x52f7a3 (0x34d034 bytes) 000421[0] 1736263547.505743 mapwrite 0x6346b7 thru 0x642aef (0xe439 bytes) 000423[1] 1736263547.621328 mapread 0x08dd22 thru 0x091ab8 (0x3d97 bytes) 000424[1] 1736263547.648456 read 0x2cb852 thru 0x2d25dc (0x6d8b bytes) 000425[1] 1736263547.666993 mapwrite 0x150de2 thru 0x159a3c (0x8c5b bytes) 000426[0] 1736263547.818640 read 0x6263b6 thru 0x62ee48 (0x8a93 bytes) 000427[0] 1736263547.825643 trunc from 0x642af0 to 0x2a7879 (0x39b276 bytes) 000428[1] 1736263547.862044 read 0x215e99 thru 0x216dcc (0xf34 bytes) 000429[1] 1736263547.899420 write 0x555a1e thru 0x555a3e (0x21 bytes) 000431[0] 1736263547.920368 trunc from 0x555a3f to 0x06332e (0x4f2710 bytes) 000432[0] 1736263547.951406 mapwrite 0x1a3800 thru 0x1b1ce4 (0xe4e5 bytes) 000436[0] 1736263548.096179 mapread 0x1207e0 thru 0x12bb81 (0xb3a2 bytes) 000438[0] 1736263548.144033 read 0x0b6c2d thru 0x0bb4fe (0x48d2 bytes) 000439[1] 1736263548.156169 read 0x0bfc6c thru 0x0c43b9 (0x474e bytes) 000440[0] 1736263548.182018 read 0x03cc95 thru 0x04266f (0x59db bytes) 000441[1] 1736263548.196554 mapread 0x0af3f2 thru 0x0b0e5f (0x1a6e bytes) 000442[1] 1736263548.223305 mapread 0x084c22 thru 0x0875a2 (0x2981 bytes) 000443[0] 1736263548.253571 mapread 0x06614c thru 0x06bd14 (0x5bc9 bytes) 000445[1] 1736263548.254643 read 0x0a3a3d thru 0x0a4454 (0xa18 bytes) 000447[1] 1736263548.264697 trunc from 0x1b1ce5 to 0x696408 (0x4e4724 bytes) 000451[0] 1736263548.312792 write 0x418f3f thru 0x424aee (0xbbb0 bytes) 000452[1] 1736263548.340680 mapread 0x4a5514 thru 0x4ac7cf (0x72bc bytes) 000454[0] 1736263548.440819 read 0x4830cd thru 0x48991f (0x6853 bytes) 000455[0] 1736263548.454275 mapwrite 0x6d70b9 thru 0x6e2d9c (0xbce4 bytes) 000456[0] 1736263548.572057 mapwrite 0x97190a thru 0x9728e6 (0xfdd bytes) 000458[0] 1736263548.627578 mapwrite 0x54ede7 thru 0x5578b6 (0x8ad0 bytes) 000460[1] 1736263548.708848 mapread 0x8554d0 thru 0x861a07 (0xc538 bytes) 000464[0] 1736263548.814353 mapwrite 0x6b6b27 thru 0x6c4ca3 (0xe17d bytes) 000468[1] 1736263548.965891 trunc from 0x9728e7 to 0x4d240a (0x4a04dc bytes) 000472[1] 1736263549.020707 mapwrite 0x651b86 thru 0x65528c (0x3707 bytes) 000473[0] 1736263549.108275 write 0x001022 thru 0x00be1f (0xadfe bytes) 000475[1] 1736263549.144658 write 0x0b2611 thru 0x0bdab4 (0xb4a4 bytes) 000476[1] 1736263549.163608 mapwrite 0x970832 thru 0x974371 (0x3b40 bytes) 000477[0] 1736263549.259862 mapread 0x5092b3 thru 0x50abfa (0x1948 bytes) 000478[1] 1736263549.286752 mapwrite 0x3b88cc thru 0x3c2928 (0xa05d bytes) 000481[0] 1736263549.375121 write 0x8f908a thru 0x9017c0 (0x8737 bytes) 000483[0] 1736263549.412066 mapread 0x40ca71 thru 0x41aadb (0xe06b bytes) 000485[1] 1736263549.467323 read 0x5466d1 thru 0x54b978 (0x52a8 bytes) 000486[0] 1736263549.501702 mapwrite 0x41735f thru 0x418444 (0x10e6 bytes) 000487[0] 1736263549.546282 mapread 0x77b5aa thru 0x783c04 (0x865b bytes) 000489[1] 1736263549.592371 write 0x91723e thru 0x91ef8f (0x7d52 bytes) 000492[0] 1736263549.651059 read 0x91ea4c thru 0x92a7ea (0xbd9f bytes) 000493[0] 1736263549.688712 write 0x0dbca7 thru 0x0e7c76 (0xbfd0 bytes) 000494[1] 1736263549.694384 mapwrite 0x71c3a4 thru 0x71cd6f (0x9cc bytes) 000495[0] 1736263549.762358 mapwrite 0x233c31 thru 0x238081 (0x4451 bytes) 000497[0] 1736263549.863350 mapread 0x45ffcf thru 0x464425 (0x4457 bytes) 000498[0] 1736263549.901506 mapwrite 0x974bf7 thru 0x981d80 (0xd18a bytes) 000499[1] 1736263549.973987 mapwrite 0x87fb2f thru 0x88bdfa (0xc2cc bytes) 000500[1] 1736263550.083869 read 0x121e30 thru 0x12daed (0xbcbe bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 3550 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x3b07 000003[0] 1736263550.510305 trunc from 00000000 to 0x003b06 (0x3b07 bytes) 000004[1] 1736263550.533857 read 0x001000 thru 0x002fb0 (0x1fb1 bytes) 000006[0] 1736263550.547912 read_OD 0x001000 thru 0x002fff (0x2000 bytes) 000007[1] 1736263550.553468 mapwrite 0x670000 thru 0x67c4b7 (0xc4b8 bytes) 000009[0] 1736263550.669783 write_OD 0x969000 thru 0x96ffff (0x7000 bytes) truncating to largest ever: 0x6cafad 000012[1] 1736263550.733079 trunc from 0x970000 to 0x6cafac (0x2a5053 bytes) 000013[0] 1736263550.779540 trunc from 0x6cafad to 0x53947d (0x191b2f bytes) truncating to largest ever: 0x9df85b 000014[0] 1736263550.802368 trunc from 0x53947e to 0x9df85a (0x4a63dd bytes) 000015[1] 1736263550.815361 trunc from 0x9df85b to 0x6de436 (0x301424 bytes) 000020[0] 1736263550.862094 write_OD 0x81d000 thru 0x825fff (0x9000 bytes) 000021[0] 1736263550.914311 mapwrite 0x79c000 thru 0x7a309f (0x70a0 bytes) 000022[1] 1736263551.001512 mapread 0x811000 thru 0x81957e (0x857f bytes) 000023[1] 1736263551.029122 mapwrite 0x8e4000 thru 0x8f33b3 (0xf3b4 bytes) 000025[0] 1736263551.127129 trunc from 0x8f33b4 to 0x19f8bd (0x753af6 bytes) 000026[0] 1736263551.166962 write_OD 0x307000 thru 0x310fff (0xa000 bytes) 000027[0] 1736263551.224621 trunc from 0x311000 to 0x7d180d (0x4c080e bytes) 000028[1] 1736263551.253613 mapread 0x296000 thru 0x29a5e4 (0x45e5 bytes) 000030[1] 1736263551.288684 mapread 0x58d000 thru 0x596750 (0x9751 bytes) 000031[1] 1736263551.300936 mapwrite 0x7a6000 thru 0x7b076c (0xa76d bytes) 000032[0] 1736263551.386406 mapwrite 0x0b5000 thru 0x0beec4 (0x9ec5 bytes) 000035[1] 1736263551.445443 read 0x2da000 thru 0x2e115e (0x715f bytes) 000038[1] 1736263551.490201 trunc from 0x7d180e to 0x551f48 (0x27f8c5 bytes) 000039[1] 1736263551.514370 mapread 0x0cf000 thru 0x0cf748 (0x749 bytes) 000044[0] 1736263551.539693 mapwrite 0x36b000 thru 0x36fcaf (0x4cb0 bytes) 000046[0] 1736263551.614861 write_OD 0x359000 thru 0x35bfff (0x3000 bytes) 000047[0] 1736263551.651956 mapread 0x010000 thru 0x01b2e6 (0xb2e7 bytes) 000051[1] 1736263551.671964 mapread 0x48f000 thru 0x491108 (0x2109 bytes) 000053[1] 1736263551.679308 trunc from 0x551f49 to 0x01b2e7 (0x536c61 bytes) 000055[1] 1736263551.710637 mapwrite 0x139000 thru 0x148cea (0xfceb bytes) 000056[0] 1736263551.839618 mapwrite 0x05e000 thru 0x06d165 (0xf166 bytes) 000058[1] 1736263551.905099 write 0x098000 thru 0x0a0df0 (0x8df1 bytes) 000061[1] 1736263551.954780 trunc from 0x148ceb to 0x2c8ab2 (0x17fdc8 bytes) 000067[0] 1736263552.017651 write_OD 0x510000 thru 0x51dfff (0xe000 bytes) 000069[1] 1736263552.081002 mapwrite 0x9e2000 thru 0x9e655b (0x455c bytes) 000072[0] 1736263552.210303 mapread 0x104000 thru 0x106ecb (0x2ecc bytes) 000073[1] 1736263552.231663 mapread 0x5e8000 thru 0x5f3350 (0xb351 bytes) 000074[1] 1736263552.272487 mapwrite 0x2ef000 thru 0x2f925b (0xa25c bytes) 000075[0] 1736263552.371610 trunc from 0x9e655c to 0x8fa194 (0xec3c7 bytes) 000078[1] 1736263552.394668 mapwrite 0x698000 thru 0x69b0b0 (0x30b1 bytes) 000079[0] 1736263552.459169 read_OD 0x662000 thru 0x666fff (0x5000 bytes) 000082[1] 1736263552.478784 mapwrite 0x800000 thru 0x80f371 (0xf372 bytes) 000084[0] 1736263552.595486 trunc from 0x8fa195 to 0x062778 (0x897a1c bytes) 000085[1] 1736263552.611538 read 0x039000 thru 0x03bbde (0x2bdf bytes) 000086[1] 1736263552.633466 mapwrite 0x0ca000 thru 0x0d88f6 (0xe8f7 bytes) 000087[0] 1736263552.728984 mapread 0x003000 thru 0x00b936 (0x8937 bytes) 000088[0] 1736263552.768791 read_OD 0x0ae000 thru 0x0bafff (0xd000 bytes) 000089[1] 1736263552.781512 read 0x0b2000 thru 0x0be69e (0xc69f bytes) 000090[0] 1736263552.792458 trunc from 0x0d88f7 to 0x490f1e (0x3b8628 bytes) 000091[1] 1736263552.811513 mapread 0x231000 thru 0x23da82 (0xca83 bytes) 000093[0] 1736263552.860642 trunc from 0x490f1f to 0x79a79c (0x30987e bytes) 000094[1] 1736263552.884001 write 0x5a6000 thru 0x5ac159 (0x615a bytes) 000095[1] 1736263552.901310 write 0x2b6000 thru 0x2c0ff7 (0xaff8 bytes) 000097[0] 1736263552.910134 mapwrite 0x64f000 thru 0x65b2d0 (0xc2d1 bytes) 000101[0] 1736263553.057714 read_OD 0x111000 thru 0x11bfff (0xb000 bytes) 000103[0] 1736263553.080726 write_OD 0x8c2000 thru 0x8c7fff (0x6000 bytes) 000104[1] 1736263553.119030 mapread 0x60c000 thru 0x61ba90 (0xfa91 bytes) 000105[1] 1736263553.181778 write 0x41d000 thru 0x42bd65 (0xed66 bytes) 000106[1] 1736263553.216722 write 0x05e000 thru 0x060add (0x2ade bytes) 000107[1] 1736263553.237005 mapread 0x60b000 thru 0x6138b0 (0x88b1 bytes) 000108[1] 1736263553.305253 trunc from 0x8c8000 to 0x942d7a (0x7ad7b bytes) 000110[0] 1736263553.326003 mapwrite 0x0ce000 thru 0x0d8d3b (0xad3c bytes) 000112[0] 1736263553.504730 read_OD 0x864000 thru 0x871fff (0xe000 bytes) 000113[1] 1736263553.531798 mapwrite 0x04d000 thru 0x050c13 (0x3c14 bytes) 000115[1] 1736263553.620677 mapread 0x5ae000 thru 0x5aeccb (0xccc bytes) 000116[1] 1736263553.641309 read 0x253000 thru 0x2532fa (0x2fb bytes) 000118[0] 1736263553.716107 mapread 0x71f000 thru 0x71f7b0 (0x7b1 bytes) 000120[1] 1736263553.770483 write 0x975000 thru 0x97c2d8 (0x72d9 bytes) 000121[0] 1736263553.797484 mapread 0x569000 thru 0x577a88 (0xea89 bytes) 000122[0] 1736263553.883529 read_OD 0x5fa000 thru 0x604fff (0xb000 bytes) 000123[0] 1736263553.892003 read_OD 0x769000 thru 0x774fff (0xc000 bytes) 000124[1] 1736263553.901699 mapread 0x5e1000 thru 0x5e4085 (0x3086 bytes) 000125[1] 1736263553.920800 mapwrite 0x66e000 thru 0x67a8ff (0xc900 bytes) 000129[0] 1736263554.058184 read_OD 0x4a4000 thru 0x4b2fff (0xf000 bytes) 000130[1] 1736263554.076627 write 0x166000 thru 0x167f22 (0x1f23 bytes) 000132[0] 1736263554.098411 trunc from 0x97c2d9 to 0x5f1a82 (0x38a856 bytes) 000137[0] 1736263554.140334 write_OD 0x6d4000 thru 0x6d7fff (0x4000 bytes) 000139[1] 1736263554.194059 mapwrite 0x438000 thru 0x447b85 (0xfb86 bytes) 000143[0] 1736263554.312112 write_OD 0x8c3000 thru 0x8d0fff (0xe000 bytes) 000144[0] 1736263554.373115 trunc from 0x8d1000 to 0x1fdf40 (0x6d30bf bytes) 000145[0] 1736263554.416000 write_OD 0x18a000 thru 0x18dfff (0x4000 bytes) 000147[0] 1736263554.476909 read_OD 0x1aa000 thru 0x1acfff (0x3000 bytes) 000149[0] 1736263554.480262 write_OD 0x410000 thru 0x414fff (0x5000 bytes) 000151[1] 1736263554.536170 read 0x1f7000 thru 0x203414 (0xc415 bytes) 000153[0] 1736263554.558393 write_OD 0x6a3000 thru 0x6a9fff (0x7000 bytes) 000154[0] 1736263554.607600 read_OD 0x2c0000 thru 0x2c6fff (0x7000 bytes) 000158[0] 1736263554.617957 mapread 0x21d000 thru 0x21eb98 (0x1b99 bytes) 000159[1] 1736263554.660078 mapwrite 0x59e000 thru 0x5a13d5 (0x33d6 bytes) 000161[0] 1736263554.745325 mapread 0x2ac000 thru 0x2b029b (0x429c bytes) 000162[0] 1736263554.780388 write_OD 0x89f000 thru 0x8abfff (0xd000 bytes) 000163[1] 1736263554.824837 mapwrite 0x611000 thru 0x616382 (0x5383 bytes) 000164[0] 1736263554.933131 trunc from 0x8ac000 to 0x36d4da (0x53eb25 bytes) 000166[1] 1736263554.973334 read 0x2f0000 thru 0x2fd38b (0xd38c bytes) 000170[1] 1736263554.993535 read 0x264000 thru 0x26911f (0x5120 bytes) 000173[0] 1736263555.000321 read_OD 0x2a7000 thru 0x2affff (0x9000 bytes) 000174[0] 1736263555.004708 trunc from 0x36d4db to 0x2cd8e3 (0x9fbf7 bytes) 000176[0] 1736263555.038259 write_OD 0x491000 thru 0x498fff (0x8000 bytes) 000177[1] 1736263555.088786 read 0x255000 thru 0x261bbb (0xcbbc bytes) 000178[0] 1736263555.119522 read_OD 0x329000 thru 0x32efff (0x6000 bytes) 000179[0] 1736263555.129080 trunc from 0x499000 to 0x029b3e (0x46f4c1 bytes) 000180[0] 1736263555.168781 mapread 0x027000 thru 0x02923a (0x223b bytes) 000183[0] 1736263555.202513 mapwrite 0x818000 thru 0x81b905 (0x3906 bytes) 000184[0] 1736263555.309648 mapwrite 0x686000 thru 0x68cb82 (0x6b83 bytes) 000185[0] 1736263555.353417 mapread 0x0cd000 thru 0x0d9acd (0xcace bytes) 000186[0] 1736263555.387029 read_OD 0x50b000 thru 0x517fff (0xd000 bytes) 000187[0] 1736263555.403283 write_OD 0x240000 thru 0x24cfff (0xd000 bytes) 000188[1] 1736263555.457007 mapwrite 0x0b0000 thru 0x0bd0a1 (0xd0a2 bytes) 000189[0] 1736263555.610471 read_OD 0x40a000 thru 0x40efff (0x5000 bytes) 000195[0] 1736263555.636494 trunc from 0x81b906 to 0x40650d (0x4153f8 bytes) 000198[1] 1736263555.669581 mapread 0x3d6000 thru 0x3e057a (0xa57b bytes) 000201[0] 1736263555.717650 read_OD 0x0e2000 thru 0x0edfff (0xc000 bytes) 000202[1] 1736263555.721113 mapwrite 0x6e3000 thru 0x6e51a7 (0x21a8 bytes) 000203[1] 1736263555.790527 mapread 0x130000 thru 0x1344d0 (0x44d1 bytes) 000206[0] 1736263555.830149 mapread 0x0e7000 thru 0x0f6459 (0xf45a bytes) 000207[1] 1736263555.886366 write 0x0ef000 thru 0x0f6d5d (0x7d5e bytes) 000208[1] 1736263555.914817 mapwrite 0x580000 thru 0x583c64 (0x3c65 bytes) 000209[0] 1736263555.979724 write_OD 0x081000 thru 0x08dfff (0xd000 bytes) 000212[0] 1736263556.026398 trunc from 0x6e51a8 to 0x3a6482 (0x33ed25 bytes) 000213[1] 1736263556.066137 mapwrite 0x13a000 thru 0x13c065 (0x2066 bytes) 000214[0] 1736263556.144667 mapwrite 0x608000 thru 0x60e562 (0x6563 bytes) 000215[0] 1736263556.225867 mapwrite 0x43b000 thru 0x447359 (0xc35a bytes) 000216[0] 1736263556.325108 mapread 0x085000 thru 0x085570 (0x571 bytes) 000218[1] 1736263556.343781 mapwrite 0x6de000 thru 0x6e9421 (0xb422 bytes) 000221[1] 1736263556.474950 mapwrite 0x376000 thru 0x37c016 (0x6017 bytes) 000222[1] 1736263556.546971 mapwrite 0x586000 thru 0x5914f8 (0xb4f9 bytes) 000226[1] 1736263556.687365 write 0x741000 thru 0x743d00 (0x2d01 bytes) 000227[0] 1736263556.696399 read_OD 0x5c2000 thru 0x5cffff (0xe000 bytes) 000229[1] 1736263556.719992 mapread 0x721000 thru 0x7243a5 (0x33a6 bytes) 000230[1] 1736263556.737139 mapread 0x5d3000 thru 0x5d87f3 (0x57f4 bytes) 000235[0] 1736263556.754535 read_OD 0x083000 thru 0x090fff (0xe000 bytes) 000236[0] 1736263556.760509 read_OD 0x257000 thru 0x263fff (0xd000 bytes) 000237[1] 1736263556.764790 write 0x3c8000 thru 0x3c8fc2 (0xfc3 bytes) 000240[1] 1736263556.782926 mapwrite 0x792000 thru 0x7923dd (0x3de bytes) 000244[1] 1736263556.846212 mapwrite 0x0db000 thru 0x0db95d (0x95e bytes) 000247[1] 1736263556.897071 mapwrite 0x86e000 thru 0x87c9ae (0xe9af bytes) 000248[1] 1736263556.959677 write 0x4ec000 thru 0x4f3da5 (0x7da6 bytes) 000250[1] 1736263556.984772 trunc from 0x87c9af to 0x7089b4 (0x173ffa bytes) 000251[0] 1736263557.000437 trunc from 0x7089b5 to 0x196176 (0x57283e bytes) 000255[1] 1736263557.032854 trunc from 0x196177 to 0x51524d (0x37f0d7 bytes) 000256[0] 1736263557.063711 mapread 0x152000 thru 0x157544 (0x5545 bytes) 000257[0] 1736263557.112405 trunc from 0x51524e to 0x195329 (0x37ff24 bytes) 000258[0] 1736263557.134391 mapwrite 0x41c000 thru 0x42963a (0xd63b bytes) 000259[0] 1736263557.232003 mapread 0x099000 thru 0x0a1f6d (0x8f6e bytes) 000264[1] 1736263557.287456 write 0x582000 thru 0x585b4a (0x3b4b bytes) 000265[1] 1736263557.298534 mapread 0x1d3000 thru 0x1d8377 (0x5378 bytes) 000266[1] 1736263557.325956 trunc from 0x585b4b to 0x141574 (0x4445d6 bytes) 000268[1] 1736263557.352031 mapread 0x0b7000 thru 0x0c0b66 (0x9b67 bytes) 000269[0] 1736263557.379201 mapwrite 0x00b000 thru 0x01afed (0xffee bytes) 000270[1] 1736263557.479600 trunc from 0x141575 to 0x314240 (0x1d2ccc bytes) 000272[0] 1736263557.508476 read_OD 0x21c000 thru 0x21ffff (0x4000 bytes) 000273[1] 1736263557.519037 trunc from 0x314241 to 0x97eac7 (0x66a887 bytes) 000275[1] 1736263557.538723 mapread 0x0e2000 thru 0x0eba1a (0x9a1b bytes) 000277[0] 1736263557.577158 write_OD 0x06b000 thru 0x06efff (0x4000 bytes) 000278[1] 1736263557.626344 mapread 0x1ed000 thru 0x1f1182 (0x4183 bytes) 000282[0] 1736263557.648132 mapread 0x827000 thru 0x8282df (0x12e0 bytes) 000284[1] 1736263557.694312 trunc from 0x97eac8 to 0x51abac (0x463f1b bytes) 000285[0] 1736263557.729971 read_OD 0x12e000 thru 0x139fff (0xc000 bytes) 000288[1] 1736263557.746597 trunc from 0x51abad to 0x5ccd1a (0xb216e bytes) 000289[1] 1736263557.764853 mapwrite 0x0c3000 thru 0x0d12bf (0xe2c0 bytes) 000292[1] 1736263557.880388 write 0x686000 thru 0x69323f (0xd240 bytes) 000293[1] 1736263557.881736 write 0x74f000 thru 0x75bb59 (0xcb5a bytes) 000295[1] 1736263557.890132 mapwrite 0x80f000 thru 0x81b76c (0xc76d bytes) 000297[1] 1736263557.989149 write 0x1bd000 thru 0x1c043d (0x343e bytes) 000300[0] 1736263558.007161 trunc from 0x81b76d to 0x57c250 (0x29f51c bytes) 000301[0] 1736263558.032358 write_OD 0x52f000 thru 0x532fff (0x4000 bytes) 000303[0] 1736263558.073438 read_OD 0x0c4000 thru 0x0cbfff (0x8000 bytes) 000304[1] 1736263558.081351 read 0x0a2000 thru 0x0af5dc (0xd5dd bytes) 000305[1] 1736263558.109927 write 0x3c9000 thru 0x3cad49 (0x1d4a bytes) 000306[1] 1736263558.124266 mapread 0x14a000 thru 0x155d23 (0xbd24 bytes) 000308[0] 1736263558.176937 mapread 0x28d000 thru 0x297496 (0xa497 bytes) 000309[1] 1736263558.235124 mapread 0x25f000 thru 0x26b987 (0xc988 bytes) 000310[1] 1736263558.280988 read 0x27c000 thru 0x284f6d (0x8f6e bytes) 000311[1] 1736263558.304235 trunc from 0x57c251 to 0x8d1115 (0x354ec5 bytes) 000312[0] 1736263558.334444 mapread 0x36a000 thru 0x36f0b0 (0x50b1 bytes) 000314[1] 1736263558.362417 write 0x44b000 thru 0x44ccd5 (0x1cd6 bytes) 000315[0] 1736263558.393269 trunc from 0x8d1116 to 0x1c8138 (0x708fdd bytes) 000316[0] 1736263558.411127 mapread 0x05e000 thru 0x062e3a (0x4e3b bytes) 000317[0] 1736263558.426156 mapwrite 0x482000 thru 0x487cb4 (0x5cb5 bytes) 000318[0] 1736263558.500283 trunc from 0x487cb5 to 0x51fbf6 (0x97f42 bytes) 000319[0] 1736263558.511223 mapread 0x356000 thru 0x3579e3 (0x19e4 bytes) 000321[0] 1736263558.524192 mapwrite 0x6a2000 thru 0x6ad1a5 (0xb1a6 bytes) 000322[0] 1736263558.610342 mapread 0x55b000 thru 0x56aa4d (0xfa4e bytes) 000324[0] 1736263558.685327 mapwrite 0x826000 thru 0x82d820 (0x7821 bytes) 000325[1] 1736263558.758438 trunc from 0x82d821 to 0x498fbf (0x394861 bytes) 000326[1] 1736263558.776596 mapwrite 0x8f3000 thru 0x8fc420 (0x9421 bytes) 000329[0] 1736263558.837208 read_OD 0x890000 thru 0x890fff (0x1000 bytes) 000331[1] 1736263558.851068 mapwrite 0x7a4000 thru 0x7b223a (0xe23b bytes) 000332[1] 1736263558.928481 write 0x90b000 thru 0x90f485 (0x4486 bytes) 000333[0] 1736263558.939544 mapwrite 0x053000 thru 0x055198 (0x2199 bytes) 000335[0] 1736263559.005922 mapwrite 0x7a4000 thru 0x7a44a9 (0x4aa bytes) 000336[1] 1736263559.040638 write 0x732000 thru 0x73898a (0x698b bytes) 000338[0] 1736263559.071393 mapwrite 0x154000 thru 0x15b0e0 (0x70e1 bytes) 000344[1] 1736263559.181394 mapread 0x2db000 thru 0x2e2b46 (0x7b47 bytes) 000345[0] 1736263559.215142 trunc from 0x90f486 to 0x8150a4 (0xfa3e1 bytes) 000346[0] 1736263559.236114 write_OD 0x2bd000 thru 0x2c3fff (0x7000 bytes) 000347[1] 1736263559.265722 write 0x855000 thru 0x8609c2 (0xb9c3 bytes) 000349[1] 1736263559.278706 mapread 0x079000 thru 0x086935 (0xd936 bytes) 000353[0] 1736263559.336916 mapwrite 0x230000 thru 0x232b63 (0x2b64 bytes) 000354[0] 1736263559.393634 trunc from 0x8609c3 to 0x4d7e71 (0x388b51 bytes) 000355[1] 1736263559.403350 write 0x20f000 thru 0x21d913 (0xe914 bytes) 000356[0] 1736263559.414644 read_OD 0x458000 thru 0x461fff (0xa000 bytes) 000357[1] 1736263559.436466 trunc from 0x4d7e72 to 0x010c0f (0x4c7262 bytes) 000358[1] 1736263559.464198 write 0x004000 thru 0x0078ac (0x38ad bytes) 000359[0] 1736263559.474613 mapread 0x010000 thru 0x010a1c (0xa1d bytes) 000365[1] 1736263559.496248 mapread 0x00a000 thru 0x010034 (0x6035 bytes) 000367[1] 1736263559.518819 trunc from 0x010c10 to 0x449976 (0x438d67 bytes) 000368[1] 1736263559.534797 mapread 0x3a5000 thru 0x3ab9a7 (0x69a8 bytes) 000374[0] 1736263559.577065 mapwrite 0x461000 thru 0x4616d7 (0x6d8 bytes) 000377[1] 1736263559.640008 mapwrite 0x2fc000 thru 0x2ff711 (0x3712 bytes) 000378[1] 1736263559.702745 read 0x2c3000 thru 0x2c63fc (0x33fd bytes) 000380[0] 1736263559.728934 mapread 0x30a000 thru 0x312e89 (0x8e8a bytes) 000381[1] 1736263559.774121 write 0x0e1000 thru 0x0ea51d (0x951e bytes) 000382[1] 1736263559.792555 read 0x17f000 thru 0x1829b6 (0x39b7 bytes) 000383[1] 1736263559.810673 read 0x387000 thru 0x3908fe (0x98ff bytes) 000384[1] 1736263559.811851 trunc from 0x4616d8 to 0x32df50 (0x133787 bytes) 000388[0] 1736263559.851427 mapwrite 0x8cd000 thru 0x8d63e6 (0x93e7 bytes) 000393[1] 1736263559.989779 mapread 0x32c000 thru 0x336b40 (0xab41 bytes) 000394[1] 1736263560.039814 trunc from 0x8d63e7 to 0x7137c6 (0x1c2c20 bytes) 000396[0] 1736263560.051554 mapwrite 0x707000 thru 0x7110f7 (0xa0f8 bytes) 000398[1] 1736263560.128718 write 0x254000 thru 0x261872 (0xd873 bytes) 000399[1] 1736263560.147605 write 0x199000 thru 0x1a347d (0xa47e bytes) 000402[0] 1736263560.163609 write_OD 0x768000 thru 0x768fff (0x1000 bytes) 000403[1] 1736263560.193338 write 0x09d000 thru 0x0a4f38 (0x7f39 bytes) 000405[0] 1736263560.204970 trunc from 0x769000 to 0x8cc8ee (0x1638ef bytes) 000406[0] 1736263560.225772 trunc from 0x8cc8ef to 0x965425 (0x98b37 bytes) 000407[0] 1736263560.242231 mapread 0x8c4000 thru 0x8cc842 (0x8843 bytes) 000408[0] 1736263560.285110 read_OD 0x93c000 thru 0x947fff (0xc000 bytes) 000411[1] 1736263560.299335 mapwrite 0x50d000 thru 0x51b9b2 (0xe9b3 bytes) 000412[1] 1736263560.397457 mapwrite 0x7f8000 thru 0x804142 (0xc143 bytes) 000415[1] 1736263560.489332 trunc from 0x965426 to 0x296470 (0x6cefb5 bytes) 000416[0] 1736263560.513060 mapwrite 0x7fd000 thru 0x803938 (0x6939 bytes) 000418[1] 1736263560.608043 write 0x20c000 thru 0x2194be (0xd4bf bytes) 000419[1] 1736263560.628187 read 0x31c000 thru 0x3235f0 (0x75f1 bytes) 000422[0] 1736263560.661759 mapread 0x468000 thru 0x4775ef (0xf5f0 bytes) 000423[1] 1736263560.727798 mapread 0x026000 thru 0x030b2d (0xab2e bytes) 000424[1] 1736263560.758136 write 0x4f5000 thru 0x501d55 (0xcd56 bytes) 000425[0] 1736263560.773379 trunc from 0x803939 to 0x78c202 (0x77736 bytes) 000427[1] 1736263560.818976 trunc from 0x78c203 to 0x582253 (0x209faf bytes) 000429[1] 1736263560.845333 read 0x15b000 thru 0x168599 (0xd59a bytes) 000430[0] 1736263560.871631 write_OD 0x659000 thru 0x65efff (0x6000 bytes) 000433[0] 1736263560.931141 write_OD 0x5d9000 thru 0x5dbfff (0x3000 bytes) 000435[0] 1736263560.965376 trunc from 0x65f000 to 0x9bda0d (0x35ea0e bytes) 000439[1] 1736263561.004755 write 0x3ef000 thru 0x3f6edb (0x7edc bytes) 000441[1] 1736263561.032386 write 0x5a6000 thru 0x5a96d3 (0x36d4 bytes) 000444[0] 1736263561.049747 read_OD 0x58b000 thru 0x592fff (0x8000 bytes) 000445[1] 1736263561.083333 mapwrite 0x177000 thru 0x17f4f7 (0x84f8 bytes) 000446[0] 1736263561.177004 mapwrite 0x844000 thru 0x85148d (0xd48e bytes) 000447[1] 1736263561.277536 mapwrite 0x576000 thru 0x584f97 (0xef98 bytes) 000448[0] 1736263561.400974 write_OD 0x78e000 thru 0x794fff (0x7000 bytes) 000450[1] 1736263561.445605 write 0x539000 thru 0x540fc0 (0x7fc1 bytes) 000451[0] 1736263561.468183 mapread 0x75f000 thru 0x763f79 (0x4f7a bytes) 000452[0] 1736263561.497993 write_OD 0x3c5000 thru 0x3cafff (0x6000 bytes) 000453[1] 1736263561.539073 read 0x892000 thru 0x89fc9c (0xdc9d bytes) 000458[1] 1736263561.553824 mapwrite 0x0c8000 thru 0x0d293b (0xa93c bytes) 000461[0] 1736263561.642289 write_OD 0x21c000 thru 0x224fff (0x9000 bytes) 000462[1] 1736263561.695143 read 0x784000 thru 0x78e9b0 (0xa9b1 bytes) 000464[0] 1736263561.714081 mapwrite 0x434000 thru 0x43de0b (0x9e0c bytes) 000465[0] 1736263561.801843 read_OD 0x8a7000 thru 0x8b5fff (0xf000 bytes) 000466[0] 1736263561.815916 write_OD 0x431000 thru 0x435fff (0x5000 bytes) 000467[1] 1736263561.860049 read 0x53d000 thru 0x53d3c1 (0x3c2 bytes) 000469[1] 1736263561.898351 mapwrite 0x6b2000 thru 0x6be6a8 (0xc6a9 bytes) 000470[1] 1736263562.032599 read 0x775000 thru 0x7837a2 (0xe7a3 bytes) 000471[0] 1736263562.051806 mapread 0x0ce000 thru 0x0d57bb (0x77bc bytes) 000472[0] 1736263562.104241 read_OD 0x22a000 thru 0x234fff (0xb000 bytes) 000474[0] 1736263562.108915 mapread 0x84f000 thru 0x857a06 (0x8a07 bytes) 000477[0] 1736263562.141395 trunc from 0x9bda0e to 0x629c76 (0x393d97 bytes) 000479[0] 1736263562.194099 read_OD 0x3d0000 thru 0x3d7fff (0x8000 bytes) 000481[1] 1736263562.213215 mapread 0x39d000 thru 0x3aad9d (0xdd9e bytes) 000482[0] 1736263562.275103 mapwrite 0x835000 thru 0x841c09 (0xcc0a bytes) 000484[1] 1736263562.378436 mapread 0x662000 thru 0x66e7f7 (0xc7f8 bytes) 000488[0] 1736263562.438119 write_OD 0x91e000 thru 0x923fff (0x6000 bytes) 000489[0] 1736263562.481417 read_OD 0x071000 thru 0x073fff (0x3000 bytes) 000490[1] 1736263562.490231 mapwrite 0x4d4000 thru 0x4d6b7d (0x2b7e bytes) 000492[1] 1736263562.548847 mapread 0x738000 thru 0x73e73d (0x673e bytes) 000493[0] 1736263562.573278 mapwrite 0x976000 thru 0x9764fa (0x4fb bytes) 000494[1] 1736263562.667551 mapread 0x777000 thru 0x785650 (0xe651 bytes) 000495[0] 1736263562.746351 mapwrite 0x31a000 thru 0x327307 (0xd308 bytes) 000496[1] 1736263562.828576 mapwrite 0x5a0000 thru 0x5a1af1 (0x1af2 bytes) 000499[1] 1736263562.882597 read 0x26b000 thru 0x271546 (0x6547 bytes) All operations completed A-OK! PASS 16a (29s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 10:26:04 (1736263564) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.169898 s, 12.3 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.11912 s, 17.6 MB/s Chance of close/open is 1 in 50 Seed set to 3565 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x533eb5 000001[1] 1736263565.659772 trunc from 00000000 to 0x533eb4 (0x533eb5 bytes) 000005[1] 1736263565.667232 mapwrite 0x938d19 thru 0x93a370 (0x1658 bytes) 000007[0] 1736263565.761584 trunc from 0x93a371 to 0x532377 (0x407ff9 bytes) 000009[1] 1736263565.799047 read 0x07af65 thru 0x07c0a8 (0x1144 bytes) 000010[0] 1736263565.814739 read 0x055f0c thru 0x0569c6 (0xabb bytes) 000012[1] 1736263565.830323 trunc from 0x532378 to 0x38ae1d (0x1a755a bytes) 000014[0] 1736263565.861561 read 0x30f069 thru 0x30f4cb (0x463 bytes) 000015[0] 1736263565.888942 trunc from 0x38ae1e to 0x26f147 (0x11bcd6 bytes) 000019[0] 1736263565.938551 trunc from 0x26f148 to 0x48a21d (0x21b0d6 bytes) 000020[1] 1736263565.950279 read 0x2042c5 thru 0x2045f0 (0x32c bytes) 000024[0] 1736263565.982989 mapwrite 0x84d136 thru 0x84e3ca (0x1295 bytes) 000025[1] 1736263566.075764 read 0x331458 thru 0x33171f (0x2c8 bytes) 000028[1] 1736263566.121781 write 0x2433ad thru 0x243cd2 (0x926 bytes) 000031[1] 1736263566.183746 mapread 0x4b9d29 thru 0x4ba65a (0x932 bytes) 000033[1] 1736263566.193052 read 0x42be83 thru 0x42cc41 (0xdbf bytes) 000034[0] 1736263566.194200 mapwrite 0x0d35f6 thru 0x0d475f (0x116a bytes) 000036[0] 1736263566.256108 mapread 0x14d3f0 thru 0x14e3a3 (0xfb4 bytes) 000037[1] 1736263566.266588 mapwrite 0x16dd60 thru 0x16e19f (0x440 bytes) 000038[0] 1736263566.328773 read 0x0b9aa6 thru 0x0baa45 (0xfa0 bytes) 000039[0] 1736263566.347482 mapwrite 0x16d72e thru 0x16d75c (0x2f bytes) 000041[0] 1736263566.449714 mapread 0x5c5da6 thru 0x5c7d6c (0x1fc7 bytes) 000043[0] 1736263566.456676 mapwrite 0x90f2f3 thru 0x9101d1 (0xedf bytes) 000044[0] 1736263566.500618 mapread 0x20c0b1 thru 0x20cb07 (0xa57 bytes) 000045[0] 1736263566.501009 write 0x52d322 thru 0x52dd45 (0xa24 bytes) 000046[0] 1736263566.501471 mapwrite 0x07b49d thru 0x07c9dd (0x1541 bytes) 000047[1] 1736263566.558611 write 0x4c925c thru 0x4cade8 (0x1b8d bytes) 000048[0] 1736263566.581318 mapwrite 0x6cba81 thru 0x6cd58e (0x1b0e bytes) 000049[1] 1736263566.624830 read 0x61d1ac thru 0x61d965 (0x7ba bytes) 000052[0] 1736263566.665413 write 0x120c17 thru 0x12195d (0xd47 bytes) 000055[0] 1736263566.687085 read 0x7bae30 thru 0x7bcd85 (0x1f56 bytes) 000060[0] 1736263566.704528 read 0x1c9618 thru 0x1ca7c6 (0x11af bytes) 000061[1] 1736263566.705377 write 0x8edc08 thru 0x8ef49f (0x1898 bytes) 000062[1] 1736263566.755374 read 0x57a169 thru 0x57b0ec (0xf84 bytes) 000063[1] 1736263566.792353 mapread 0x700a89 thru 0x7017bf (0xd37 bytes) 000066[0] 1736263566.800246 read 0x2fa257 thru 0x2fb2fc (0x10a6 bytes) 000067[0] 1736263566.855386 mapread 0x5c81c1 thru 0x5c8dd5 (0xc15 bytes) 000069[1] 1736263566.859729 mapwrite 0x065617 thru 0x067447 (0x1e31 bytes) 000071[1] 1736263566.931363 mapwrite 0x2bece1 thru 0x2c0c57 (0x1f77 bytes) 000072[1] 1736263566.993139 read 0x1d645d thru 0x1d67dc (0x380 bytes) 000073[1] 1736263567.029538 mapread 0x443bb9 thru 0x4453d6 (0x181e bytes) 000076[0] 1736263567.089438 mapwrite 0x2b43c5 thru 0x2b63c2 (0x1ffe bytes) 000077[1] 1736263567.154847 mapread 0x6d2858 thru 0x6d3382 (0xb2b bytes) 000079[1] 1736263567.168736 mapread 0x112ff1 thru 0x113b19 (0xb29 bytes) 000080[0] 1736263567.178780 write 0x749f0c thru 0x74b62e (0x1723 bytes) 000082[1] 1736263567.213191 write 0x7dacc8 thru 0x7dc7e0 (0x1b19 bytes) 000083[1] 1736263567.229494 trunc from 0x9101d2 to 0x0d1f27 (0x83e2aa bytes) 000085[0] 1736263567.260191 write 0x0f041a thru 0x0f070b (0x2f2 bytes) 000086[0] 1736263567.270667 write 0x5bae27 thru 0x5bc2d7 (0x14b1 bytes) 000087[1] 1736263567.276679 mapread 0x435586 thru 0x435cb4 (0x72f bytes) 000090[0] 1736263567.305996 mapwrite 0x3cafe9 thru 0x3cb441 (0x459 bytes) 000091[1] 1736263567.362535 mapwrite 0x665e2e thru 0x667b26 (0x1cf9 bytes) truncating to largest ever: 0x815822 000095[0] 1736263567.473274 trunc from 0x667b27 to 0x815821 (0x1adcfb bytes) 000100[0] 1736263567.500440 read 0x7230aa thru 0x724d82 (0x1cd9 bytes) 000107[1] 1736263567.539705 mapwrite 0x697cf3 thru 0x697e6d (0x17b bytes) 000109[0] 1736263567.603899 read 0x7921f9 thru 0x792c9a (0xaa2 bytes) 000111[1] 1736263567.623762 mapwrite 0x40f7a2 thru 0x410973 (0x11d2 bytes) 000115[0] 1736263567.724423 mapwrite 0x9ecc8e thru 0x9ecf9d (0x310 bytes) 000116[0] 1736263567.777344 read 0x9becea thru 0x9c0a4e (0x1d65 bytes) 000123[1] 1736263567.840759 mapread 0x91adf9 thru 0x91ccca (0x1ed2 bytes) 000126[0] 1736263567.884066 mapread 0x666a54 thru 0x6677f3 (0xda0 bytes) 000127[1] 1736263567.900845 mapwrite 0x25e05b thru 0x25fd24 (0x1cca bytes) 000129[0] 1736263567.982604 mapread 0x84cacb thru 0x84e604 (0x1b3a bytes) 000131[1] 1736263568.009334 mapwrite 0x3e41fd thru 0x3e603b (0x1e3f bytes) 000132[0] 1736263568.055174 trunc from 0x9ecf9e to 0x7a6c71 (0x24632c bytes) 000134[1] 1736263568.081777 mapread 0x70a642 thru 0x70b0c2 (0xa81 bytes) 000135[1] 1736263568.093874 mapread 0x417f8a thru 0x419315 (0x138c bytes) 000136[0] 1736263568.106247 write 0x9bb2a5 thru 0x9bb75d (0x4b9 bytes) 000138[0] 1736263568.128904 mapread 0x50e71b thru 0x50fba3 (0x1489 bytes) truncating to largest ever: 0x8765c8 000139[0] 1736263568.136600 trunc from 0x9bb75e to 0x8765c7 (0x145196 bytes) 000140[1] 1736263568.159622 write 0x51597c thru 0x516834 (0xeb9 bytes) 000141[1] 1736263568.183168 trunc from 0x8765c8 to 0x57244f (0x304178 bytes) 000142[1] 1736263568.203155 mapwrite 0x5d8c83 thru 0x5d8cd8 (0x56 bytes) 000144[1] 1736263568.242276 mapread 0x291fbd thru 0x2926ad (0x6f1 bytes) 000145[1] 1736263568.248899 write 0x1ccaf7 thru 0x1ce064 (0x156e bytes) 000146[0] 1736263568.257350 mapwrite 0x3bae48 thru 0x3bbf67 (0x1120 bytes) 000147[1] 1736263568.331641 trunc from 0x5d8cd9 to 0x815ce8 (0x23d010 bytes) 000148[0] 1736263568.354840 write 0x108c33 thru 0x10a621 (0x19ef bytes) 000150[0] 1736263568.387953 trunc from 0x815ce9 to 0x3c166e (0x45467a bytes) 000154[0] 1736263568.401246 mapwrite 0x386f77 thru 0x3884a5 (0x152f bytes) 000155[0] 1736263568.447873 mapwrite 0x49811e thru 0x49878d (0x670 bytes) 000156[1] 1736263568.493952 mapread 0x42f121 thru 0x430d33 (0x1c13 bytes) 000157[0] 1736263568.519246 mapread 0x3cb8bd thru 0x3ccfff (0x1743 bytes) 000159[0] 1736263568.542343 mapread 0x190683 thru 0x191ee4 (0x1862 bytes) 000160[1] 1736263568.549555 mapread 0x2206da thru 0x22214d (0x1a74 bytes) truncating to largest ever: 0x9540de 000162[0] 1736263568.574065 trunc from 0x49878e to 0x9540dd (0x4bb950 bytes) 000163[0] 1736263568.595461 write 0x1233c4 thru 0x12460b (0x1248 bytes) 000166[0] 1736263568.614691 mapread 0x0b9ca1 thru 0x0bb20e (0x156e bytes) 000168[1] 1736263568.634574 read 0x83ec22 thru 0x83f90b (0xcea bytes) 000169[0] 1736263568.661159 mapread 0x8554d5 thru 0x85736a (0x1e96 bytes) 000170[1] 1736263568.683181 write 0x163dd4 thru 0x164521 (0x74e bytes) 000172[1] 1736263568.715297 write 0x4dcd31 thru 0x4ddb2e (0xdfe bytes) 000173[0] 1736263568.723316 mapread 0x793269 thru 0x795245 (0x1fdd bytes) 000174[0] 1736263568.759274 mapwrite 0x915a06 thru 0x917296 (0x1891 bytes) 000175[1] 1736263568.825529 write 0x152a45 thru 0x153d14 (0x12d0 bytes) 000176[1] 1736263568.835667 write 0x0ac8e0 thru 0x0acbf8 (0x319 bytes) 000178[1] 1736263568.839266 write 0x86d688 thru 0x86effc (0x1975 bytes) 000179[0] 1736263568.846981 trunc from 0x9540de to 0x537a7f (0x41c65e bytes) 000180[1] 1736263568.881012 mapread 0x141ee4 thru 0x14372e (0x184b bytes) 000181[1] 1736263568.913814 mapwrite 0x0fa530 thru 0x0fc4b0 (0x1f81 bytes) 000184[0] 1736263568.979752 write 0x6f1151 thru 0x6f1155 (0x5 bytes) 000185[0] 1736263568.986759 trunc from 0x6f1156 to 0x7cf41e (0xde2c9 bytes) 000190[1] 1736263569.037009 read 0x38deaa thru 0x38e94e (0xaa5 bytes) 000191[0] 1736263569.058806 mapwrite 0x89c2ab thru 0x89d519 (0x126f bytes) 000192[1] 1736263569.137623 mapread 0x87e88d thru 0x87f002 (0x776 bytes) 000194[1] 1736263569.167761 write 0x5996d0 thru 0x59b205 (0x1b36 bytes) 000200[1] 1736263569.212118 mapwrite 0x29343b thru 0x293b42 (0x708 bytes) 000203[1] 1736263569.261102 mapread 0x666185 thru 0x6672c3 (0x113f bytes) 000205[0] 1736263569.277657 mapread 0x082079 thru 0x08258d (0x515 bytes) 000208[0] 1736263569.310392 trunc from 0x89d51a to 0x469844 (0x433cd5 bytes) 000209[1] 1736263569.331345 trunc from 0x469845 to 0x1da0c4 (0x28f780 bytes) 000210[0] 1736263569.346999 read 0x0fb6c0 thru 0x0fd22d (0x1b6e bytes) 000212[1] 1736263569.383587 read 0x1a6fd9 thru 0x1a80d8 (0x1100 bytes) 000213[0] 1736263569.399633 read 0x155db5 thru 0x15778f (0x19db bytes) 000214[0] 1736263569.418755 read 0x0ab9d1 thru 0x0ad217 (0x1847 bytes) 000216[1] 1736263569.426142 trunc from 0x1da0c5 to 0x20e456 (0x34392 bytes) 000217[1] 1736263569.451285 write 0x7e5e5f thru 0x7e649e (0x640 bytes) 000218[0] 1736263569.453033 write 0x1923e2 thru 0x19317a (0xd99 bytes) 000223[1] 1736263569.519178 mapwrite 0x7ea97e thru 0x7ec342 (0x19c5 bytes) 000224[0] 1736263569.611349 write 0x0fb402 thru 0x0fcba5 (0x17a4 bytes) 000225[0] 1736263569.644750 trunc from 0x7ec343 to 0x4f6de5 (0x2f555d bytes) 000227[1] 1736263569.665656 mapwrite 0x80c93b thru 0x80e8ed (0x1fb3 bytes) 000228[1] 1736263569.729973 read 0x7ffff5 thru 0x801e7c (0x1e88 bytes) 000229[0] 1736263569.741868 trunc from 0x80e8ee to 0x38b658 (0x483295 bytes) 000232[0] 1736263569.777949 mapread 0x070c93 thru 0x071007 (0x375 bytes) 000233[0] 1736263569.792530 mapread 0x08c780 thru 0x08d77d (0xffe bytes) 000234[1] 1736263569.804273 trunc from 0x38b659 to 0x867c50 (0x4dc5f8 bytes) 000235[1] 1736263569.830991 mapread 0x7828cb thru 0x782985 (0xbb bytes) 000237[0] 1736263569.838948 write 0x120418 thru 0x121bf5 (0x17de bytes) 000239[0] 1736263569.875866 mapwrite 0x4fd23e thru 0x4fdaf3 (0x8b6 bytes) 000240[0] 1736263569.934867 write 0x6bd3ad thru 0x6bdcb6 (0x90a bytes) 000242[0] 1736263569.941814 mapwrite 0x76c6cc thru 0x76d6f3 (0x1028 bytes) 000244[0] 1736263569.990289 mapwrite 0x23e3eb thru 0x23eb50 (0x766 bytes) 000245[1] 1736263570.048828 trunc from 0x867c51 to 0x136edc (0x730d74 bytes) 000246[1] 1736263570.073018 mapwrite 0x4e9184 thru 0x4e9492 (0x30f bytes) 000247[1] 1736263570.139122 trunc from 0x4e9493 to 0x6f2971 (0x2094df bytes) 000250[0] 1736263570.177498 mapwrite 0x538f20 thru 0x539d43 (0xe24 bytes) 000252[0] 1736263570.245150 write 0x1f30cd thru 0x1f5065 (0x1f99 bytes) 000253[1] 1736263570.266830 write 0x46f0c4 thru 0x4708a6 (0x17e3 bytes) 000254[0] 1736263570.277485 read 0x19e113 thru 0x19e51b (0x409 bytes) 000255[1] 1736263570.294023 read 0x19421b thru 0x194564 (0x34a bytes) 000256[1] 1736263570.349128 mapwrite 0x5821e3 thru 0x5829f3 (0x811 bytes) 000258[1] 1736263570.423717 mapwrite 0x18f51d thru 0x191462 (0x1f46 bytes) 000259[0] 1736263570.462182 write 0x5b1af1 thru 0x5b1c24 (0x134 bytes) 000261[1] 1736263570.487026 write 0x100556 thru 0x102395 (0x1e40 bytes) 000266[0] 1736263570.527774 read 0x4f83f8 thru 0x4f871b (0x324 bytes) 000267[0] 1736263570.551998 write 0x892223 thru 0x8930df (0xebd bytes) 000270[1] 1736263570.575317 write 0x980ba7 thru 0x9824ca (0x1924 bytes) 000271[0] 1736263570.586510 write 0x342997 thru 0x3440b6 (0x1720 bytes) 000272[1] 1736263570.621190 mapread 0x781180 thru 0x7825c6 (0x1447 bytes) 000273[1] 1736263570.641975 mapwrite 0x164885 thru 0x16659b (0x1d17 bytes) 000276[0] 1736263570.715083 mapread 0x321173 thru 0x3220dc (0xf6a bytes) 000277[0] 1736263570.732829 read 0x227954 thru 0x228ee7 (0x1594 bytes) 000278[1] 1736263570.771307 trunc from 0x9824cb to 0x924934 (0x5db96 bytes) 000279[0] 1736263570.808693 write 0x090bba thru 0x092625 (0x1a6c bytes) 000281[1] 1736263570.835996 write 0x95310e thru 0x9535a8 (0x49b bytes) 000282[0] 1736263570.844184 write 0x1f47bc thru 0x1f6401 (0x1c46 bytes) 000283[0] 1736263570.866823 read 0x8f3b0b thru 0x8f4a4a (0xf40 bytes) 000284[0] 1736263570.873308 write 0x31770a thru 0x317bc2 (0x4b9 bytes) 000288[0] 1736263570.896421 mapread 0x7dfdec thru 0x7e08b7 (0xacc bytes) 000290[1] 1736263570.911789 read 0x0cbe56 thru 0x0cd0b1 (0x125c bytes) 000292[1] 1736263570.939606 trunc from 0x9535a9 to 0x2a8f19 (0x6aa68f bytes) 000293[1] 1736263570.962565 mapread 0x1fea25 thru 0x1feadf (0xbb bytes) 000294[1] 1736263570.979380 read 0x16d964 thru 0x16ef60 (0x15fd bytes) 000297[1] 1736263570.991850 mapwrite 0x5d7295 thru 0x5d7f84 (0xcf0 bytes) 000300[1] 1736263571.045371 read 0x49b8d1 thru 0x49ced0 (0x1600 bytes) 000301[1] 1736263571.064233 read 0x2274f2 thru 0x227cf1 (0x800 bytes) 000302[1] 1736263571.064647 mapwrite 0x5518cd thru 0x551e06 (0x53a bytes) 000305[1] 1736263571.136282 mapwrite 0x2694aa thru 0x269a43 (0x59a bytes) 000306[1] 1736263571.166277 mapwrite 0x59b888 thru 0x59d60e (0x1d87 bytes) 000309[0] 1736263571.211547 mapread 0x0a4d67 thru 0x0a59f9 (0xc93 bytes) truncating to largest ever: 0x9fb655 000311[0] 1736263571.237648 trunc from 0x5d7f85 to 0x9fb654 (0x4236d0 bytes) 000312[1] 1736263571.266300 write 0x8cb7ea thru 0x8cc3b4 (0xbcb bytes) 000316[1] 1736263571.309999 mapread 0x6a1398 thru 0x6a1b63 (0x7cc bytes) 000317[1] 1736263571.323514 mapread 0x14949e thru 0x14a743 (0x12a6 bytes) 000319[0] 1736263571.334282 mapread 0x166674 thru 0x167ea5 (0x1832 bytes) 000320[0] 1736263571.341871 mapread 0x8d5d08 thru 0x8d63de (0x6d7 bytes) 000321[1] 1736263571.357864 read 0x226f10 thru 0x228a5e (0x1b4f bytes) 000324[1] 1736263571.402365 mapwrite 0x5fc253 thru 0x5fd9eb (0x1799 bytes) 000325[1] 1736263571.455594 write 0x410b67 thru 0x412219 (0x16b3 bytes) 000327[1] 1736263571.473764 mapwrite 0x311478 thru 0x3116dc (0x265 bytes) 000328[0] 1736263571.505625 mapread 0x2c4bf5 thru 0x2c53bf (0x7cb bytes) 000330[0] 1736263571.531057 trunc from 0x9fb655 to 0x36a405 (0x69124f bytes) 000332[0] 1736263571.547023 trunc from 0x36a406 to 0x4bc362 (0x151f5d bytes) 000333[1] 1736263571.554372 read 0x4b28eb thru 0x4b3b3d (0x1253 bytes) 000335[1] 1736263571.574535 mapread 0x0de5b7 thru 0x0df0ff (0xb49 bytes) 000336[1] 1736263571.582583 trunc from 0x4bc363 to 0x0271ce (0x495194 bytes) 000340[0] 1736263571.610727 write 0x0eb855 thru 0x0ed241 (0x19ed bytes) 000343[1] 1736263571.639848 read 0x02d613 thru 0x02ebec (0x15da bytes) 000347[0] 1736263571.674525 mapread 0x0e7371 thru 0x0e8591 (0x1221 bytes) 000348[1] 1736263571.683859 trunc from 0x0ed242 to 0x783f77 (0x696d36 bytes) 000349[1] 1736263571.696797 trunc from 0x783f78 to 0x0ce0fb (0x6b5e7c bytes) 000350[1] 1736263571.703835 mapread 0x0105c5 thru 0x010f81 (0x9bd bytes) 000356[1] 1736263571.762713 read 0x0a7043 thru 0x0a73cc (0x38a bytes) 000357[1] 1736263571.774904 read 0x05d510 thru 0x05f19d (0x1c8e bytes) 000358[1] 1736263571.783147 mapwrite 0x258da6 thru 0x25a372 (0x15cd bytes) 000359[1] 1736263571.833439 mapwrite 0x0b9e24 thru 0x0bb444 (0x1621 bytes) 000360[0] 1736263571.883478 mapread 0x257865 thru 0x258f71 (0x170d bytes) 000366[0] 1736263571.940865 trunc from 0x25a373 to 0x9187ec (0x6be47a bytes) 000367[0] 1736263571.961806 mapread 0x1a1cd2 thru 0x1a2398 (0x6c7 bytes) 000368[0] 1736263571.966291 trunc from 0x9187ed to 0x558fbf (0x3bf82d bytes) 000369[1] 1736263571.976809 trunc from 0x558fc0 to 0x979d06 (0x420d47 bytes) 000370[0] 1736263572.001031 mapread 0x6631e1 thru 0x664d5b (0x1b7b bytes) 000374[0] 1736263572.044668 read 0x8779c4 thru 0x8789e9 (0x1026 bytes) 000375[0] 1736263572.048221 write 0x05d3a1 thru 0x05d510 (0x170 bytes) 000377[1] 1736263572.073222 write 0x77c721 thru 0x77dc93 (0x1573 bytes) 000378[0] 1736263572.091180 trunc from 0x979d07 to 0x76d76f (0x20c597 bytes) 000379[0] 1736263572.118780 read 0x6a52b9 thru 0x6a683c (0x1584 bytes) 000380[0] 1736263572.136618 mapread 0x6b4ca2 thru 0x6b5daf (0x110e bytes) 000383[0] 1736263572.161340 trunc from 0x76d770 to 0x4b0036 (0x2bd739 bytes) 000384[1] 1736263572.176795 mapwrite 0x451397 thru 0x452a8b (0x16f5 bytes) 000385[1] 1736263572.253551 mapread 0x0447b0 thru 0x044a88 (0x2d9 bytes) 000386[1] 1736263572.267382 write 0x99b078 thru 0x99c5d0 (0x1559 bytes) 000387[1] 1736263572.274236 write 0x4c3ba3 thru 0x4c5aae (0x1f0c bytes) 000388[1] 1736263572.280491 write 0x8d8c64 thru 0x8d957d (0x91a bytes) 000389[0] 1736263572.283300 write 0x3c35f7 thru 0x3c5250 (0x1c5a bytes) 000390[0] 1736263572.296567 mapwrite 0x7f6265 thru 0x7f7d35 (0x1ad1 bytes) 000393[0] 1736263572.378207 mapwrite 0x664cd6 thru 0x665ab8 (0xde3 bytes) 000397[0] 1736263572.446405 trunc from 0x99c5d1 to 0x0c8348 (0x8d4288 bytes) 000401[0] 1736263572.506893 mapwrite 0x7d98cb thru 0x7db5b5 (0x1ceb bytes) 000403[1] 1736263572.582810 write 0x84df27 thru 0x84f757 (0x1831 bytes) 000404[1] 1736263572.599959 read 0x42101f thru 0x421f63 (0xf45 bytes) 000405[0] 1736263572.622359 trunc from 0x84f758 to 0x9ad6bf (0x15df68 bytes) 000407[1] 1736263572.656735 mapwrite 0x64fc6d thru 0x650395 (0x729 bytes) 000408[1] 1736263572.725969 mapread 0x4a4f17 thru 0x4a5195 (0x27f bytes) 000409[0] 1736263572.746155 trunc from 0x9ad6c0 to 0x3596b0 (0x65400f bytes) 000410[1] 1736263572.764122 read 0x04d574 thru 0x04e72b (0x11b8 bytes) 000415[1] 1736263572.841799 mapwrite 0x3d0a78 thru 0x3d13a0 (0x929 bytes) 000416[0] 1736263572.960837 mapwrite 0x33497e thru 0x33536f (0x9f2 bytes) 000419[0] 1736263573.065520 read 0x23b841 thru 0x23b8ad (0x6d bytes) 000420[1] 1736263573.086091 trunc from 0x3d13a1 to 0x145057 (0x28c349 bytes) truncating to largest ever: 0x9ff03e 000421[1] 1736263573.122252 trunc from 0x145058 to 0x9ff03d (0x8b9fe6 bytes) 000424[1] 1736263573.145740 mapread 0x68b52c thru 0x68bf64 (0xa39 bytes) 000425[0] 1736263573.160297 write 0x1206f2 thru 0x1214b7 (0xdc6 bytes) 000435[0] 1736263573.264531 mapwrite 0x405e6f thru 0x4078f3 (0x1a85 bytes) 000436[0] 1736263573.356433 trunc from 0x9ff03e to 0x0126dd (0x9ec960 bytes) 000437[1] 1736263573.375210 read 0x010e5f thru 0x011c3a (0xddc bytes) 000441[1] 1736263573.397756 mapread 0x007fd8 thru 0x0097c7 (0x17f0 bytes) 000442[1] 1736263573.415226 write 0x3e4581 thru 0x3e61b1 (0x1c31 bytes) 000443[1] 1736263573.425244 read 0x1d4934 thru 0x1d554c (0xc19 bytes) 000446[1] 1736263573.462597 trunc from 0x3e61b2 to 0x48bf30 (0xa5d7f bytes) 000448[0] 1736263573.500614 write 0x2af294 thru 0x2b0250 (0xfbd bytes) 000450[1] 1736263573.531142 read 0x0fd2fe thru 0x0fd49f (0x1a2 bytes) 000453[1] 1736263573.574924 trunc from 0x48bf31 to 0x45a0ce (0x31e62 bytes) 000454[0] 1736263573.598368 mapwrite 0x5dfa1c thru 0x5e096c (0xf51 bytes) 000455[0] 1736263573.675088 mapwrite 0x46d98c thru 0x46e7d0 (0xe45 bytes) 000456[0] 1736263573.736002 read 0x2a8609 thru 0x2a97e9 (0x11e1 bytes) 000458[0] 1736263573.774509 mapread 0x3b768a thru 0x3b866a (0xfe1 bytes) 000459[1] 1736263573.784586 trunc from 0x5e096d to 0x5898dd (0x5708f bytes) 000460[1] 1736263573.819072 trunc from 0x5898de to 0x132a6b (0x456e72 bytes) 000462[1] 1736263573.869200 write 0x937d1a thru 0x939017 (0x12fe bytes) 000463[1] 1736263573.880469 mapread 0x8f7e01 thru 0x8f81f5 (0x3f5 bytes) 000468[0] 1736263573.907575 trunc from 0x939018 to 0x770e63 (0x1c81b4 bytes) 000469[0] 1736263573.931822 write 0x7a468f thru 0x7a4cd4 (0x646 bytes) 000470[1] 1736263573.932639 mapwrite 0x77bea6 thru 0x77de00 (0x1f5b bytes) 000472[0] 1736263574.000212 write 0x0c6a92 thru 0x0c85f8 (0x1b67 bytes) 000479[1] 1736263574.099252 read 0x26dcbc thru 0x26f920 (0x1c65 bytes) 000481[0] 1736263574.134230 read 0x4fff9b thru 0x5003c5 (0x42b bytes) 000483[0] 1736263574.178524 trunc from 0x7a4cd5 to 0x2a61bc (0x4feb18 bytes) 000484[0] 1736263574.212231 mapread 0x04dfb7 thru 0x04ff39 (0x1f83 bytes) 000485[0] 1736263574.228975 read 0x20226f thru 0x2039a3 (0x1735 bytes) 000486[0] 1736263574.243022 read 0x16374a thru 0x164084 (0x93b bytes) 000488[1] 1736263574.253153 mapread 0x1dd79a thru 0x1df13b (0x19a2 bytes) 000489[0] 1736263574.283481 mapread 0x1ec3eb thru 0x1ee1cb (0x1de1 bytes) 000490[0] 1736263574.299899 mapread 0x0815af thru 0x082876 (0x12c8 bytes) 000491[0] 1736263574.312167 mapread 0x22db8b thru 0x22fa06 (0x1e7c bytes) 000493[1] 1736263574.337342 trunc from 0x2a61bd to 0x5098b0 (0x2636f4 bytes) 000495[0] 1736263574.365297 mapread 0x0aa81b thru 0x0abdf5 (0x15db bytes) 000497[0] 1736263574.393907 trunc from 0x5098b1 to 0x2bbe7b (0x24da35 bytes) 000498[1] 1736263574.413195 read 0x06086b thru 0x061e3c (0x15d2 bytes) All operations completed A-OK! PASS 16b (11s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 10:26:15 (1736263575) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (2s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 10:26:17 (1736263577) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.135493 s, 15.5 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.130205 s, 16.1 MB/s ldlm.namespaces.MGC192.168.203.140@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b3eee800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b652f000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b3eee800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b652f000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b3eee800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b652f000.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.02015 s, 20.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.10175 s, 95.2 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.734015 s, 143 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.14344 s, 48.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.11026 s, 25.5 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.23674 s, 84.8 MB/s PASS 16d (18s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 10:26:35 (1736263595) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0839314 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0574682 s, 18.2 MB/s PASS 16e (2s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 10:26:37 (1736263597) TIMEOUT PASS 16f (22s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 10:26:59 (1736263619) TIMEOUT PASS 16g (22s) == sanityn test 16i: read after truncate file ============ 10:27:21 (1736263641) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (2s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 10:27:23 (1736263643) fail_loc=0x8000030a PASS 17 (4s) == sanityn test 18: mmap sanity check =========================================================================================== 10:27:27 (1736263647) mmap test1: basic mmap operation (PASS, 0.024174s) mmap test2: MAP_PRIVATE not write back (PASS, 0.014391s) mmap test3: concurrent mmap ops on two nodes (PASS, 3.6282s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.0894s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.033596s) mmap test6: check mmap write/read content on two nodes (PASS, 0.042021s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0169s) PASS 18 (9s) == sanityn test 19: test concurrent uncached read races ========================================================================= 10:27:36 (1736263656) oleg340-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg340-client: oleg340-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 10:27:37 (1736263657) PASS 20 (2s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 10:27:39 (1736263659) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (1s) == sanityn test 23: others should see updated atime while another read============================================================== 10:27:40 (1736263660) now is 1736263661 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6871 new atime is 1736263722 PASS 23 (63s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 10:28:43 (1736263723) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 9216 3760128 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 15360 7523328 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 9216 3760128 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 15360 7523328 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 376 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.1K 368 229.7K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 376 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 368 229.7K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 9.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 15.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117944 376 117568 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235600 368 235232 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 9216 3760128 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 15360 7523328 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 376 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 368 229.7K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 117944 376 117568 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre[OST:1] filesystem_summary: 235600 368 235232 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117944 376 117568 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235600 368 235232 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 9216 3760128 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 15360 7523328 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 9216 3760128 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 15360 7523328 1% /mnt/lustre2 PASS 24a (2s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 10:28:45 (1736263725) PASS 24b (1s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 10:28:46 (1736263726) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (2s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 10:28:48 (1736263728) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 10:28:49 (1736263729) utime: good mknod times 1736263728 <= 1736263729 <= 1736263729 for /mnt/lustre/f26a utime: good mknod times 1736263728 <= 1736263729 <= 1736263729 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (2s) == sanityn test 26b: sync mtime between ost and mds ====== 10:28:51 (1736263731) PASS 26b (4s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 10:28:56 (1736263736) 0 PASS 30 (3s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 10:28:59 (1736263739) fail_loc=0x314 PASS 31a (3s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 10:29:02 (1736263742) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg340-server mds-ost sync done. sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.203.140@tcp:/lustre 7542784 16384 7520256 1% /mnt/lustre2 PASS 31b (11s) == sanityn test 31r: open-rename(replace) race =========== 10:29:13 (1736263753) -rw-r--r-- 1 root root 0 Jan 7 10:29 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 mv: failed to access '/mnt/lustre2/f31r.sanityn.target': Input/output error PASS 31r (5s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 10:29:18 (1736263758) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 10:29:19 (1736263759) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 10:29:20 (1736263760) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 10:29:21 (1736263761) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (0s) == sanityn test 34: no lock timeout under IO ============= 10:29:21 (1736263761) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg340-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b3eee800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b652f000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800b3eee800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8800b652f000.ost_server_uuid in FULL state after 0 sec oleg340-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b3eee800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b652f000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800b3eee800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800b652f000.ost_server_uuid in FULL state after 0 sec oleg340-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b3eee800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b652f000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800b3eee800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8800b652f000.ost_server_uuid in IDLE state after 0 sec oleg340-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b3eee800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b652f000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800b3eee800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8800b652f000.ost_server_uuid in FULL state after 0 sec oleg340-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b3eee800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b652f000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800b3eee800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8800b652f000.ost_server_uuid in IDLE state after 0 sec oleg340-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b3eee800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b652f000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800b3eee800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8800b652f000.ost_server_uuid in FULL state after 0 sec PASS 34 (72s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 10:30:33 (1736263833) fail_loc=0x80000317 Race attempt 0 Wait for 15463 15602 for 60 sec... - open/close 3594 (time 1736263844.78 total 10.00 last 359.39) total: 4000 open/close in 11.02 seconds: 362.93 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.203.140@tcp:/lustre 7.2G 17M 7.2G 1% /mnt/lustre 192.168.203.140@tcp:/lustre 7.2G 17M 7.2G 1% /mnt/lustre2 PASS 35 (63s) == sanityn test 36: handle ESTALE/open-unlink correctly == 10:31:36 (1736263896) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg340-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.311832 s, 168 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6871 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7521280) after_dd(7470080) after(7521280) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.326908 s, 160 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6871 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7521280) after_dd(7470080) after(7521280) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.32494 s, 161 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6871 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7521280) after_dd(7470080) after(7521280) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.307991 s, 170 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6871 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7521280) after_dd(7467008) after(7518208) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.315467 s, 166 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6871 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7518208) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.28894 s, 181 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6871 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7522304) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.262986 s, 199 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6871 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7522304) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.270695 s, 194 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6871 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7522304) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.270276 s, 194 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6871 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7522304) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.314851 s, 167 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6871 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7522304) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.32271 s, 162 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6871 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7522304) after_dd(7471104) after(7522304) PASS 36 (228s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 10:35:24 (1736264124) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6871 total: 10000 create in 8.91 seconds: 1122.01 ops/second PASS 37 (11s) == sanityn test 39a: test from 11063 ============================================================================================ 10:35:35 (1736264135) repeat after cancel_lru_locks PASS 39a (1s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 10:35:37 (1736264137) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 10:35:39 (1736264139) repeat after cancel_lru_locks PASS 39c (2s) == sanityn test 39d: sync write should update mtime ====== 10:35:41 (1736264141) fail_loc=0x411 fail_loc=0 PASS 39d (2s) == sanityn test 40a: pdirops: create vs others ======================================================================== 10:35:43 (1736264143) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (7s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 10:35:50 (1736264150) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (7s) == sanityn test 40c: pdirops: link and others ======================================================================== 10:35:57 (1736264157) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (6s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 10:36:03 (1736264163) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (7s) == sanityn test 40e: pdirops: rename and others ======================================================================== 10:36:10 (1736264170) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (7s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 10:36:17 (1736264177) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (4s) == sanityn test 41b: pdirops: create vs create ======================================================================== 10:36:21 (1736264181) open: File exists Conflict PASS 41b (4s) == sanityn test 41c: pdirops: create vs link ======================================================================== 10:36:25 (1736264185) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (4s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 10:36:29 (1736264189) Conflict PASS 41d (5s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 10:36:34 (1736264194) Conflict PASS 41e (4s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 10:36:38 (1736264198) Conflict PASS 41f (4s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 10:36:42 (1736264202) Conflict PASS 41g (4s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 10:36:46 (1736264206) Conflict PASS 41h (4s) == sanityn test 41i: reint_open: create vs create ======== 10:36:50 (1736264210) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (648s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 10:47:38 (1736264858) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (4s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 10:47:42 (1736264862) open: File exists Conflict PASS 42b (4s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 10:47:46 (1736264866) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (4s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 10:47:50 (1736264870) Conflict PASS 42d (4s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 10:47:55 (1736264875) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (4s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 10:47:59 (1736264879) Conflict PASS 42f (5s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 10:48:04 (1736264884) Conflict PASS 42g (4s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 10:48:08 (1736264888) Conflict PASS 42h (4s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 10:48:12 (1736264892) PASS 43a (11s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 10:48:23 (1736264903) Conflict PASS 43b (4s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 10:48:27 (1736264907) Conflict PASS 43c (4s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 10:48:32 (1736264912) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (5s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 10:48:36 (1736264916) Conflict PASS 43e (4s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 10:48:40 (1736264920) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (4s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 10:48:44 (1736264924) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (4s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 10:48:48 (1736264928) Conflict PASS 43h (4s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 10:48:52 (1736264932) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 10:48:53 (1736264933) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (33s) == sanityn test 43k: unlink vs create ==================== 10:49:26 (1736264966) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (356s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 10:55:22 (1736265322) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Tue Jan 7 10:55:25 EST 2025 PASS 44a (4s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 10:55:26 (1736265326) open: File exists Conflict PASS 44b (5s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 10:55:31 (1736265331) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (4s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 10:55:35 (1736265335) Conflict PASS 44d (4s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 10:55:39 (1736265339) Conflict PASS 44e (4s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 10:55:43 (1736265343) Conflict PASS 44f (5s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 10:55:48 (1736265348) Conflict PASS 44g (4s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 10:55:52 (1736265352) Conflict PASS 44h (4s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 10:55:56 (1736265356) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 10:55:57 (1736265357) PASS 45a (19s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 10:56:16 (1736265376) Conflict PASS 45b (4s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 10:56:20 (1736265380) Conflict PASS 45c (4s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 10:56:24 (1736265384) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (4s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 10:56:28 (1736265388) Conflict PASS 45e (4s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 10:56:32 (1736265392) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (5s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 10:56:37 (1736265397) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (4s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 10:56:41 (1736265401) Conflict PASS 45h (3s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 10:56:44 (1736265404) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 10:56:45 (1736265405) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (363s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 11:02:49 (1736265769) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (4s) == sanityn test 46b: pdirops: link vs create ======================================================================== 11:02:53 (1736265773) open: File exists Conflict PASS 46b (4s) == sanityn test 46c: pdirops: link vs link ======================================================================== 11:02:57 (1736265777) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (4s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 11:03:01 (1736265781) Conflict PASS 46d (4s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 11:03:05 (1736265785) Conflict PASS 46e (5s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 11:03:10 (1736265790) Conflict PASS 46f (4s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 11:03:14 (1736265794) Conflict PASS 46g (4s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 11:03:18 (1736265798) Conflict PASS 46h (4s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 11:03:22 (1736265802) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 11:03:23 (1736265803) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 11:03:24 (1736265804) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 11:03:25 (1736265805) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (0s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 11:03:25 (1736265805) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 11:03:26 (1736265806) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 11:03:27 (1736265807) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 11:03:28 (1736265808) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (0s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 11:03:28 (1736265808) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.00408249 s, 2.5 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 11:03:35 (1736265815) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00349023 s, 45.6 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (24380) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 11:03:39 (1736265819) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00337018 s, 304 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143276 s, 73.2 MB/s 3145728 PASS 51b (13s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 11:03:52 (1736265832) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00294997 s, 347 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 11:03:58 (1736265838) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00935066 s, 112 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (3s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 11:04:01 (1736265841) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2433 0x981 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2433 0x981 0 read lease(1) has applied. PASS 51e (3s) == sanityn test 54: rename locking ======================= 11:04:04 (1736265844) llite.lustre-ffff8800b3eee800.xattr_cache=1 llite.lustre-ffff8800b652f000.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504963 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 11:04:05.000000000 -0500 Modify: 2025-01-07 11:04:05.000000000 -0500 Change: 2025-01-07 11:04:05.000000000 -0500 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504966 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 11:04:10.000000000 -0500 Modify: 2025-01-07 11:04:10.000000000 -0500 Change: 2025-01-07 11:04:10.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504969 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 11:04:15.000000000 -0500 Modify: 2025-01-07 11:04:15.000000000 -0500 Change: 2025-01-07 11:04:15.000000000 -0500 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 11:04:15.000000000 -0500 Modify: 2025-01-07 11:04:16.000000000 -0500 Change: 2025-01-07 11:04:16.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504972 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 11:04:21.000000000 -0500 Modify: 2025-01-07 11:04:21.000000000 -0500 Change: 2025-01-07 11:04:21.000000000 -0500 Birth: - File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 11:04:21.000000000 -0500 Modify: 2025-01-07 11:04:21.000000000 -0500 Change: 2025-01-07 11:04:21.000000000 -0500 Birth: - PASS 54 (23s) == sanityn test 55a: rename vs unlink target dir ========= 11:04:27 (1736265867) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 11:04:34 (1736265874) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 11:04:41 (1736265881) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 11:04:53 (1736265893) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) == sanityn test 60: Verify data_version behaviour ======== 11:05:02 (1736265902) PASS 60 (1s) == sanityn test 70a: cd directory && rm directory ======== 11:05:03 (1736265903) PASS 70a (2s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 11:05:05 (1736265905) PASS 70b (1s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 11:05:06 (1736265906) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00218821 s, 29.9 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 11:05:07 (1736265907) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00327575 s, 20.0 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 11:05:08 (1736265908) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 11:05:09 (1736265909) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00360325 s, 18.2 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (0s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 11:05:09 (1736265909) llite.lustre-ffff8800b3eee800.xattr_cache=1 llite.lustre-ffff8800b652f000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 11:05:11 (1736265911) llite.lustre-ffff8800b3eee800.xattr_cache=1 llite.lustre-ffff8800b652f000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (1s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 11:05:12 (1736265912) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00291505 s, 351 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00197684 s, 518 kB/s 3022: get lock2 3023: get lock1 3023: done 3023: get lock2 3022: done 3022: get lock1 fcntl cmd 7 failed: Resource deadlock avoided 3022: cannot get lock1: Resource deadlock avoided fcntl cmd 7 failed: Resource deadlock avoided 3023: cannot get lock2: Resource deadlock avoided 3023: exit rc=0 3022: done 3022: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 11:05:18 (1736265918) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0132834 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0066956 s, 157 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 3657 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 11:05:25 (1736265925) opening files: ........ closing files: ........ PASS 76 (13s) == sanityn test 77a: check FIFO NRS policy =============== 11:05:38 (1736265938) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0939751 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116064 s, 90.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143585 s, 73.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0344531 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0305979 s, 34.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0367274 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318732 s, 32.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295275 s, 35.5 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276749 s, 37.9 MB/s , 0.0298709 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0412383 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0362526 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.030427 s, 34.5 MB/s , 0.0365551 s, 28.7 MB/s , 0.0416374 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0431246 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398961 s, 26.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3eee800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3eee800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00863227 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00705913 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00895681 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00798271 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00794781 s, 132 MB/s , 0.00249992 s, 419 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117323 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0092174 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00152543 s, 687 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00172 s, 610 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.011913 s, 88.0 MB/s , 0.0110837 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112984 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102689 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104908 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115576 s, 90.7 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg340-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) == sanityn test 77b: check CRR-N NRS policy ============== 11:05:40 (1736265940) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0983893 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00877626 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0075854 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00804443 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164163 s, 63.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112965 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0221773 s, 47.3 MB/s , 0.0121753 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125824 s, 83.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209169 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392231 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0409234 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0471969 s, 22.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0462397 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0505326 s, 20.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0491294 s, 21.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3eee800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0498957 s, 21.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00599699 s, 175 MB/s 1048576 bytes (1.0 MB) copied, 0.00653746 s, 160 MB/s , 0.00980648 s, 107 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00921035 s, 114 MB/s , 0.00401206 s, 261 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00772987 s, 136 MB/s , 0.0100526 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108783 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118235 s, 88.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copiedldlm.namespaces.lustre-OST0001-osc-ffff8800b3eee800.lock_unused_count=1 , 0.0102432 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115097 s, 91.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00851293 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112673 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117449 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122916 s, 85.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121174 s, 86.5 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg340-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0955192 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00746239 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00943871 s, 111 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171874 s, 61.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0122846 s, 85.4 MB/s , 0.0103238 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185571 s, 56.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111994 s, 93.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112489 s, 93.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171019 s, 61.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235759 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0257263 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239962 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.038391 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395796 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318207 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0394043 s, 26.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3eee800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3eee800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0096078 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00929015 s, 113 MB/s 1048576 bytes (1.0 MB) copied, 0.00222014 s, 472 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00686964 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00795434 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00326734 s, 321 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948732 s, 111 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102721 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00978756 s, 107 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00401181 s, 261 MB/s , 0.0106883 s, 98.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00382018 s, 274 MB/s 1048576 bytes (1.0 MB) copied, 0.00401345 s, 261 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00175673 s, 597 MB/s 1048576 bytes (1.0 MB) copied, 0.0118914 s, 88.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00154758 s, 678 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg340-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) == sanityn test 77c: check ORR NRS policy ================ 11:05:43 (1736265943) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.10888 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132463 s, 79.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150994 s, 69.4 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011843 s, 88.5 MB/s , 0.0196378 s, 53.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180799 s, 58.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294997 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333408 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295966 s, 35.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0383704 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315509 s, 33.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.024354 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317423 s, 33.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0323821 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026146 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0347793 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392552 s, 26.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3eee800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3eee800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0077131 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0084461 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00984859 s, 106 MB/s 1048576 bytes (1.0 MB) copied, 0.00916924 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00242728 s, 432 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00890215 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00917285 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00119048 s, 881 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00946564 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00952439 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00951955 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00856681 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00917152 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00885243 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104823 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00910304 s, 115 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg340-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0911818 s, 184 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00814205 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117303 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117801 s, 89.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170699 s, 61.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130589 s, 80.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114479 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229671 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263462 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341699 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0257621 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.032678 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181788 s, 57.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238068 s, 44.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3eee800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025317 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321968 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026582 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008387 s, 125 MB/s , 0.0105598 s, 99.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00874661 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152168 s, 68.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179402 s, 58.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153054 s, 68.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163878 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147542 s, 71.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166895 s, 62.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176859 s, 59.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186472 s, 56.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01785 s, 58.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158642 s, 66.1 MB/s 1048576 bytes (1.0 MB) copied, 0.01915 s, 54.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0202233 s, 51.8 MB/s , 0.0205225 s, 51.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3eee800.lock_unused_count=1 removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg340-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.08839 s, 190 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00680972 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00953216 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018361 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265928 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266013 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0221171 s, 47.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238776 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278162 s, 37.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3eee800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3eee800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341952 s, 30.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304669 s, 34.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0309732 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354348 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423491 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0368196 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.036949 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447587 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0075161 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00851289 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00869863 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107166 s, 97.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0087321 s, 120 MB/s 1+0 records in 1+0 records out , 0.0066792 s, 157 MB/s 1048576 bytes (1.0 MB) copied, 0.00157588 s, 665 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00543218 s, 193 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103816 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102932 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105088 s, 99.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00855852 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00982713 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00931496 s, 113 MB/s , 0.00922949 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01178 s, 89.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3eee800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3eee800.lock_unused_count=3 removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg340-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (4s) == sanityn test 77d: check TRR nrs policy ================ 11:05:47 (1736265947) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0919455 s, 182 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00817684 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118979 s, 88.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.016264 s, 64.5 MB/s , 0.0177381 s, 59.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162361 s, 64.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307333 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198019 s, 53.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253314 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026223 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247343 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194976 s, 53.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423086 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0311747 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319144 s, 32.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3eee800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.028928 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388652 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00908023 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00208713 s, 502 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00579901 s, 181 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985847 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00402439 s, 261 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00661284 s, 159 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0076581 s, 137 MB/s , 0.00586543 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0064643 s, 162 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0078206 s, 134 MB/s , 0.00788849 s, 133 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0100632 s, 104 MB/s 1048576 bytes (1.0 MB) copied, 0.00132073 s, 794 MB/s , 0.00924456 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00135646 s, 773 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00142153 s, 738 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3eee800.lock_unused_count=2 removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg340-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.120582 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0066343 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120138 s, 87.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151529 s, 69.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202732 s, 51.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190346 s, 55.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0330786 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341331 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172519 s, 60.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304091 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.028414 s, 36.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0324377 s, 32.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0307534 s, 34.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0313065 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256836 s, 40.8 MB/s , 0.0264402 s, 39.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3eee800.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3eee800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283139 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00543179 s, 193 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00874039 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114884 s, 91.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112361 s, 93.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0075482 s, 139 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00766613 s, 137 MB/s , 0.00796525 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00843533 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00853106 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00947112 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985922 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101385 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010186 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00994377 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115953 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101432 s, 103 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg340-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.131614 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00979757 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207278 s, 50.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195562 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019951 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145655 s, 72.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0319876 s, 32.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0314111 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332324 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0427953 s, 24.5 MB/s , 0.0334341 s, 31.4 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3eee800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0362551 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0407502 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.033254 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354314 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318563 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.029976 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00932089 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00822257 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00690181 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00707477 s, 148 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00861466 s, 122 MB/s , 0.00944136 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00843506 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00139124 s, 754 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00742298 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0045261 s, 232 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00255656 s, 410 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0098698 s, 106 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116382 s, 90.1 MB/s 1048576 bytes (1.0 MB) copied, 0.01298 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00165945 s, 632 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00138674 s, 756 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg340-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (4s) == sanityn test 77e: check TBF NID nrs policy ============ 11:05:51 (1736265951) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.203.40@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0834904 s, 201 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00812847 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152607 s, 68.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017813 s, 58.9 MB/s 1048576 bytes (1.0 MB) copied, 0.00868387 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137458 s, 76.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238599 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0287895 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0205217 s, 51.1 MB/s , 0.019271 s, 54.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184172 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318132 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118931 s, 88.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294466 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283309 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00599718 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242373 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100237 s, 105 MB/s , 0.00728051 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00901163 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00661485 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00813046 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00842877 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105211 s, 99.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00188568 s, 556 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107401 s, 97.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108813 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102563 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108379 s, 96.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117275 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00951329 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121062 s, 86.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129889 s, 80.7 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg340-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.109419 s, 153 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00819486 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00960931 s, 109 MB/s 1048576 bytes (1.0 MB) copied, 0.0109843 s, 95.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130324 s, 80.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010046 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021766 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265335 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178652 s, 58.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272648 s, 38.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00946512 s, 111 MB/s , 0.0167423 s, 62.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228685 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121383 s, 86.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0309149 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022588 s, 46.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00681518 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00814265 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00725629 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00999789 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00829662 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00868267 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0095812 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00880103 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985854 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0097981 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918908 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010204 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102805 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100123 s, 105 MB/s , 0.0101664 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115644 s, 90.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122826 s, 85.4 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg340-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.101015 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00991361 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158921 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210646 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211996 s, 49.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205388 s, 51.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306299 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150746 s, 69.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343203 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328769 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100132 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295391 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.031081 s, 33.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0306946 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332131 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285027 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189659 s, 55.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3eee800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00643854 s, 163 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00644703 s, 163 MB/s , 0.00673383 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00832407 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00765859 s, 137 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00990521 s, 106 MB/s , 0.00762765 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00796775 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00886319 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00904661 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00879243 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00893187 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00879704 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948839 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00832427 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101878 s, 103 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg340-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0856994 s, 196 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00687598 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144213 s, 72.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107454 s, 97.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023208 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189662 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.029516 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246019 s, 42.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0238573 s, 44.0 MB/s , 0.0248908 s, 42.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246924 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277168 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019029 s, 55.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241369 s, 43.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0179624 s, 58.4 MB/s , 0.00916791 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171623 s, 61.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00827445 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00661942 s, 158 MB/s 1048576 bytes (1.0 MB) copied, 0.0013083 s, 801 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00841313 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0076569 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00773878 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00815597 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00667041 s, 157 MB/s , 0.00402471 s, 261 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101674 s, 103 MB/s , 0.00244624 s, 429 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00223121 s, 470 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0099811 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124343 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00120681 s, 869 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00129516 s, 810 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3eee800.lock_unused_count=1 removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg340-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (6s) == sanityn test 77f: check TBF JobID nrs policy ========== 11:05:57 (1736265957) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg340-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.130508 s, 129 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00838974 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111685 s, 93.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0350078 s, 30.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0377005 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0337938 s, 31.0 MB/s , 0.0438415 s, 23.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411861 s, 25.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0400793 s, 26.2 MB/s 1048576 bytes (1.0 MB) copied, 0.047022 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0454301 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395265 s, 26.5 MB/s , 0.0448735 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495257 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0430726 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0428257 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0453192 s, 23.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00670906 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00738614 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00887871 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00909515 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00414744 s, 253 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00973888 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126065 s, 83.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100754 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00932699 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01249 s, 84.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107595 s, 97.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104837 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108946 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119127 s, 88.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118938 s, 88.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136074 s, 77.1 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg340-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg340-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.148335 s, 113 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0079842 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111555 s, 94.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154794 s, 67.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016676 s, 62.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189616 s, 55.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0200772 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188099 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175028 s, 59.9 MB/s , 0.0245567 s, 42.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120441 s, 87.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180605 s, 58.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398735 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207256 s, 50.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197633 s, 53.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343051 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237965 s, 44.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00659903 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00551935 s, 190 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00706731 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00722773 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00795036 s, 132 MB/s 1048576 bytes (1.0 MB) copied, 0.00915315 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118968 s, 88.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00696419 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108605 s, 96.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00784362 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109416 s, 95.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00900545 s, 116 MB/s 1048576 bytes (1.0 MB) copied, 0.0122442 s, 85.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00945679 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104956 s, 99.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114544 s, 91.5 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg340-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg340-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.119427 s, 140 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128341 s, 81.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169116 s, 62.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163848 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271485 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0384876 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0379882 s, 27.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0369381 s, 28.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0457524 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.04158 s, 25.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0467336 s, 22.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0488983 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0325863 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0406342 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0377024 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0445787 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0425017 s, 24.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3eee800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3eee800.lock_unused_count=4 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0063211 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00681961 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107806 s, 97.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100348 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00903455 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984446 s, 107 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0109344 s, 95.9 MB/s , 0.00988272 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105994 s, 98.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103273 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106153 s, 98.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0103384 s, 101 MB/s , 0.011286 s, 92.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118321 s, 88.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102756 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012933 s, 81.1 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg340-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg340-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.106392 s, 158 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130017 s, 80.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115813 s, 90.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0334586 s, 31.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0408809 s, 25.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0403263 s, 26.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0391069 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389807 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0374366 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0393344 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0383396 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0432826 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378547 s, 27.7 MB/s , 0.0443957 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0463892 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0399269 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0529954 s, 19.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3eee800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3eee800.lock_unused_count=7 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00924319 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00200758 s, 522 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00186818 s, 561 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00152657 s, 687 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00937261 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0091764 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100968 s, 104 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3eee800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117464 s, 89.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0116354 s, 90.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114189 s, 91.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162613 s, 64.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00916245 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00345716 s, 303 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00141744 s, 740 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110794 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0017219 s, 609 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg340-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (6s) == sanityn test 77g: Change TBF type directly ============ 11:06:03 (1736265963) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (3s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 11:06:06 (1736265966) oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg340-client: oleg340-server: ssh exited with exit code 2 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg340-client: oleg340-server: ssh exited with exit code 12 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg340-client: oleg340-server: ssh exited with exit code 12 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg340-client: oleg340-server: ssh exited with exit code 12 PASS 77h (3s) == sanityn test 77i: Change rank of TBF rule ============= 11:06:09 (1736265969) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (6s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 11:06:15 (1736265975) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0908024 s, 185 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00693902 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114064 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110817 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151072 s, 69.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159144 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124811 s, 84.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251923 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214172 s, 49.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253486 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133288 s, 78.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016569 s, 63.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017768 s, 59.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151482 s, 69.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246772 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173704 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251207 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00708672 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00781208 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00809377 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100376 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0097146 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00813968 s, 129 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00946983 s, 111 MB/s , 0.00851959 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.009324 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108714 s, 96.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104261 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101888 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104515 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104368 s, 100 MB/s , 0.0117432 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118376 s, 88.6 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg340-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92643 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4342 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (49s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 11:07:04 (1736266024) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg340-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.645509 s, 26.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158117 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156997 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.173139 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.170592 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.181864 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.17521 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178115 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.17538 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.356118 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.365819 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.359679 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.361416 s, 2.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.37883 s, 2.8 MB/s , 0.380681 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.380736 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.38409 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14298 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142837 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143319 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143571 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143829 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14703 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145261 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.145151 s, 7.2 MB/s 1048576 bytes (1.0 MB) copied, 0.14707 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145792 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146651 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145654 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145569 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146038 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14756 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148387 s, 7.1 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg340-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8317 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9671 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg340-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.674947 s, 24.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.133523 s, 7.9 MB/s , 0.129472 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13125 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13336 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153506 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156417 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142315 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148531 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.33861 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.336077 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.355864 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.34202 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.364726 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.366473 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.374459 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.392815 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.134378 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.138438 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.137421 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.135285 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.137457 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.138223 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13746 s, 7.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.138476 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.139204 s, 7.5 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.141356 s, 7.4 MB/s , 0.140951 s, 7.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.139938 s, 7.5 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.140358 s, 7.5 MB/s , 0.140535 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.141404 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143128 s, 7.3 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg340-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8472 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9502 s, 5.3 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (132s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 11:09:16 (1736266156) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg340-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.218317 s, 76.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172793 s, 60.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134344 s, 78.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235083 s, 44.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0221392 s, 47.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253955 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163768 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115155 s, 91.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123948 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0519118 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0743452 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0772101 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0753545 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0685472 s, 15.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0674601 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0877933 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0758786 s, 13.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010491 s, 99.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00963129 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128467 s, 81.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117313 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110973 s, 94.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115043 s, 91.1 MB/s , 0.0112291 s, 93.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115196 s, 91.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118864 s, 88.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129702 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131252 s, 79.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139385 s, 75.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135358 s, 77.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135372 s, 77.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143399 s, 73.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172008 s, 61.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg340-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95355 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74893 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.40@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.40@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.162178 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132555 s, 79.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210531 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100093 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100645 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176041 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148965 s, 70.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178483 s, 58.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00673216 s, 156 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0519611 s, 20.2 MB/s , 0.0535861 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0535329 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0616089 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0632376 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.078897 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1431 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151406 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106321 s, 98.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.009203 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124458 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115667 s, 90.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0121224 s, 86.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0112202 s, 93.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151924 s, 69.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122624 s, 85.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0129413 s, 81.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0125989 s, 83.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013054 s, 80.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129929 s, 80.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142268 s, 73.7 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0151664 s, 69.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0133981 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014055 s, 74.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg340-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93704 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74833 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.40@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg340-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.176334 s, 95.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141934 s, 73.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0329587 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0305672 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343115 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0424207 s, 24.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398686 s, 26.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0260739 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0458178 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0429014 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342844 s, 30.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0468483 s, 22.4 MB/s , 0.0484695 s, 21.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.065398 s, 16.0 MB/s 1048576 bytes (1.0 MB) copied, 0.068627 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0506453 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0633791 s, 16.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106072 s, 98.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127134 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143773 s, 72.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130969 s, 80.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124334 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124275 s, 84.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143348 s, 73.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177145 s, 59.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0145642 s, 72.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144116 s, 72.8 MB/s , 0.0133842 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153766 s, 68.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0153025 s, 68.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153893 s, 68.1 MB/s , 0.0151984 s, 69.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149161 s, 70.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg340-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95685 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.89156 s, 21.4 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg340-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.249848 s, 67.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161795 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0302396 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189725 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356817 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259288 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285051 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0359334 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270702 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157665 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155346 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.30019 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.311693 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.353675 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.348088 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.339308 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.352902 s, 3.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0111681 s, 93.9 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0148451 s, 70.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0142843 s, 73.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016635 s, 63.0 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0192573 s, 54.5 MB/s , 0.0150894 s, 69.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157079 s, 66.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143877 s, 72.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148454 s, 70.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165958 s, 63.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161915 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01488 s, 70.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184552 s, 56.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0133168 s, 78.7 MB/s , 0.0153899 s, 68.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179519 s, 58.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg340-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.83112 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7412 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95051 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74685 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg340-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.482283 s, 34.8 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137057 s, 76.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211975 s, 49.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.043442 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0409085 s, 25.6 MB/s , 0.041794 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294638 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343859 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0424001 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.335725 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.329007 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.336589 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.340819 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.342425 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.346066 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.361574 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.365601 s, 2.9 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00966575 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00943538 s, 111 MB/s , 0.0101824 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116982 s, 89.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015457 s, 67.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113375 s, 92.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120758 s, 86.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119857 s, 87.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124355 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131294 s, 79.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120578 s, 87.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120282 s, 87.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012408 s, 84.5 MB/s , 0.0111309 s, 94.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143292 s, 73.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014861 s, 70.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg340-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.6706 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4384 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg340-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.164475 s, 102 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132942 s, 78.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227961 s, 46.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161994 s, 64.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0312987 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259842 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0301711 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206305 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241212 s, 43.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0516053 s, 20.3 MB/s , 0.0425658 s, 24.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0599413 s, 17.5 MB/s , 0.0475983 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0525591 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0541935 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0565238 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0610633 s, 17.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00735852 s, 142 MB/s 1+0 records in 1+0 records out , 0.00207366 s, 506 MB/s 1048576 bytes (1.0 MB) copied, 0.00749835 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972841 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123084 s, 85.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0096803 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0098162 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102915 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110958 s, 94.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108394 s, 96.7 MB/s , 0.00982935 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109773 s, 95.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104534 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00934882 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00980362 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152399 s, 68.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg340-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.965 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74471 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg340-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.245421 s, 68.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101441 s, 103 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226831 s, 46.2 MB/s 1048576 bytes (1.0 MB) copied, 0.019272 s, 54.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0221234 s, 47.4 MB/s , 0.0160881 s, 65.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0264323 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0309681 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342846 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158966 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.165462 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.181359 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.191357 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.195072 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.197287 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.317971 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.316983 s, 3.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104824 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00743431 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00927407 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00871542 s, 120 MB/s 1048576 bytes (1.0 MB) copied, 0.00983054 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100335 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948691 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129503 s, 81.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0125034 s, 83.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116579 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125573 s, 83.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123266 s, 85.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127362 s, 82.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133632 s, 78.5 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142233 s, 73.7 MB/s , 0.0134139 s, 78.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg340-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77413 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73218 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77959 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7415 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (356s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 11:15:12 (1736266512) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (3s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 11:15:15 (1736266515) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg340-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.97927 s, 264 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg340-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0415 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (51s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 11:16:06 (1736266566) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0991772 s, 169 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0077626 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0080728 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103822 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207082 s, 50.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307784 s, 34.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0339866 s, 30.9 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0220716 s, 47.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0251641 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371139 s, 28.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3eee800.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3eee800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308717 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267944 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0370686 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0299374 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297154 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332485 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0376256 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00928885 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00185888 s, 564 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00321316 s, 326 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00887223 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111567 s, 94.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00564721 s, 186 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00222813 s, 471 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00946073 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102656 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106921 s, 98.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108201 s, 96.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111876 s, 93.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118192 s, 88.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111934 s, 93.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00180097 s, 582 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133802 s, 78.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3eee800.lock_unused_count=2 removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg340-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94008 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.974 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.18785 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00633573 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117239 s, 89.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155491 s, 67.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00973033 s, 108 MB/s 1048576 bytes (1.0 MB) copied, 0.0163239 s, 64.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00657001 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01266 s, 82.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00709683 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0714356 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0758142 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0857523 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0805996 s, 13.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.171775 s, 6.1 MB/s , 0.158927 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.271569 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.266332 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152041 s, 69.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171081 s, 61.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139169 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129227 s, 81.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0154538 s, 67.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0143213 s, 73.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145348 s, 72.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151028 s, 69.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165172 s, 63.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0178673 s, 58.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161743 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158664 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184343 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191057 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177594 s, 59.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201272 s, 52.1 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg340-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg340-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95909 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97439 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (64s) == sanityn test 77o: Changing rank should not panic ====== 11:17:10 (1736266630) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (3s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 11:17:13 (1736266633) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (26s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 11:17:39 (1736266659) mds.MDS.mdt.nrs_policies=tbf oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg340-client: oleg340-server: ssh exited with exit code 36 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (11s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 11:17:50 (1736266670) oleg340-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: Resource temporarily unavailable pdsh@oleg340-client: oleg340-server: ssh exited with exit code 11 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (3s) == sanityn test 79: xattr: intent error ================== 11:17:53 (1736266673) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505431 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 11:17:53.000000000 -0500 Modify: 2025-01-07 11:17:53.000000000 -0500 Change: 2025-01-07 11:17:53.000000000 -0500 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 11:18:05 (1736266685) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (0s) == sanityn test 80b: Accessing directory during migration ========================================================== 11:18:06 (1736266686) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (0s) == sanityn test 81a: rename and stat under striped directory ========================================================== 11:18:06 (1736266686) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 11:18:07 (1736266687) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 11:18:08 (1736266688) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (0s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 11:18:08 (1736266688) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6871 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6871 PASS 82 (2s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 11:18:10 (1736266690) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 11:18:11 (1736266691) fail_loc=0x8000060b PASS 84 (7s) == sanityn test 90: open/create and unlink striped directory ========================================================== 11:18:18 (1736266698) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 11:18:19 (1736266699) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 11:18:20 (1736266700) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 11:18:21 (1736266701) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2628 0xa44 0 0 2646 0xa56 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2647 0xa57 0 1 2629 0xa45 0 rc1=0 and rc2=0 PASS 93 (8s) == sanityn test 94: signal vs CP callback race =========== 11:18:29 (1736266709) osc.lustre-OST0000-osc-ffff8800b3eee800.idle_timeout=0 osc.lustre-OST0000-osc-ffff8800b652f000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b3eee800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b652f000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0314243 s, 261 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 3.00419 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 4172 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0161207 s, 254 kB/s osc.lustre-OST0000-osc-ffff8800b3eee800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b652f000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b3eee800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b652f000.idle_timeout=debug PASS 94 (10s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 11:18:39 (1736266719) llite.lustre-ffff8800b3eee800.fast_read=0 llite.lustre-ffff8800b652f000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00206591 s, 5.9 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800b3eee800.fast_read=1 llite.lustre-ffff8800b652f000.fast_read=1 PASS 95b (7s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 11:18:46 (1736266726) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 11:18:47 (1736266727) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00136605 s, 3.0 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (1s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 11:18:48 (1736266728) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0214434 s, 97.8 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 11:18:50 (1736266730) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0212251 s, 98.8 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 11:18:52 (1736266732) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.0016046 s, 7.5 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 11:18:54 (1736266734) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000610356 s, 6.7 MB/s PASS 101a (1s) == sanityn test 101b: Discard DoM data on rename ========= 11:18:55 (1736266735) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000586689 s, 7.0 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 11:18:57 (1736266737) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000443379 s, 9.2 MB/s PASS 101c (2s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 11:19:00 (1736266740) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1736266740 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1736266743 Finishing test test23 at 1736266743 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 11:19:07 (1736266747) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 11:19:08 (1736266748) fail_loc=0x416 PASS 105 (47s) == sanityn test 106a: Verify the btime via statx() ======= 11:19:55 (1736266795) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (2s) == sanityn test 106b: Glimpse RPCs test for statx ======== 11:19:57 (1736266797) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) == sanityn test 106c: Verify statx attributes mask ======= 11:19:58 (1736266798) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) == sanityn test 107a: Basic grouplock conflict =========== 11:19:59 (1736266799) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0549124 s, 191 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2654 0xa5e 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6871 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6871 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 11:20:03 (1736266803) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0811908 s, 129 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2633 0xa49 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6871 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (9s) == sanityn test 108a: lseek: parallel updates ============ 11:20:12 (1736266812) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.0028541 s, 3.5 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.19882 s, 2.0 MB/s PASS 108a (6s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 11:20:18 (1736266818) 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 5 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 6 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 8 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 10 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 12 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 15 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' oleg340-server: insmod: ERROR: could not insert module /home/green/git/lustre-release/lustre/../libcfs/libcfs/libcfs.ko: File exists pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Iteration 22 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 24 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 28 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 29 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 36 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 47 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 49 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' oleg340-server: insmod: ERROR: could not insert module /home/green/git/lustre-release/lustre/../libcfs/libcfs/libcfs.ko: File exists pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Iteration 50 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre 192.168.203.140@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.140@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: Force libcfs to create 2 CPU partitions Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre Starting client: oleg340-client.virtnet: -o user_xattr,flock oleg340-server@tcp:/lustre /mnt/lustre2 PASS 109 (433s) == sanityn test 110: do not grant another lock on resend ========================================================== 11:27:31 (1736267251) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 11:27:32.000000000 -0500 Modify: 2025-01-07 11:27:32.000000000 -0500 Change: 2025-01-07 11:27:32.000000000 -0500 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg340-client.virtnet oleg340-client.virtnet PASS 110 (37s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 11:28:08 (1736267288) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 11:28:09 (1736267289) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 11:28:10 (1736267290) lustre-OST0000-osc-ffff88009d9d0800 active. lustre-OST0001-osc-ffff88009d9d0800 active. lustre-OST0000-osc-ffff88009d9d0800 active. lustre-OST0001-osc-ffff88009d9d0800 active. MGC192.168.203.140@tcp active. PASS 113 (2s) cleanup: ====================================================== == sanityn test complete, duration 3970 sec ============== 11:28:13 (1736267293) Stopping clients: oleg340-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg340-client.virtnet /mnt/lustre2 opts: