-----============= acceptance-small: sanityn ============----- Fri Mar 22 03:09:34 EDT 2024 excepting tests: 27 28 102 skipping tests SLOW=no: 33a === sanityn: start setup 03:09:36 (1711091376) === Starting client oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 Started clients oleg258-client.virtnet: 192.168.202.158@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project) oleg258-client.virtnet: executing check_config_client /mnt/lustre oleg258-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg258-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800b58aa800.idle_timeout=debug osc.lustre-OST0000-osc-ffff88012a56e000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b58aa800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012a56e000.idle_timeout=debug disable quota as required oleg258-server: oleg258-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 03:09:43 (1711091383) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119223 s, 88.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6749] debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 03:09:45 (1711091385) PASS 0 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 03:09:48 (1711091388) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 03:09:51 (1711091391) -rw-r--r-- 1 root root 0 Mar 22 03:09 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 03:09:54 (1711091394) -rw-r--r-- 1 root root 0 Mar 22 03:09 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 03:09:57 (1711091397) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 03:10:00 (1711091400) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2e: check chmod on root is propagated to others ========================================================== 03:10:03 (1711091403) total 1 -rwxrwxrwx 1 root root 0 Mar 22 03:09 f2a -rwxrwxrwx 1 root root 0 Mar 22 03:09 f2b total 1 -rwxrwxrwx 1 root root 0 Mar 22 03:09 f2a -rwxrwxrwx 1 root root 0 Mar 22 03:09 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00258899 s, 198 kB/s PASS 2e (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 03:10:06 (1711091406) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 2g: check blocks update on sync write ==== 03:10:09 (1711091409) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0910012 s, 23.0 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 03:10:12 (1711091412) PASS 3 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 4: fstat validation on multiple mount points ==================================================================== 03:10:15 (1711091415) PASS 4 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 03:10:19 (1711091419) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 6: remove of open file on other node ============================================================================ 03:10:22 (1711091422) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 7: remove of open directory on other node ======================================================================= 03:10:25 (1711091425) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 8: remove of open special file on other node ==================================================================== 03:10:28 (1711091428) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 03:10:31 (1711091431) PASS 9a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 9b: append to striped sparse file ======== 03:10:34 (1711091434) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112587 s, 93.1 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0110359 s, 0.3 kB/s Data read (expecting 'foo'): foo PASS 9b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 03:10:38 (1711091438) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00161692 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00231614 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00328097 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00242243 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00301289 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00427726 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00441728 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00363416 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00250135 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00288298 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00234206 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00245553 s, 0.4 kB/s PASS 10a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 03:10:41 (1711091441) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00267346 s, 1.1 MB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00731579 s, 560 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000286749 s, 10.7 MB/s PASS 10b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 11: execution of file opened for write should return error ============================================================== 03:10:44 (1711091444) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6749 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 301: /mnt/lustre2/d11/f: Text file busy PASS 11 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 03:10:48 (1711091448) warning: '-runtime' deprecated, use '-t runtime' instead - link 3086 (time 1711091458.52 total 10.00 last 308.52) - link 6216 (time 1711091468.52 total 20.00 last 312.93) - link 9539 (time 1711091478.52 total 30.01 last 332.22) - link 12732 (time 1711091488.52 total 40.01 last 319.27) - link 15602 (time 1711091498.52 total 50.01 last 286.93) - link 19026 (time 1711091508.52 total 60.01 last 342.39) - link 20000 (time 1711091511.40 total 62.89 last 338.34) - link 23395 (time 1711091521.41 total 72.89 last 339.44) - link 26847 (time 1711091531.41 total 82.89 last 345.14) - link 30000 (time 1711091540.51 total 92.00 last 346.38) total: 32760 link in 100.00 seconds: 327.60 ops/second using seed 2064900893 running for 100 seconds - stat 10000 (time 1711091452 ; total 3 ; last 3) - stat 20000 (time 1711091455 ; total 6 ; last 3) - stat 30000 (time 1711091458 ; total 9 ; last 3) - stat 40000 (time 1711091461 ; total 12 ; last 3) - stat 50000 (time 1711091464 ; total 15 ; last 3) - stat 60000 (time 1711091467 ; total 18 ; last 3) - stat 70000 (time 1711091470 ; total 21 ; last 3) - stat 80000 (time 1711091473 ; total 24 ; last 3) - stat 90000 (time 1711091476 ; total 27 ; last 3) - stat 100000 (time 1711091478 ; total 29 ; last 2) - stat 110000 (time 1711091481 ; total 32 ; last 3) - stat 120000 (time 1711091484 ; total 35 ; last 3) - stat 130000 (time 1711091487 ; total 38 ; last 3) - stat 140000 (time 1711091490 ; total 41 ; last 3) - stat 150000 (time 1711091494 ; total 45 ; last 4) - stat 160000 (time 1711091497 ; total 48 ; last 3) - stat 170000 (time 1711091501 ; total 52 ; last 4) - stat 180000 (time 1711091509 ; total 60 ; last 8) - stat 190000 (time 1711091518 ; total 69 ; last 9) - stat 200000 (time 1711091527 ; total 78 ; last 9) - stat 210000 (time 1711091535 ; total 86 ; last 8) - stat 220000 (time 1711091544 ; total 95 ; last 9) total: 226180 stats in 100 seconds: 2261.800049 stats/second - unlinked 0 (time 1711091499 ; total 0 ; last 0) - unlinked 10000 (time 1711091545 ; total 46 ; last 46) - unlinked 20000 (time 1711091570 ; total 71 ; last 25) - unlinked 30000 (time 1711091594 ; total 95 ; last 24) unlink(/mnt/lustre2/lockdir/lockfile32760) error: No such file or directory total: 32759 unlinks in 103 seconds: 318.048553 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (23151) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (23153) - No such process PASS 12 (155s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 13: test directory page revocation ======= 03:13:24 (1711091604) f13 f13 PASS 13 (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 03:13:29 (1711091609) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6749 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 336: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4661: 24761 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 03:13:32 (1711091612) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4661: 25499 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 03:13:35 (1711091615) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 351: 26236 Terminated $DIR1/$tdir/sleep 60 PASS 14b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 03:13:39 (1711091619) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 363: 26977 Terminated $DIR1/$tdir/sleep 60 PASS 14c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 14d: chmod of executing file is still possible ================================================================== 03:13:42 (1711091622) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 374: 27716 Terminated $DIR1/$tdir/sleep 60 PASS 14d (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 15: test out-of-space with multiple writers ===================================================================== 03:13:46 (1711091626) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7521280 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7521280kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (10s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16a: 500 iterations of dual-mount fsx ==== 03:13:58 (1711091638) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.175384 s, 47.8 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.168234 s, 49.9 MB/s Chance of close/open is 1 in 50 Seed set to 1639 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[1] 1711091639.807400 write 0x08847c thru 0x08a76d (0x22f2 bytes) 000004[1] 1711091639.814874 write 0x1afc24 thru 0x1b9639 (0x9a16 bytes) 000006[0] 1711091639.815942 mapwrite 0x68237f thru 0x68d697 (0xb319 bytes) 000008[0] 1711091639.876032 read 0x463a0d thru 0x46e49b (0xaa8f bytes) 000011[0] 1711091639.894064 mapread 0x65fae7 thru 0x6622ef (0x2809 bytes) 000012[1] 1711091639.899501 mapwrite 0x68c181 thru 0x698a62 (0xc8e2 bytes) 000013[1] 1711091639.934921 mapwrite 0x90d398 thru 0x90e780 (0x13e9 bytes) 000014[1] 1711091639.967939 mapread 0x08d0c7 thru 0x09baa7 (0xe9e1 bytes) 000016[0] 1711091639.975936 write 0x59ecbf thru 0x5a3167 (0x44a9 bytes) 000018[0] 1711091639.993436 read 0x1226e4 thru 0x123336 (0xc53 bytes) truncating to largest ever: 0x39cb51 000019[1] 1711091640.020944 trunc from 0x90e781 to 0x39cb50 (0x571c30 bytes) truncating to largest ever: 0x4c55b8 000020[0] 1711091640.048967 trunc from 0x39cb51 to 0x4c55b7 (0x128a67 bytes) 000023[1] 1711091640.057905 mapwrite 0x18332b thru 0x18fd1b (0xc9f1 bytes) truncating to largest ever: 0x97d680 000024[0] 1711091640.087741 trunc from 0x4c55b8 to 0x97d67f (0x4b80c8 bytes) 000025[1] 1711091640.095217 mapwrite 0x2a7f04 thru 0x2a8779 (0x876 bytes) 000026[0] 1711091640.122312 trunc from 0x97d680 to 0x6f4c61 (0x288a1e bytes) 000027[0] 1711091640.128760 mapread 0x3328b9 thru 0x3401fd (0xd945 bytes) 000028[0] 1711091640.156507 mapread 0x4ffb77 thru 0x50a360 (0xa7ea bytes) 000029[1] 1711091640.175282 write 0x6ba726 thru 0x6bf40c (0x4ce7 bytes) 000031[0] 1711091640.187028 write 0x82488f thru 0x828008 (0x377a bytes) 000033[0] 1711091640.195649 read 0x2b8d49 thru 0x2bfc3a (0x6ef2 bytes) 000036[1] 1711091640.223604 trunc from 0x828009 to 0x4c8547 (0x35fac1 bytes) 000037[0] 1711091640.236385 write 0x4634cb thru 0x46de58 (0xa98e bytes) 000042[0] 1711091640.265942 mapread 0x412817 thru 0x4189de (0x61c8 bytes) 000045[0] 1711091640.281610 read 0x14cc77 thru 0x156df0 (0xa17a bytes) 000046[1] 1711091640.302665 write 0x292bf5 thru 0x2a2ad3 (0xfedf bytes) 000047[1] 1711091640.317299 trunc from 0x4c8548 to 0x95686e (0x48e327 bytes) 000052[1] 1711091640.346797 mapread 0x64563b thru 0x649970 (0x4336 bytes) 000053[0] 1711091640.360650 trunc from 0x95686f to 0x3c0f9d (0x5958d1 bytes) 000056[1] 1711091640.390916 trunc from 0x3c0f9e to 0x4f4354 (0x1333b7 bytes) 000058[1] 1711091640.414573 read 0x3c9b66 thru 0x3d4890 (0xad2b bytes) 000059[0] 1711091640.422917 trunc from 0x4f4355 to 0x4a4341 (0x50013 bytes) 000061[1] 1711091640.440705 trunc from 0x4a4342 to 0x3f7025 (0xad31c bytes) 000069[1] 1711091640.467884 mapwrite 0x022653 thru 0x030739 (0xe0e7 bytes) 000070[1] 1711091640.490162 read 0x1df8e4 thru 0x1e3960 (0x407d bytes) 000071[1] 1711091640.502826 write 0x3bf417 thru 0x3c2d48 (0x3932 bytes) 000072[0] 1711091640.505155 write 0x123431 thru 0x124146 (0xd16 bytes) 000075[1] 1711091640.525144 read 0x3838fc thru 0x38e3da (0xaadf bytes) 000076[1] 1711091640.545388 read 0x2f3fc9 thru 0x302fe1 (0xf019 bytes) 000079[0] 1711091640.552969 read 0x3cf931 thru 0x3d4e2a (0x54fa bytes) 000080[1] 1711091640.575791 trunc from 0x3f7026 to 0x8d358c (0x4dc567 bytes) 000082[0] 1711091640.593481 write 0x190f06 thru 0x1919ff (0xafa bytes) 000083[1] 1711091640.602325 mapwrite 0x795629 thru 0x795c96 (0x66e bytes) 000086[0] 1711091640.629815 write 0x24bb91 thru 0x257fc5 (0xc435 bytes) 000087[1] 1711091640.636002 mapread 0x600ce3 thru 0x60f18c (0xe4aa bytes) 000088[0] 1711091640.664915 mapread 0x79f466 thru 0x7ae21c (0xedb7 bytes) 000089[1] 1711091640.689116 mapwrite 0x447c0f thru 0x44adea (0x31dc bytes) 000090[1] 1711091640.719015 mapwrite 0x2d14b5 thru 0x2d4940 (0x348c bytes) 000091[0] 1711091640.758511 mapwrite 0x0744a3 thru 0x07e03e (0x9b9c bytes) 000092[0] 1711091640.802403 write 0x971d3d thru 0x97b25c (0x9520 bytes) 000093[0] 1711091640.806733 write 0x6e0892 thru 0x6e7b04 (0x7273 bytes) 000097[0] 1711091640.824160 mapread 0x911e59 thru 0x91f10f (0xd2b7 bytes) 000099[0] 1711091640.831704 read 0x1a48c7 thru 0x1b130a (0xca44 bytes) 000105[1] 1711091640.868645 mapread 0x2358c2 thru 0x24103b (0xb77a bytes) 000107[0] 1711091640.902995 write 0x06f338 thru 0x0757b5 (0x647e bytes) 000109[1] 1711091640.911339 mapread 0x837498 thru 0x83c15d (0x4cc6 bytes) 000110[0] 1711091640.926815 read 0x87e698 thru 0x88a01f (0xb988 bytes) 000111[0] 1711091640.949076 trunc from 0x97b25d to 0x255467 (0x725df5 bytes) 000113[1] 1711091640.971419 mapread 0x122af4 thru 0x130666 (0xdb73 bytes) 000114[1] 1711091640.991786 mapwrite 0x6dbde9 thru 0x6e0923 (0x4b3b bytes) 000116[1] 1711091641.030176 read 0x46c8fe thru 0x46cc3d (0x340 bytes) 000117[1] 1711091641.034088 read 0x1661f1 thru 0x16e299 (0x80a9 bytes) 000118[1] 1711091641.055137 mapread 0x011e6c thru 0x017783 (0x5918 bytes) 000123[0] 1711091641.074641 mapread 0x2abf5e thru 0x2bb1f1 (0xf294 bytes) 000124[1] 1711091641.105588 trunc from 0x6e0924 to 0x7d40e7 (0xf37c4 bytes) 000125[1] 1711091641.113054 write 0x1a88d8 thru 0x1ab4da (0x2c03 bytes) 000132[1] 1711091641.129937 read 0x79721a thru 0x79b133 (0x3f1a bytes) 000137[0] 1711091641.140374 trunc from 0x7d40e8 to 0x1ca1db (0x609f0c bytes) 000142[0] 1711091641.175997 mapread 0x0d58e2 thru 0x0da41c (0x4b3b bytes) 000143[0] 1711091641.188745 mapread 0x1a0feb thru 0x1af684 (0xe69a bytes) 000144[1] 1711091641.210603 write 0x61fb3c thru 0x62aa75 (0xaf3a bytes) 000147[1] 1711091641.228403 write 0x9beeb6 thru 0x9caf9c (0xc0e7 bytes) 000149[1] 1711091641.232919 write 0x30c17b thru 0x31032c (0x41b2 bytes) 000151[1] 1711091641.251521 trunc from 0x9caf9d to 0x6b5198 (0x315e04 bytes) 000155[1] 1711091641.271846 trunc from 0x6b5199 to 0x1e251e (0x4d2c7a bytes) 000156[1] 1711091641.280594 mapread 0x1211c2 thru 0x129d21 (0x8b60 bytes) 000158[1] 1711091641.293882 mapread 0x16391b thru 0x168084 (0x476a bytes) 000159[1] 1711091641.294374 read 0x031492 thru 0x035422 (0x3f91 bytes) 000160[0] 1711091641.294596 read 0x178d32 thru 0x187326 (0xe5f5 bytes) 000161[1] 1711091641.313643 mapread 0x1a6248 thru 0x1b2b03 (0xc8bc bytes) 000162[0] 1711091641.331463 read 0x0e27d0 thru 0x0eee6a (0xc69b bytes) 000164[0] 1711091641.332255 mapwrite 0x616bcd thru 0x626195 (0xf5c9 bytes) 000165[0] 1711091641.400405 trunc from 0x626196 to 0x12a303 (0x4fbe92 bytes) 000168[1] 1711091641.423734 trunc from 0x12a304 to 0x583c57 (0x459954 bytes) 000169[1] 1711091641.444105 mapwrite 0x0be80a thru 0x0c4447 (0x5c3e bytes) 000170[0] 1711091641.507321 read 0x2471b1 thru 0x253fc2 (0xce12 bytes) 000171[1] 1711091641.542930 write 0x71ba7b thru 0x726c6a (0xb1f0 bytes) 000172[0] 1711091641.544627 write 0x148369 thru 0x14b1f8 (0x2e90 bytes) 000176[0] 1711091641.598287 trunc from 0x726c6b to 0x47842a (0x2ae840 bytes) 000179[1] 1711091641.622732 write 0x4c7a0b thru 0x4d480e (0xce04 bytes) 000180[0] 1711091641.633361 mapread 0x1c71f1 thru 0x1c7d83 (0xb93 bytes) 000182[1] 1711091641.652313 mapwrite 0x49cfc4 thru 0x4a431c (0x7359 bytes) 000183[1] 1711091641.698065 trunc from 0x4d480f to 0x1a60f9 (0x32e715 bytes) 000184[1] 1711091641.721145 mapwrite 0x087e11 thru 0x08c0a8 (0x4298 bytes) 000185[1] 1711091641.770423 trunc from 0x1a60fa to 0x79538a (0x5ef291 bytes) 000188[0] 1711091641.782878 write 0x15204e thru 0x15de68 (0xbe1b bytes) 000189[1] 1711091641.800387 mapread 0x153813 thru 0x15ead5 (0xb2c3 bytes) 000195[1] 1711091641.838773 trunc from 0x79538b to 0x5d8e12 (0x1bc578 bytes) 000196[0] 1711091641.857890 write 0x55f49f thru 0x55f6c2 (0x224 bytes) 000197[1] 1711091641.886661 read 0x0cfb51 thru 0x0dcf40 (0xd3f0 bytes) 000198[1] 1711091641.898118 mapwrite 0x049f4e thru 0x0533b8 (0x946b bytes) 000201[1] 1711091641.966594 read 0x1f9f0d thru 0x1fdcde (0x3dd2 bytes) 000204[0] 1711091642.005407 trunc from 0x5d8e13 to 0x225dde (0x3b3034 bytes) 000206[1] 1711091642.027361 trunc from 0x225ddf to 0x481e50 (0x25c072 bytes) 000207[0] 1711091642.043933 write 0x9b8f51 thru 0x9b9ec3 (0xf73 bytes) 000213[0] 1711091642.081677 trunc from 0x9b9ec4 to 0x643c12 (0x3762b1 bytes) 000214[1] 1711091642.097266 mapwrite 0x8b8ab0 thru 0x8c14fb (0x8a4c bytes) 000217[0] 1711091642.182652 mapread 0x68f0d1 thru 0x698d5b (0x9c8b bytes) 000220[0] 1711091642.225363 mapread 0x74eef4 thru 0x757fcf (0x90dc bytes) 000221[1] 1711091642.237237 mapread 0x57200f thru 0x57a99c (0x898e bytes) 000224[1] 1711091642.265952 trunc from 0x8c14fc to 0x71e3ee (0x1a310d bytes) 000226[0] 1711091642.293927 mapwrite 0x006c50 thru 0x00ab78 (0x3f29 bytes) 000228[0] 1711091642.352789 read 0x41c08d thru 0x429937 (0xd8ab bytes) 000230[1] 1711091642.405463 trunc from 0x71e3ef to 0x567276 (0x1b7178 bytes) 000231[1] 1711091642.439041 trunc from 0x567277 to 0x902e33 (0x39bbbd bytes) 000233[0] 1711091642.457794 trunc from 0x902e34 to 0x812fdf (0xefe54 bytes) 000235[0] 1711091642.483862 read 0x76e5fa thru 0x770138 (0x1b3f bytes) 000237[1] 1711091642.526260 trunc from 0x812fe0 to 0x17f947 (0x693698 bytes) 000239[1] 1711091642.556629 mapread 0x0fafea thru 0x1083fd (0xd414 bytes) 000240[1] 1711091642.579462 read 0x0eea2d thru 0x0fe4d3 (0xfaa7 bytes) 000245[1] 1711091642.598896 mapread 0x051268 thru 0x05c899 (0xb632 bytes) 000247[0] 1711091642.607380 mapwrite 0x7c2c4d thru 0x7c3edc (0x1290 bytes) 000248[1] 1711091642.641741 write 0x16257b thru 0x168db2 (0x6838 bytes) 000250[0] 1711091642.680271 read 0x7256bd thru 0x733f87 (0xe8cb bytes) 000251[1] 1711091642.719268 mapread 0x5be308 thru 0x5cbf95 (0xdc8e bytes) 000252[0] 1711091642.756464 write 0x9ff57c thru 0x9fffff (0xa84 bytes) 000253[1] 1711091642.772457 mapwrite 0x525daf thru 0x533a09 (0xdc5b bytes) 000254[1] 1711091642.865297 mapwrite 0x7b37b6 thru 0x7c2ff9 (0xf844 bytes) 000255[1] 1711091642.919726 read 0x3f9737 thru 0x405797 (0xc061 bytes) 000256[0] 1711091642.949376 trunc from 0xa00000 to 0x4c1c10 (0x53e3ef bytes) 000258[1] 1711091642.967078 write 0x6212fa thru 0x6293e3 (0x80ea bytes) 000263[0] 1711091642.986136 read 0x0d277a thru 0x0dfbfd (0xd484 bytes) 000265[0] 1711091643.009089 mapread 0x1ab800 thru 0x1affa8 (0x47a9 bytes) 000267[1] 1711091643.020093 trunc from 0x6293e4 to 0x77961d (0x15023a bytes) 000269[1] 1711091643.030695 read 0x200829 thru 0x203450 (0x2c28 bytes) 000270[1] 1711091643.055980 read 0x711a7a thru 0x7188fe (0x6e85 bytes) 000271[0] 1711091643.068148 trunc from 0x77961e to 0x91aac3 (0x1a14a6 bytes) 000272[0] 1711091643.091684 mapwrite 0x05baf4 thru 0x065a4b (0x9f58 bytes) 000273[0] 1711091643.163484 mapread 0x75236a thru 0x7542cd (0x1f64 bytes) 000275[0] 1711091643.164191 trunc from 0x91aac4 to 0x55ccb1 (0x3bde12 bytes) 000276[1] 1711091643.173725 mapwrite 0x76b65a thru 0x7740d0 (0x8a77 bytes) 000278[0] 1711091643.235910 mapwrite 0x8c77a9 thru 0x8c8708 (0xf60 bytes) 000279[0] 1711091643.270924 trunc from 0x8c8709 to 0x132505 (0x796203 bytes) 000280[0] 1711091643.276179 mapread 0x054353 thru 0x0634f0 (0xf19e bytes) 000281[1] 1711091643.298105 write 0x0f5b1a thru 0x0fcfce (0x74b5 bytes) 000282[0] 1711091643.310924 mapwrite 0x26c178 thru 0x271752 (0x55db bytes) 000284[0] 1711091643.354649 read 0x125dc3 thru 0x131f89 (0xc1c7 bytes) 000288[1] 1711091643.376537 write 0x239641 thru 0x23db0b (0x44cb bytes) 000291[1] 1711091643.402847 mapwrite 0x7a5d92 thru 0x7b2ef5 (0xd164 bytes) 000294[0] 1711091643.473461 write 0x0c1f92 thru 0x0ca96e (0x89dd bytes) 000295[0] 1711091643.489155 write 0x74c27d thru 0x7532c3 (0x7047 bytes) 000298[0] 1711091643.502401 mapread 0x5cc72d thru 0x5ce194 (0x1a68 bytes) 000299[1] 1711091643.509389 trunc from 0x7b2ef6 to 0x84a856 (0x97961 bytes) 000302[1] 1711091643.525813 mapread 0x6104b7 thru 0x61409c (0x3be6 bytes) 000303[1] 1711091643.542558 write 0x4b7d87 thru 0x4b8e19 (0x1093 bytes) 000305[1] 1711091643.550039 trunc from 0x84a857 to 0x45250e (0x3f8348 bytes) 000306[0] 1711091643.559753 mapread 0x44fbeb thru 0x45250e (0x2924 bytes) 000310[0] 1711091643.577369 mapread 0x37f33e thru 0x383a9e (0x4761 bytes) 000315[1] 1711091643.593651 mapwrite 0x76ce54 thru 0x7762ef (0x949c bytes) 000316[0] 1711091643.651690 trunc from 0x7762f0 to 0x123d7e (0x652571 bytes) 000319[0] 1711091643.674279 read 0x1142f8 thru 0x11e397 (0xa0a0 bytes) 000320[0] 1711091643.678704 mapread 0x09fe39 thru 0x0a591d (0x5ae5 bytes) 000324[1] 1711091643.703101 read 0x0f6750 thru 0x0f74ee (0xd9f bytes) 000325[1] 1711091643.718151 write 0x01c18e thru 0x01fadb (0x394e bytes) 000328[1] 1711091643.748299 write 0x21386e thru 0x215ac3 (0x2256 bytes) 000330[1] 1711091643.763143 read 0x14e238 thru 0x1519e7 (0x37b0 bytes) 000332[1] 1711091643.790137 mapwrite 0x7a4f56 thru 0x7b05f5 (0xb6a0 bytes) 000333[0] 1711091643.872703 trunc from 0x7b05f6 to 0x3f666b (0x3b9f8a bytes) 000337[1] 1711091643.905749 mapwrite 0x962803 thru 0x971a72 (0xf270 bytes) 000338[1] 1711091643.961825 mapread 0x4bb546 thru 0x4bd574 (0x202f bytes) 000340[1] 1711091643.971155 read 0x516646 thru 0x518a82 (0x243d bytes) 000341[0] 1711091643.999452 read 0x05413c thru 0x05ad05 (0x6bca bytes) 000343[0] 1711091644.050714 mapread 0x626706 thru 0x62f817 (0x9112 bytes) 000344[1] 1711091644.090098 read 0x45da11 thru 0x46133a (0x392a bytes) 000345[0] 1711091644.102503 read 0x91b11c thru 0x922d3f (0x7c24 bytes) 000347[1] 1711091644.117410 mapwrite 0x9c496d thru 0x9c750e (0x2ba2 bytes) 000350[0] 1711091644.236918 mapread 0x67861e thru 0x678fc1 (0x9a4 bytes) 000351[1] 1711091644.256433 read 0x32ed59 thru 0x32f6d4 (0x97c bytes) 000354[0] 1711091644.257441 mapwrite 0x3b2be4 thru 0x3bdc7a (0xb097 bytes) 000356[0] 1711091644.347248 write 0x6b5759 thru 0x6bd5a7 (0x7e4f bytes) 000357[1] 1711091644.355715 trunc from 0x9c750f to 0x82752c (0x19ffe2 bytes) 000358[1] 1711091644.383091 write 0x8111d0 thru 0x8134e5 (0x2316 bytes) 000360[1] 1711091644.441251 mapread 0x76d18e thru 0x776295 (0x9108 bytes) 000361[1] 1711091644.446322 read 0x3971ae thru 0x397f0d (0xd60 bytes) 000362[1] 1711091644.446752 mapwrite 0x11a44c thru 0x120b73 (0x6728 bytes) 000363[1] 1711091644.470065 mapwrite 0x551f83 thru 0x552717 (0x795 bytes) 000367[0] 1711091644.492267 mapwrite 0x0ac9b2 thru 0x0b0e26 (0x4475 bytes) 000368[0] 1711091644.531072 mapread 0x4fd6cc thru 0x5061bd (0x8af2 bytes) 000372[0] 1711091644.568098 write 0x2652d7 thru 0x26833e (0x3068 bytes) 000376[1] 1711091644.586472 read 0x1931e4 thru 0x1a2cf1 (0xfb0e bytes) 000379[1] 1711091644.621726 read 0x557d6b thru 0x55d350 (0x55e6 bytes) 000380[1] 1711091644.630603 write 0x37d87a thru 0x383392 (0x5b19 bytes) 000381[1] 1711091644.657245 read 0x58842e thru 0x594fdc (0xcbaf bytes) 000382[0] 1711091644.661113 trunc from 0x82752d to 0x52af7d (0x2fc5af bytes) 000385[1] 1711091644.685371 trunc from 0x52af7e to 0x1ea3ce (0x340baf bytes) 000386[0] 1711091644.695502 mapwrite 0x5ce1b8 thru 0x5d851b (0xa364 bytes) 000387[0] 1711091644.729788 trunc from 0x5d851c to 0x84d861 (0x275346 bytes) 000388[0] 1711091644.737446 trunc from 0x84d862 to 0x3865fc (0x4c7265 bytes) 000393[1] 1711091644.751838 read 0x219bb6 thru 0x220389 (0x67d4 bytes) 000397[0] 1711091644.779464 trunc from 0x3865fd to 0x303a2d (0x82bcf bytes) 000398[0] 1711091644.788356 mapread 0x1678d6 thru 0x16ebfa (0x7325 bytes) 000399[1] 1711091644.797151 write 0x5e1c7d thru 0x5e9548 (0x78cc bytes) 000401[0] 1711091644.807322 mapwrite 0x0a2fa9 thru 0x0acf81 (0x9fd9 bytes) 000402[1] 1711091644.841449 mapwrite 0x0faec9 thru 0x1080e8 (0xd220 bytes) 000404[1] 1711091644.878389 mapread 0x30e8b8 thru 0x31803a (0x9783 bytes) 000405[0] 1711091644.894089 mapread 0x415828 thru 0x416e28 (0x1601 bytes) 000407[0] 1711091644.906407 read 0x4a392d thru 0x4a95ed (0x5cc1 bytes) 000411[1] 1711091644.913537 mapread 0x3e48ad thru 0x3f0a64 (0xc1b8 bytes) 000413[0] 1711091644.932603 write 0x4ac676 thru 0x4aef8c (0x2917 bytes) 000423[1] 1711091644.972811 trunc from 0x5e9549 to 0x1bfec3 (0x429685 bytes) 000425[0] 1711091644.986225 mapwrite 0x7c473c thru 0x7d387d (0xf142 bytes) 000428[0] 1711091645.043551 write 0x0008ff thru 0x000d9a (0x49c bytes) 000429[1] 1711091645.049041 mapwrite 0x308ef9 thru 0x30975d (0x865 bytes) 000431[0] 1711091645.100896 read 0x2054ae thru 0x20d463 (0x7fb6 bytes) 000432[1] 1711091645.129807 mapread 0x23fdb0 thru 0x241c0c (0x1e5d bytes) 000434[0] 1711091645.142068 write 0x3046b0 thru 0x307d2a (0x367b bytes) 000436[1] 1711091645.154631 trunc from 0x7d387e to 0x50e1d8 (0x2c56a5 bytes) 000438[0] 1711091645.177037 read 0x0b8e6f thru 0x0c282c (0x99be bytes) 000439[1] 1711091645.206442 trunc from 0x50e1d9 to 0x607acb (0xf98f3 bytes) 000444[0] 1711091645.236052 read 0x49254b thru 0x49aa65 (0x851b bytes) 000447[0] 1711091645.263175 trunc from 0x607acc to 0x4aa6e2 (0x15d3e9 bytes) 000449[1] 1711091645.278833 read 0x188a5e thru 0x19651d (0xdac0 bytes) 000452[0] 1711091645.324099 read 0x1720cd thru 0x17c99c (0xa8d0 bytes) 000454[1] 1711091645.358660 read 0x14ab32 thru 0x1554a1 (0xa970 bytes) 000456[0] 1711091645.366437 trunc from 0x4aa6e3 to 0x6d3805 (0x229123 bytes) 000459[0] 1711091645.401382 mapread 0x554cf6 thru 0x558144 (0x344f bytes) 000460[0] 1711091645.424848 write 0x97692d thru 0x9841db (0xd8af bytes) 000463[0] 1711091645.440677 mapwrite 0x3f4a0d thru 0x3f8819 (0x3e0d bytes) 000466[0] 1711091645.495094 trunc from 0x9841dc to 0x01624a (0x96df91 bytes) 000471[0] 1711091645.532372 write 0x53baf2 thru 0x5445b5 (0x8ac4 bytes) 000473[1] 1711091645.535182 read 0x2a6a7c thru 0x2ab7c6 (0x4d4b bytes) 000478[1] 1711091645.581974 trunc from 0x5445b6 to 0x962451 (0x41de9c bytes) 000479[0] 1711091645.603044 write 0x09cdd0 thru 0x09d90c (0xb3d bytes) 000482[1] 1711091645.634668 mapread 0x95601c thru 0x95f084 (0x9069 bytes) 000486[1] 1711091645.715161 mapwrite 0x1aba3d thru 0x1b33f8 (0x79bc bytes) 000488[0] 1711091645.796971 read 0x12167d thru 0x1286e4 (0x7068 bytes) 000491[0] 1711091645.844840 mapwrite 0x516a00 thru 0x51909c (0x269d bytes) 000494[1] 1711091645.939277 read 0x1da144 thru 0x1e490d (0xa7ca bytes) 000495[0] 1711091645.965776 trunc from 0x962452 to 0x0591fa (0x909257 bytes) 000496[1] 1711091645.990011 trunc from 0x0591fb to 0x6ce6f1 (0x6754f7 bytes) 000500[1] 1711091646.014830 read 0x494003 thru 0x4a1104 (0xd102 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 1646 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read 000005[0] 1711091646.340332 mapwrite 0x3ca000 thru 0x3cf436 (0x5437 bytes) 000008[1] 1711091646.388959 mapread 0x005000 thru 0x00f1c3 (0xa1c4 bytes) 000009[0] 1711091646.406281 mapread 0x07e000 thru 0x08023d (0x223e bytes) truncating to largest ever: 0xc1c99 000010[0] 1711091646.413435 trunc from 0x3cf437 to 0x0c1c98 (0x30d79e bytes) 000011[1] 1711091646.427018 mapwrite 0x906000 thru 0x90db5c (0x7b5d bytes) 000012[1] 1711091646.461265 mapwrite 0x3c3000 thru 0x3cb8ae (0x88af bytes) 000014[1] 1711091646.498844 mapwrite 0x9d0000 thru 0x9deb8b (0xeb8c bytes) 000017[1] 1711091646.536317 read 0x281000 thru 0x283313 (0x2314 bytes) 000018[1] 1711091646.538580 mapwrite 0x4cb000 thru 0x4cfabc (0x4abd bytes) 000019[0] 1711091646.564117 read 0x9be000 thru 0x9c9f81 (0xbf82 bytes) 000023[1] 1711091646.574728 read 0x140000 thru 0x14bc40 (0xbc41 bytes) 000024[1] 1711091646.611003 read 0x14a000 thru 0x159a14 (0xfa15 bytes) 000025[1] 1711091646.611407 read 0x580000 thru 0x588fa9 (0x8faa bytes) truncating to largest ever: 0x53419d 000027[1] 1711091646.611954 trunc from 0x9deb8c to 0x53419c (0x4aa9ef bytes) 000028[1] 1711091646.629812 mapwrite 0x007000 thru 0x0093e2 (0x23e3 bytes) 000029[1] 1711091646.673576 mapread 0x0fc000 thru 0x107ba9 (0xbbaa bytes) 000030[1] 1711091646.674112 read 0x1ab000 thru 0x1b5d73 (0xad74 bytes) 000032[1] 1711091646.682783 trunc from 0x53419d to 0x18e587 (0x3a5c15 bytes) 000034[0] 1711091646.696401 mapread 0x0dc000 thru 0x0debd8 (0x2bd9 bytes) 000035[1] 1711091646.707305 write 0x9f5000 thru 0x9f5e58 (0xe59 bytes) 000037[1] 1711091646.712058 mapwrite 0x10b000 thru 0x11761d (0xc61e bytes) truncating to largest ever: 0x7bc76f 000040[1] 1711091646.757790 trunc from 0x9f5e59 to 0x7bc76e (0x2396ea bytes) 000042[1] 1711091646.771962 read 0x70e000 thru 0x716c32 (0x8c33 bytes) 000044[0] 1711091646.778316 mapwrite 0x953000 thru 0x95c772 (0x9773 bytes) 000046[1] 1711091646.828332 write 0x10d000 thru 0x1161d2 (0x91d3 bytes) 000047[1] 1711091646.838256 trunc from 0x95c773 to 0x31343b (0x649337 bytes) 000052[0] 1711091646.852157 mapread 0x270000 thru 0x27d4a7 (0xd4a8 bytes) truncating to largest ever: 0x802306 000054[0] 1711091646.880661 trunc from 0x31343c to 0x802305 (0x4eeeca bytes) 000057[0] 1711091646.914443 read 0x795000 thru 0x7a2079 (0xd07a bytes) 000062[0] 1711091646.986243 trunc from 0x802306 to 0x44001c (0x3c22e9 bytes) 000063[1] 1711091647.006348 write 0x8d2000 thru 0x8e157b (0xf57c bytes) 000064[0] 1711091647.017618 read 0x195000 thru 0x19a613 (0x5614 bytes) 000067[0] 1711091647.076428 mapwrite 0x0b1000 thru 0x0b8b8c (0x7b8d bytes) 000068[0] 1711091647.130434 trunc from 0x8e157c to 0x5c4f54 (0x31c627 bytes) 000071[1] 1711091647.142410 write 0x7b3000 thru 0x7bf1f9 (0xc1fa bytes) 000072[0] 1711091647.149809 write 0x89a000 thru 0x8a4cec (0xaced bytes) 000073[0] 1711091647.159674 mapread 0x2fd000 thru 0x2ff015 (0x2016 bytes) 000081[1] 1711091647.179945 write 0x623000 thru 0x62778e (0x478f bytes) 000082[0] 1711091647.202551 read 0x58f000 thru 0x59dd51 (0xed52 bytes) truncating to largest ever: 0x90de1d 000085[1] 1711091647.249587 trunc from 0x8a4ced to 0x90de1c (0x69130 bytes) 000086[0] 1711091647.265388 read 0x0ed000 thru 0x0f262a (0x562b bytes) 000088[1] 1711091647.292782 trunc from 0x90de1d to 0x421b8d (0x4ec28f bytes) 000090[0] 1711091647.317470 write 0x6a3000 thru 0x6b099a (0xd99b bytes) 000093[1] 1711091647.322416 trunc from 0x6b099b to 0x4fcadf (0x1b3ebb bytes) 000094[0] 1711091647.338706 read 0x028000 thru 0x034828 (0xc829 bytes) 000095[0] 1711091647.358327 write 0x841000 thru 0x842ab7 (0x1ab8 bytes) 000096[1] 1711091647.370358 mapwrite 0x721000 thru 0x7223ed (0x13ee bytes) 000099[1] 1711091647.412094 trunc from 0x842ab8 to 0x67ebe4 (0x1c3ed3 bytes) 000101[1] 1711091647.418855 mapwrite 0x960000 thru 0x96774b (0x774c bytes) 000102[0] 1711091647.448986 write 0x301000 thru 0x30409c (0x309d bytes) 000103[1] 1711091647.474876 mapread 0x0b3000 thru 0x0bf18a (0xc18b bytes) 000104[1] 1711091647.497070 mapwrite 0x884000 thru 0x88a93a (0x693b bytes) 000106[1] 1711091647.522330 mapread 0x489000 thru 0x48e8a0 (0x58a1 bytes) 000109[1] 1711091647.528801 mapread 0x312000 thru 0x31a6a9 (0x86aa bytes) 000110[0] 1711091647.538709 read 0x0e6000 thru 0x0e6327 (0x328 bytes) 000112[0] 1711091647.573554 write 0x3c5000 thru 0x3c6859 (0x185a bytes) 000114[0] 1711091647.583881 mapwrite 0x8a4000 thru 0x8b266e (0xe66f bytes) 000115[0] 1711091647.616955 write 0x79b000 thru 0x79c4e7 (0x14e8 bytes) 000122[1] 1711091647.649935 mapread 0x7c6000 thru 0x7c7fbf (0x1fc0 bytes) 000123[1] 1711091647.659353 read 0x64b000 thru 0x6584c3 (0xd4c4 bytes) 000125[0] 1711091647.701729 write 0x983000 thru 0x983687 (0x688 bytes) 000127[0] 1711091647.713204 read 0x911000 thru 0x9134dc (0x24dd bytes) 000128[0] 1711091647.749965 mapread 0x68b000 thru 0x698496 (0xd497 bytes) 000129[0] 1711091647.750417 mapread 0x8f5000 thru 0x8fd793 (0x8794 bytes) 000130[1] 1711091647.750702 write 0x295000 thru 0x29bf90 (0x6f91 bytes) 000132[1] 1711091647.773524 read 0x941000 thru 0x94be3e (0xae3f bytes) 000133[0] 1711091647.822711 mapwrite 0x624000 thru 0x631d08 (0xdd09 bytes) 000134[0] 1711091647.908273 read 0x148000 thru 0x1538e8 (0xb8e9 bytes) 000135[0] 1711091647.960006 mapwrite 0x6ff000 thru 0x70e8d0 (0xf8d1 bytes) 000140[0] 1711091648.002764 mapwrite 0x022000 thru 0x022766 (0x767 bytes) 000144[0] 1711091648.026341 read 0x69e000 thru 0x6a2b86 (0x4b87 bytes) 000145[1] 1711091648.029927 mapwrite 0x3ed000 thru 0x3fb401 (0xe402 bytes) 000147[1] 1711091648.076625 read 0x0a7000 thru 0x0b22f8 (0xb2f9 bytes) 000148[1] 1711091648.109727 mapwrite 0x97d000 thru 0x986aa9 (0x9aaa bytes) 000149[0] 1711091648.134576 trunc from 0x986aaa to 0x02f759 (0x957350 bytes) 000150[0] 1711091648.148561 read 0x013000 thru 0x021a4f (0xea50 bytes) 000153[0] 1711091648.156190 mapread 0x018000 thru 0x019e96 (0x1e97 bytes) 000154[0] 1711091648.159912 mapwrite 0x846000 thru 0x84626f (0x270 bytes) truncating to largest ever: 0x99bba5 000155[1] 1711091648.215798 trunc from 0x846270 to 0x99bba4 (0x155935 bytes) 000159[1] 1711091648.233456 mapread 0x826000 thru 0x827083 (0x1084 bytes) 000160[0] 1711091648.237461 read 0x5fb000 thru 0x6098bc (0xe8bd bytes) 000161[1] 1711091648.275384 write 0x690000 thru 0x699409 (0x940a bytes) 000164[1] 1711091648.291428 mapread 0x890000 thru 0x896334 (0x6335 bytes) 000170[1] 1711091648.333566 mapwrite 0x419000 thru 0x4216da (0x86db bytes) 000171[0] 1711091648.387382 mapread 0x37b000 thru 0x381ffa (0x6ffb bytes) 000173[0] 1711091648.411349 write 0x9e5000 thru 0x9ed997 (0x8998 bytes) 000174[1] 1711091648.423174 mapread 0x2f3000 thru 0x2f6dae (0x3daf bytes) 000175[0] 1711091648.448739 mapwrite 0x1c0000 thru 0x1c0185 (0x186 bytes) 000176[0] 1711091648.494079 mapread 0x9da000 thru 0x9dd80f (0x3810 bytes) 000180[1] 1711091648.501672 mapread 0x88c000 thru 0x88d9f7 (0x19f8 bytes) 000181[0] 1711091648.515982 read 0x1fe000 thru 0x20c49c (0xe49d bytes) 000182[1] 1711091648.571773 trunc from 0x9ed998 to 0x4b2250 (0x53b747 bytes) 000183[1] 1711091648.585372 write 0x8eb000 thru 0x8ec2d1 (0x12d2 bytes) 000185[0] 1711091648.586593 mapread 0x877000 thru 0x87e6d1 (0x76d2 bytes) 000186[1] 1711091648.601018 mapread 0x5f7000 thru 0x5f7c44 (0xc45 bytes) 000187[0] 1711091648.603625 write 0x234000 thru 0x23ff52 (0xbf53 bytes) 000189[0] 1711091648.607058 mapread 0x5e0000 thru 0x5eda42 (0xda43 bytes) 000190[0] 1711091648.622024 write 0x47a000 thru 0x47eaed (0x4aee bytes) 000191[0] 1711091648.626776 trunc from 0x8ec2d2 to 0x825d6c (0xc6565 bytes) 000194[1] 1711091648.631265 mapread 0x6e5000 thru 0x6e5fca (0xfcb bytes) 000195[0] 1711091648.638865 read 0x363000 thru 0x3701f9 (0xd1fa bytes) 000196[0] 1711091648.663257 trunc from 0x825d6d to 0x4dcfe6 (0x348d86 bytes) 000198[0] 1711091648.683587 mapread 0x452000 thru 0x45a145 (0x8146 bytes) 000201[0] 1711091648.709657 read 0x288000 thru 0x295545 (0xd546 bytes) 000204[0] 1711091648.751974 read 0x422000 thru 0x429862 (0x7863 bytes) 000205[1] 1711091648.758363 mapwrite 0x2fb000 thru 0x2fbadd (0xade bytes) 000206[1] 1711091648.797023 read 0x37e000 thru 0x389d3b (0xbd3c bytes) 000207[0] 1711091648.830307 write 0x53b000 thru 0x53b46b (0x46c bytes) 000208[1] 1711091648.849641 write 0x5b3000 thru 0x5ba92b (0x792c bytes) 000210[0] 1711091648.871106 mapwrite 0x181000 thru 0x187480 (0x6481 bytes) 000211[0] 1711091648.951766 mapread 0x52b000 thru 0x533d5c (0x8d5d bytes) 000218[1] 1711091648.974609 mapwrite 0x4a1000 thru 0x4a587f (0x4880 bytes) 000219[1] 1711091649.041588 read 0x3dc000 thru 0x3de0ef (0x20f0 bytes) 000220[0] 1711091649.081780 mapread 0x348000 thru 0x351b5d (0x9b5e bytes) 000225[0] 1711091649.124997 write 0x9ca000 thru 0x9d13da (0x73db bytes) 000227[0] 1711091649.139136 trunc from 0x9d13db to 0x97d352 (0x54088 bytes) 000229[1] 1711091649.146652 mapwrite 0x43b000 thru 0x444029 (0x902a bytes) 000230[0] 1711091649.218558 trunc from 0x97d353 to 0x1378e8 (0x845a6a bytes) 000231[0] 1711091649.242586 write 0x45f000 thru 0x46ecb5 (0xfcb6 bytes) 000233[1] 1711091649.247982 mapread 0x03c000 thru 0x03fd81 (0x3d82 bytes) 000234[0] 1711091649.269444 mapwrite 0x4ce000 thru 0x4d282c (0x482d bytes) 000238[1] 1711091649.326710 write 0x0f3000 thru 0x0f6a4f (0x3a50 bytes) 000240[0] 1711091649.336671 read 0x3c8000 thru 0x3d709e (0xf09f bytes) 000241[0] 1711091649.373188 write 0x514000 thru 0x519b2d (0x5b2e bytes) 000244[1] 1711091649.392142 mapread 0x2fa000 thru 0x305502 (0xb503 bytes) 000245[1] 1711091649.426349 mapwrite 0x8aa000 thru 0x8b0ee3 (0x6ee4 bytes) 000246[1] 1711091649.474987 mapread 0x44d000 thru 0x450ef4 (0x3ef5 bytes) 000248[1] 1711091649.489402 trunc from 0x8b0ee4 to 0x92c21b (0x7b338 bytes) truncating to largest ever: 0x9bea31 000249[0] 1711091649.496380 trunc from 0x92c21c to 0x9bea30 (0x92815 bytes) 000251[1] 1711091649.516536 mapwrite 0x506000 thru 0x5088ba (0x28bb bytes) 000253[0] 1711091649.567179 read 0x91c000 thru 0x92121b (0x521c bytes) 000254[1] 1711091649.624093 read 0x230000 thru 0x2309bc (0x9bd bytes) 000257[0] 1711091649.663928 write 0x11e000 thru 0x11f2e9 (0x12ea bytes) 000258[1] 1711091649.685872 mapwrite 0x09d000 thru 0x0a1c21 (0x4c22 bytes) 000260[1] 1711091649.726835 mapwrite 0x488000 thru 0x4910cf (0x90d0 bytes) 000261[0] 1711091649.765147 trunc from 0x9bea31 to 0x161f7f (0x85cab1 bytes) 000262[0] 1711091649.780643 mapwrite 0x2c6000 thru 0x2cb436 (0x5437 bytes) 000266[1] 1711091649.852102 trunc from 0x2cb437 to 0x5b2854 (0x2e741e bytes) 000269[0] 1711091649.878333 mapwrite 0x549000 thru 0x551d59 (0x8d5a bytes) 000270[0] 1711091649.930256 write 0x212000 thru 0x214933 (0x2934 bytes) 000274[0] 1711091649.942856 mapwrite 0x731000 thru 0x737c4b (0x6c4c bytes) 000275[0] 1711091649.983283 read 0x63d000 thru 0x643375 (0x6376 bytes) 000287[1] 1711091650.034986 mapread 0x2c8000 thru 0x2d0ec8 (0x8ec9 bytes) 000289[1] 1711091650.069686 mapread 0x4b8000 thru 0x4bff16 (0x7f17 bytes) 000290[0] 1711091650.086064 read 0x02e000 thru 0x03c4c9 (0xe4ca bytes) 000292[1] 1711091650.124723 mapwrite 0x2ed000 thru 0x2f64ad (0x94ae bytes) 000293[0] 1711091650.190901 mapread 0x65a000 thru 0x6645b0 (0xa5b1 bytes) 000294[1] 1711091650.216477 write 0x022000 thru 0x02da92 (0xba93 bytes) 000297[1] 1711091650.228841 mapwrite 0x559000 thru 0x560a4f (0x7a50 bytes) 000298[1] 1711091650.260941 mapwrite 0x647000 thru 0x655b3b (0xeb3c bytes) 000299[0] 1711091650.318150 mapread 0x629000 thru 0x62c068 (0x3069 bytes) 000302[1] 1711091650.338181 write 0x900000 thru 0x90942a (0x942b bytes) 000304[0] 1711091650.350595 mapwrite 0x245000 thru 0x24f41d (0xa41e bytes) 000306[0] 1711091650.421914 trunc from 0x90942b to 0x4b469d (0x454d8d bytes) 000307[1] 1711091650.431060 read 0x347000 thru 0x34c583 (0x5584 bytes) 000308[0] 1711091650.458320 read 0x499000 thru 0x4a1eb5 (0x8eb6 bytes) 000310[1] 1711091650.486297 trunc from 0x4b469e to 0x7707a4 (0x2bc107 bytes) 000311[0] 1711091650.508659 read 0x52f000 thru 0x53ce36 (0xde37 bytes) 000312[1] 1711091650.549584 mapwrite 0x458000 thru 0x462d75 (0xad76 bytes) 000313[1] 1711091650.615548 mapread 0x293000 thru 0x296909 (0x390a bytes) 000314[1] 1711091650.621952 read 0x43e000 thru 0x44b01f (0xd020 bytes) truncating to largest ever: 0x9e04fb 000316[1] 1711091650.646700 trunc from 0x7707a5 to 0x9e04fa (0x26fd56 bytes) 000318[1] 1711091650.662809 trunc from 0x9e04fb to 0x6c7fed (0x31850d bytes) 000319[1] 1711091650.678415 trunc from 0x6c7fee to 0x084b94 (0x643459 bytes) 000320[0] 1711091650.702423 mapwrite 0x26e000 thru 0x26f3f8 (0x13f9 bytes) 000321[0] 1711091650.754155 trunc from 0x26f3f9 to 0x0d6d17 (0x1986e1 bytes) 000324[1] 1711091650.785521 read 0x077000 thru 0x07c3aa (0x53ab bytes) 000326[0] 1711091650.805294 trunc from 0x0d6d18 to 0x10e819 (0x37b02 bytes) 000327[1] 1711091650.818775 mapread 0x080000 thru 0x08c201 (0xc202 bytes) 000328[1] 1711091650.860146 read 0x0af000 thru 0x0b9212 (0xa213 bytes) 000331[0] 1711091650.873132 mapwrite 0x95f000 thru 0x963a40 (0x4a41 bytes) 000334[0] 1711091650.938685 trunc from 0x963a41 to 0x4dab59 (0x488ee7 bytes) 000335[0] 1711091650.951829 mapwrite 0x025000 thru 0x033a03 (0xea04 bytes) 000337[1] 1711091651.011765 mapread 0x303000 thru 0x30ee56 (0xbe57 bytes) 000339[0] 1711091651.045677 read 0x43d000 thru 0x44b8fd (0xe8fe bytes) 000340[1] 1711091651.073165 mapwrite 0x405000 thru 0x40f074 (0xa075 bytes) 000341[1] 1711091651.150027 write 0x443000 thru 0x44d127 (0xa128 bytes) 000342[0] 1711091651.153152 write 0x0d1000 thru 0x0d8c04 (0x7c05 bytes) 000343[0] 1711091651.170749 write 0x6a6000 thru 0x6a7081 (0x1082 bytes) 000344[1] 1711091651.178550 mapwrite 0x93a000 thru 0x93d830 (0x3831 bytes) 000345[0] 1711091651.240909 mapread 0x2ea000 thru 0x2f7545 (0xd546 bytes) 000346[1] 1711091651.318415 mapwrite 0x0a9000 thru 0x0a9419 (0x41a bytes) 000348[0] 1711091651.361949 trunc from 0x93d831 to 0x2f27a3 (0x64b08d bytes) 000350[0] 1711091651.386658 mapwrite 0x661000 thru 0x670f54 (0xff55 bytes) 000352[0] 1711091651.473282 mapwrite 0x670000 thru 0x672c27 (0x2c28 bytes) 000356[1] 1711091651.516308 mapread 0x0d5000 thru 0x0df26c (0xa26d bytes) 000358[1] 1711091651.545905 read 0x4c3000 thru 0x4cba22 (0x8a23 bytes) 000359[0] 1711091651.582672 trunc from 0x672c28 to 0x658330 (0x1a8f7 bytes) 000360[0] 1711091651.602578 trunc from 0x658331 to 0x50461a (0x153d16 bytes) 000361[0] 1711091651.611796 mapwrite 0x4c0000 thru 0x4cca1c (0xca1d bytes) 000364[0] 1711091651.660692 trunc from 0x50461b to 0x91c988 (0x41836e bytes) 000365[1] 1711091651.669275 write 0x7fc000 thru 0x808472 (0xc473 bytes) 000366[0] 1711091651.688954 mapwrite 0x8e6000 thru 0x8f3aba (0xdabb bytes) 000367[0] 1711091651.735174 mapread 0x6df000 thru 0x6e1f3e (0x2f3f bytes) 000370[0] 1711091651.746113 mapread 0x3ba000 thru 0x3bfcc2 (0x5cc3 bytes) 000372[0] 1711091651.763798 write 0x969000 thru 0x96e0a8 (0x50a9 bytes) 000373[0] 1711091651.764649 mapread 0x22e000 thru 0x232d60 (0x4d61 bytes) 000376[0] 1711091651.781392 read 0x3cf000 thru 0x3d752c (0x852d bytes) 000379[1] 1711091651.826933 write 0x2a1000 thru 0x2ad9b2 (0xc9b3 bytes) 000381[1] 1711091651.861020 mapwrite 0x633000 thru 0x642d0e (0xfd0f bytes) 000384[1] 1711091651.919643 mapwrite 0x3cb000 thru 0x3cc3a3 (0x13a4 bytes) 000385[1] 1711091651.952261 mapwrite 0x80f000 thru 0x8142bb (0x52bc bytes) 000388[1] 1711091651.989721 write 0x3c0000 thru 0x3cf8c2 (0xf8c3 bytes) 000392[1] 1711091652.005398 trunc from 0x96e0a9 to 0x31e31f (0x64fd89 bytes) 000393[0] 1711091652.015513 mapwrite 0x9be000 thru 0x9c86aa (0xa6ab bytes) 000394[0] 1711091652.065411 write 0x310000 thru 0x31f11a (0xf11b bytes) 000395[0] 1711091652.070106 trunc from 0x9c86ab to 0x7a1bee (0x226abc bytes) 000401[0] 1711091652.088069 mapread 0x433000 thru 0x43cbde (0x9bdf bytes) 000402[0] 1711091652.111085 write 0x460000 thru 0x460123 (0x124 bytes) 000403[1] 1711091652.113189 trunc from 0x7a1bef to 0x253b80 (0x54e06e bytes) 000410[0] 1711091652.141743 mapwrite 0x190000 thru 0x190164 (0x165 bytes) 000411[1] 1711091652.165749 trunc from 0x253b81 to 0x71dcfa (0x4ca17a bytes) 000413[1] 1711091652.181604 trunc from 0x71dcfb to 0x0ebcd8 (0x632022 bytes) 000414[1] 1711091652.192438 mapwrite 0x6b7000 thru 0x6c637f (0xf380 bytes) 000415[0] 1711091652.255907 trunc from 0x6c6380 to 0x5ad3f9 (0x118f86 bytes) 000416[0] 1711091652.261926 mapwrite 0x24a000 thru 0x2531bf (0x91c0 bytes) 000417[1] 1711091652.298888 write 0x0a3000 thru 0x0a38f5 (0x8f6 bytes) 000418[1] 1711091652.304096 read 0x4bf000 thru 0x4ce9a5 (0xf9a6 bytes) 000419[0] 1711091652.334123 write 0x5ec000 thru 0x5f8ca7 (0xcca8 bytes) 000423[0] 1711091652.358911 mapread 0x4bc000 thru 0x4c99ee (0xd9ef bytes) 000437[0] 1711091652.394975 read 0x04a000 thru 0x04bf70 (0x1f71 bytes) 000438[1] 1711091652.419041 write 0x051000 thru 0x05b598 (0xa599 bytes) 000439[1] 1711091652.431951 read 0x5b2000 thru 0x5b49b7 (0x29b8 bytes) 000440[0] 1711091652.432194 read 0x262000 thru 0x26487e (0x287f bytes) 000441[1] 1711091652.457189 write 0x014000 thru 0x0234aa (0xf4ab bytes) 000442[1] 1711091652.469332 read 0x395000 thru 0x3a1d32 (0xcd33 bytes) 000445[0] 1711091652.493255 trunc from 0x5f8ca8 to 0x891f04 (0x29925d bytes) 000448[1] 1711091652.513542 trunc from 0x891f05 to 0x8faf51 (0x6904d bytes) 000450[0] 1711091652.519009 mapwrite 0x51a000 thru 0x51f0d8 (0x50d9 bytes) 000452[1] 1711091652.559981 mapread 0x449000 thru 0x4563f9 (0xd3fa bytes) 000453[1] 1711091652.595790 read 0x23e000 thru 0x243469 (0x546a bytes) 000454[1] 1711091652.617696 read 0x074000 thru 0x07ae9a (0x6e9b bytes) 000456[0] 1711091652.618136 trunc from 0x8faf52 to 0x0ea032 (0x810f1f bytes) 000458[0] 1711091652.640647 mapread 0x007000 thru 0x008574 (0x1575 bytes) 000459[0] 1711091652.644348 read 0x09c000 thru 0x0a8a82 (0xca83 bytes) 000461[0] 1711091652.655694 trunc from 0x0ea033 to 0x4edb61 (0x403b2f bytes) 000462[1] 1711091652.661262 write 0x525000 thru 0x52be0b (0x6e0c bytes) 000466[1] 1711091652.678795 trunc from 0x52be0c to 0x6a5192 (0x179387 bytes) 000469[0] 1711091652.694060 write 0x1d5000 thru 0x1dbc45 (0x6c46 bytes) 000470[1] 1711091652.714258 read 0x66e000 thru 0x671d13 (0x3d14 bytes) 000474[0] 1711091652.748919 trunc from 0x6a5193 to 0x85b225 (0x1b6093 bytes) 000476[1] 1711091652.767019 write 0x78a000 thru 0x79881c (0xe81d bytes) 000477[0] 1711091652.780675 mapread 0x5b6000 thru 0x5c117b (0xb17c bytes) 000478[1] 1711091652.823495 mapwrite 0x4bd000 thru 0x4c3135 (0x6136 bytes) 000479[1] 1711091652.864423 mapread 0x232000 thru 0x240e60 (0xee61 bytes) 000480[0] 1711091652.881280 write 0x3a4000 thru 0x3aab2a (0x6b2b bytes) 000482[1] 1711091652.888975 read 0x0ae000 thru 0x0b8269 (0xa26a bytes) 000485[1] 1711091652.924964 trunc from 0x85b226 to 0x2f829e (0x562f87 bytes) 000490[0] 1711091652.952022 trunc from 0x2f829f to 0x57748c (0x27f1ee bytes) 000491[0] 1711091652.962319 trunc from 0x57748d to 0x13bfff (0x43b48d bytes) 000496[0] 1711091652.979067 read 0x0ef000 thru 0x0efaff (0xb00 bytes) 000499[0] 1711091652.986699 read 0x081000 thru 0x09081c (0xf81d bytes) 000500[1] 1711091652.990565 read 0x0b6000 thru 0x0bfa7c (0x9a7d bytes) All operations completed A-OK! PASS 16a (16s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 03:14:15 (1711091655) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.143071 s, 58.6 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.136239 s, 61.6 MB/s Chance of close/open is 1 in 50 Seed set to 1656 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x41a07 000003[0] 1711091656.782407 trunc from 00000000 to 0x041a06 (0x41a07 bytes) 000006[0] 1711091656.797535 mapread 0x00ad17 thru 0x00b1a8 (0x492 bytes) truncating to largest ever: 0x5bd753 000009[0] 1711091656.803688 trunc from 0x041a07 to 0x5bd752 (0x57bd4c bytes) 000010[1] 1711091656.809159 mapwrite 0x804be2 thru 0x805842 (0xc61 bytes) 000012[1] 1711091656.852444 mapread 0x53bc36 thru 0x53c7da (0xba5 bytes) 000016[0] 1711091656.858833 read 0x60652a thru 0x608348 (0x1e1f bytes) 000018[1] 1711091656.865909 trunc from 0x805843 to 0x2550c2 (0x5b0780 bytes) 000020[0] 1711091656.877019 mapwrite 0x03ca22 thru 0x03e45f (0x1a3e bytes) 000021[0] 1711091656.900792 read 0x19ddb1 thru 0x19f437 (0x1687 bytes) 000026[0] 1711091656.931476 mapread 0x122091 thru 0x123364 (0x12d4 bytes) 000028[1] 1711091656.939238 write 0x6c3c33 thru 0x6c577b (0x1b49 bytes) 000031[0] 1711091656.949810 read 0x2d1090 thru 0x2d13b6 (0x327 bytes) 000032[0] 1711091656.957833 write 0x7bd38c thru 0x7bd823 (0x498 bytes) 000034[1] 1711091656.965234 write 0x064cfc thru 0x064ed3 (0x1d8 bytes) 000039[0] 1711091656.986682 mapwrite 0x52f9c3 thru 0x530d31 (0x136f bytes) 000041[0] 1711091657.017688 write 0x4211cb thru 0x422319 (0x114f bytes) 000043[0] 1711091657.027327 mapread 0x22e26f thru 0x2301ae (0x1f40 bytes) 000044[1] 1711091657.039254 trunc from 0x7bd824 to 0x50f3a0 (0x2ae483 bytes) truncating to largest ever: 0x79404d 000045[0] 1711091657.051860 trunc from 0x50f3a1 to 0x79404c (0x284cac bytes) 000046[1] 1711091657.059485 write 0x142a68 thru 0x143cfe (0x1297 bytes) 000048[0] 1711091657.076484 read 0x315d30 thru 0x3174d0 (0x17a1 bytes) 000052[0] 1711091657.122704 write 0x1efb82 thru 0x1f0586 (0xa05 bytes) 000053[0] 1711091657.132149 write 0x4a7be5 thru 0x4a8375 (0x791 bytes) 000054[0] 1711091657.135846 mapread 0x1021e8 thru 0x103375 (0x118e bytes) 000055[0] 1711091657.138537 mapread 0x4f516b thru 0x4f6429 (0x12bf bytes) 000057[0] 1711091657.138969 read 0x605033 thru 0x6058b6 (0x884 bytes) 000059[1] 1711091657.139404 read 0x1c470d thru 0x1c58d7 (0x11cb bytes) 000061[0] 1711091657.150490 read 0x13ab3c thru 0x13c921 (0x1de6 bytes) 000062[0] 1711091657.172768 write 0x3c298e thru 0x3c48f1 (0x1f64 bytes) 000063[0] 1711091657.189845 read 0x2239cb thru 0x22419c (0x7d2 bytes) truncating to largest ever: 0x859873 000068[1] 1711091657.232077 trunc from 0x79404d to 0x859872 (0xc5826 bytes) 000069[1] 1711091657.266088 trunc from 0x859873 to 0x16ecbc (0x6eabb6 bytes) 000072[1] 1711091657.288747 trunc from 0x16ecbd to 0x853735 (0x6e4a79 bytes) 000074[1] 1711091657.313073 write 0x69e9c7 thru 0x69f941 (0xf7b bytes) 000075[1] 1711091657.324091 mapwrite 0x455d43 thru 0x456689 (0x947 bytes) 000076[1] 1711091657.353982 mapwrite 0x1819f8 thru 0x182134 (0x73d bytes) 000077[1] 1711091657.384511 read 0x4a3138 thru 0x4a3e19 (0xce2 bytes) 000078[0] 1711091657.431427 read 0x72bf71 thru 0x72dc5a (0x1cea bytes) 000081[1] 1711091657.453045 read 0x3c7534 thru 0x3c828e (0xd5b bytes) 000082[0] 1711091657.453468 write 0x5d9c01 thru 0x5da498 (0x898 bytes) 000084[1] 1711091657.474659 mapwrite 0x133879 thru 0x1357c5 (0x1f4d bytes) 000086[1] 1711091657.512478 mapread 0x0322b5 thru 0x03381e (0x156a bytes) 000091[1] 1711091657.551413 mapwrite 0x31cf2f thru 0x31dc02 (0xcd4 bytes) 000092[1] 1711091657.590514 read 0x3831e2 thru 0x38459e (0x13bd bytes) 000093[0] 1711091657.597051 write 0x79e7ed thru 0x79f7dd (0xff1 bytes) 000095[0] 1711091657.615672 mapread 0x04632c thru 0x047165 (0xe3a bytes) 000096[1] 1711091657.670085 read 0x2d181b thru 0x2d35e6 (0x1dcc bytes) 000099[0] 1711091657.708618 mapread 0x51aa5d thru 0x51c5ea (0x1b8e bytes) 000102[1] 1711091657.759524 write 0x640684 thru 0x641c53 (0x15d0 bytes) 000103[1] 1711091657.810626 mapwrite 0x54ad96 thru 0x54b1ba (0x425 bytes) 000105[0] 1711091657.845402 mapread 0x5698aa thru 0x56b306 (0x1a5d bytes) 000106[1] 1711091657.868637 mapwrite 0x7fab86 thru 0x7fc2ab (0x1726 bytes) 000107[1] 1711091657.908389 read 0x426cb0 thru 0x428c19 (0x1f6a bytes) truncating to largest ever: 0x8651d7 000109[1] 1711091657.942437 trunc from 0x853736 to 0x8651d6 (0x11aa1 bytes) 000110[1] 1711091657.961964 trunc from 0x8651d7 to 0x473c64 (0x3f1572 bytes) 000112[1] 1711091657.983135 mapread 0x0730a6 thru 0x07485a (0x17b5 bytes) 000113[1] 1711091657.988730 write 0x066ed4 thru 0x067421 (0x54e bytes) 000114[0] 1711091658.000102 trunc from 0x473c65 to 0x569d42 (0xf60de bytes) 000117[1] 1711091658.028298 trunc from 0x569d43 to 0x32ed6a (0x23afd8 bytes) 000118[0] 1711091658.043487 trunc from 0x32ed6b to 0x7f866d (0x4c9903 bytes) 000119[1] 1711091658.058685 mapread 0x14b881 thru 0x14d505 (0x1c85 bytes) 000120[1] 1711091658.073155 write 0x44a813 thru 0x44c4ec (0x1cda bytes) 000122[1] 1711091658.094569 mapwrite 0x2f88de thru 0x2f8e5c (0x57f bytes) 000124[1] 1711091658.139049 write 0x34f0f7 thru 0x34f5dd (0x4e7 bytes) 000125[1] 1711091658.144762 mapwrite 0x94160b thru 0x94228f (0xc85 bytes) 000131[1] 1711091658.213924 read 0x1bd217 thru 0x1bedc4 (0x1bae bytes) 000132[0] 1711091658.266497 mapread 0x0b5925 thru 0x0b6d34 (0x1410 bytes) truncating to largest ever: 0x9eefc1 000135[0] 1711091658.324097 trunc from 0x942290 to 0x9eefc0 (0xacd31 bytes) 000137[0] 1711091658.349926 mapwrite 0x7957fe thru 0x796a96 (0x1299 bytes) 000138[1] 1711091658.382556 write 0x5acadb thru 0x5adec5 (0x13eb bytes) 000144[1] 1711091658.431689 trunc from 0x9eefc1 to 0x0e2d22 (0x90c29e bytes) 000149[0] 1711091658.455699 write 0x3baad5 thru 0x3bc861 (0x1d8d bytes) 000150[0] 1711091658.464482 write 0x24e9ed thru 0x24f135 (0x749 bytes) 000156[1] 1711091658.501868 trunc from 0x3bc862 to 0x118346 (0x2a451b bytes) 000159[0] 1711091658.533817 write 0x3080e0 thru 0x309e99 (0x1dba bytes) 000160[0] 1711091658.545651 mapwrite 0x8953f9 thru 0x89576a (0x372 bytes) 000161[1] 1711091658.596755 trunc from 0x89576b to 0x3fd6b8 (0x4980b2 bytes) 000162[1] 1711091658.610371 mapread 0x321162 thru 0x323108 (0x1fa7 bytes) 000163[1] 1711091658.625971 read 0x0fe18e thru 0x0ff25d (0x10d0 bytes) 000164[0] 1711091658.664510 mapwrite 0x3b0c91 thru 0x3b17a5 (0xb15 bytes) 000165[1] 1711091658.707685 mapwrite 0x580f48 thru 0x581391 (0x44a bytes) 000166[1] 1711091658.753319 mapwrite 0x8d2305 thru 0x8d3d54 (0x1a50 bytes) 000167[0] 1711091658.787432 trunc from 0x8d3d55 to 0x0bf3c9 (0x81498b bytes) 000168[1] 1711091658.801383 read 0x004a09 thru 0x004c09 (0x201 bytes) 000170[1] 1711091658.816841 mapwrite 0x29e7ed thru 0x29ee34 (0x648 bytes) 000171[0] 1711091658.857073 write 0x55798b thru 0x55908d (0x1703 bytes) 000172[1] 1711091658.863451 write 0x1c96f9 thru 0x1c9c0d (0x515 bytes) 000178[0] 1711091658.888663 trunc from 0x55908e to 0x1aba1f (0x3ad66e bytes) 000180[0] 1711091658.904720 mapread 0x14afb1 thru 0x14cbfd (0x1c4d bytes) 000182[0] 1711091658.915508 mapwrite 0x9c19af thru 0x9c1f96 (0x5e8 bytes) 000183[1] 1711091658.942893 read 0x6e33f7 thru 0x6e528f (0x1e99 bytes) 000186[0] 1711091658.960955 write 0x522652 thru 0x522a44 (0x3f3 bytes) 000187[0] 1711091658.973048 read 0x09f28a thru 0x09f430 (0x1a7 bytes) 000189[1] 1711091659.011430 mapread 0x5ca2d9 thru 0x5cb0c5 (0xded bytes) 000191[0] 1711091659.029003 trunc from 0x9c1f97 to 0x57bd46 (0x446250 bytes) 000192[1] 1711091659.044634 mapwrite 0x1afa72 thru 0x1b1a48 (0x1fd7 bytes) 000194[0] 1711091659.134892 trunc from 0x57bd47 to 0x0786f8 (0x50364e bytes) 000196[0] 1711091659.158405 mapwrite 0x663bd1 thru 0x663bd7 (0x7 bytes) 000197[1] 1711091659.201489 mapwrite 0x238583 thru 0x238808 (0x286 bytes) 000204[1] 1711091659.275575 mapwrite 0x3a3702 thru 0x3a47b5 (0x10b4 bytes) 000205[1] 1711091659.312947 read 0x3ee1b7 thru 0x3efab1 (0x18fb bytes) 000207[1] 1711091659.365498 trunc from 0x663bd8 to 0x98ac0c (0x327035 bytes) 000209[1] 1711091659.392926 mapread 0x29cb11 thru 0x29de50 (0x1340 bytes) 000216[1] 1711091659.433440 write 0x57c26b thru 0x57c41f (0x1b5 bytes) 000220[0] 1711091659.480537 read 0x06f762 thru 0x07018b (0xa2a bytes) 000221[0] 1711091659.526491 read 0x475da3 thru 0x476736 (0x994 bytes) 000225[1] 1711091659.571509 write 0x469270 thru 0x46a9e8 (0x1779 bytes) 000226[0] 1711091659.588628 mapwrite 0x7a0fa4 thru 0x7a2418 (0x1475 bytes) 000227[1] 1711091659.619958 mapwrite 0x21deab thru 0x21f39b (0x14f1 bytes) 000228[1] 1711091659.663393 trunc from 0x98ac0d to 0x96214e (0x28abe bytes) 000230[1] 1711091659.681650 read 0x10f379 thru 0x10feb6 (0xb3e bytes) 000235[0] 1711091659.725934 mapread 0x819434 thru 0x81afb9 (0x1b86 bytes) 000237[0] 1711091659.745970 mapread 0x798fa5 thru 0x79967e (0x6da bytes) 000241[1] 1711091659.755955 mapread 0x862727 thru 0x86307b (0x955 bytes) 000242[0] 1711091659.781133 mapwrite 0x3f1b45 thru 0x3f2ee7 (0x13a3 bytes) 000244[0] 1711091659.811546 read 0x69382e thru 0x6957d8 (0x1fab bytes) 000246[0] 1711091659.853938 write 0x07267f thru 0x07424a (0x1bcc bytes) 000249[0] 1711091659.861909 write 0x1ed394 thru 0x1eea1a (0x1687 bytes) 000251[1] 1711091659.865854 trunc from 0x96214f to 0x42223c (0x53ff12 bytes) 000252[1] 1711091659.884676 write 0x5abd02 thru 0x5ac110 (0x40f bytes) 000253[1] 1711091659.890163 write 0x93d06b thru 0x93e0bf (0x1055 bytes) 000254[0] 1711091659.891169 trunc from 0x93e0c0 to 0x5f3a73 (0x34a64c bytes) 000257[1] 1711091659.908364 write 0x41654a thru 0x4170fc (0xbb3 bytes) 000258[0] 1711091659.916396 read 0x0d55fb thru 0x0d66a4 (0x10aa bytes) 000260[1] 1711091659.947891 write 0x463bfa thru 0x46423c (0x643 bytes) 000261[0] 1711091659.954124 mapwrite 0x79e38d thru 0x79fd71 (0x19e5 bytes) 000262[1] 1711091659.992931 write 0x8d0141 thru 0x8d09a2 (0x862 bytes) 000264[0] 1711091660.005528 mapwrite 0x0122c9 thru 0x0122fc (0x34 bytes) 000265[0] 1711091660.025995 trunc from 0x8d09a3 to 0x79c814 (0x13418e bytes) 000267[1] 1711091660.030600 write 0x27bcd8 thru 0x27c262 (0x58b bytes) 000268[0] 1711091660.038701 mapread 0x42295d thru 0x423a67 (0x110b bytes) 000269[0] 1711091660.043776 write 0x972821 thru 0x9732c2 (0xaa2 bytes) 000271[0] 1711091660.050986 trunc from 0x9732c3 to 0x8ec629 (0x86c99 bytes) 000272[0] 1711091660.055396 mapwrite 0x9e078d thru 0x9e0bee (0x462 bytes) 000273[0] 1711091660.076089 mapwrite 0x41f772 thru 0x420244 (0xad3 bytes) 000275[1] 1711091660.089276 read 0x051d70 thru 0x052f97 (0x1228 bytes) 000276[1] 1711091660.109597 trunc from 0x9e0bef to 0x561e29 (0x47edc5 bytes) 000280[1] 1711091660.129828 mapread 0x1d48b6 thru 0x1d681c (0x1f67 bytes) 000282[1] 1711091660.143340 read 0x23fd3d thru 0x241b9d (0x1e61 bytes) 000284[1] 1711091660.164665 mapwrite 0x2cc386 thru 0x2cd6cc (0x1347 bytes) 000289[0] 1711091660.195924 read 0x33f444 thru 0x3412c0 (0x1e7d bytes) 000291[1] 1711091660.226007 mapwrite 0x592b54 thru 0x593161 (0x60e bytes) 000299[1] 1711091660.263243 mapwrite 0x71c156 thru 0x71de6f (0x1d1a bytes) 000301[1] 1711091660.301233 trunc from 0x71de70 to 0x789c65 (0x6bdf6 bytes) 000303[1] 1711091660.314266 mapread 0x36cece thru 0x36ed55 (0x1e88 bytes) 000305[0] 1711091660.320739 mapread 0x42dc27 thru 0x42fa74 (0x1e4e bytes) 000308[0] 1711091660.350023 mapread 0x6d428f thru 0x6d4883 (0x5f5 bytes) 000309[1] 1711091660.354992 mapwrite 0x7871fd thru 0x787362 (0x166 bytes) 000310[0] 1711091660.377267 trunc from 0x789c66 to 0x0458f2 (0x744373 bytes) 000311[1] 1711091660.390477 read 0x038dca thru 0x03a5d3 (0x180a bytes) 000314[0] 1711091660.402975 write 0x6562f3 thru 0x656900 (0x60e bytes) 000315[1] 1711091660.404720 mapread 0x47ea61 thru 0x47f00b (0x5ab bytes) 000316[1] 1711091660.413446 write 0x038483 thru 0x03934c (0xeca bytes) 000319[1] 1711091660.421971 mapwrite 0x9bd4d3 thru 0x9bd985 (0x4b3 bytes) 000320[0] 1711091660.465844 mapwrite 0x0ffdef thru 0x101490 (0x16a2 bytes) 000321[0] 1711091660.494801 mapwrite 0x8968fb thru 0x8987d5 (0x1edb bytes) 000322[1] 1711091660.521745 read 0x822ba8 thru 0x8238cd (0xd26 bytes) 000325[1] 1711091660.592911 read 0x6abe5b thru 0x6ac15f (0x305 bytes) 000327[1] 1711091660.606536 write 0x453f06 thru 0x454f3a (0x1035 bytes) 000328[0] 1711091660.607352 read 0x02b5ed thru 0x02bda1 (0x7b5 bytes) 000330[0] 1711091660.638647 write 0x97a99e thru 0x97b4e3 (0xb46 bytes) 000331[1] 1711091660.657297 write 0x9cfa30 thru 0x9d15cb (0x1b9c bytes) 000332[1] 1711091660.664511 mapwrite 0x27b590 thru 0x27d13c (0x1bad bytes) 000334[1] 1711091660.728836 read 0x16771a thru 0x168b2c (0x1413 bytes) 000335[0] 1711091660.734305 write 0x4b69a5 thru 0x4b6dbf (0x41b bytes) 000336[0] 1711091660.776255 write 0x395700 thru 0x397635 (0x1f36 bytes) 000337[1] 1711091660.805915 write 0x27733e thru 0x277570 (0x233 bytes) 000339[0] 1711091660.855081 mapwrite 0x998067 thru 0x998fce (0xf68 bytes) 000341[0] 1711091660.892056 read 0x7bc8bf thru 0x7be1aa (0x18ec bytes) 000342[1] 1711091660.892400 mapread 0x6d9239 thru 0x6dab78 (0x1940 bytes) 000343[0] 1711091660.935098 mapwrite 0x14b0a8 thru 0x14b6a9 (0x602 bytes) 000345[0] 1711091660.982109 write 0x3e7bb0 thru 0x3e8cc7 (0x1118 bytes) 000350[1] 1711091660.989807 write 0x084785 thru 0x085de9 (0x1665 bytes) 000352[0] 1711091661.012398 mapwrite 0x29e759 thru 0x29f7e9 (0x1091 bytes) 000353[0] 1711091661.053017 mapwrite 0x9174d8 thru 0x917980 (0x4a9 bytes) 000354[0] 1711091661.070117 trunc from 0x9d15cc to 0x3225ad (0x6af01e bytes) 000355[1] 1711091661.088719 mapwrite 0x84c7d5 thru 0x84de0e (0x163a bytes) 000356[1] 1711091661.119729 mapwrite 0x0dda2c thru 0x0de10f (0x6e4 bytes) 000358[1] 1711091661.138821 read 0x5647ca thru 0x565188 (0x9bf bytes) 000361[1] 1711091661.181344 mapwrite 0x450535 thru 0x452431 (0x1efd bytes) 000362[0] 1711091661.196733 trunc from 0x84de0f to 0x618ea3 (0x234f6b bytes) 000363[1] 1711091661.213120 write 0x871ad4 thru 0x8724ed (0xa1a bytes) 000364[0] 1711091661.219500 write 0x61048c thru 0x610800 (0x375 bytes) 000366[1] 1711091661.228825 mapread 0x7e12bf thru 0x7e1702 (0x444 bytes) 000367[1] 1711091661.237711 mapread 0x593ba6 thru 0x5948f9 (0xd54 bytes) 000368[1] 1711091661.240018 write 0x472d8c thru 0x4745a8 (0x181d bytes) 000369[1] 1711091661.247296 write 0x32c23d thru 0x32c2b5 (0x79 bytes) 000371[1] 1711091661.252119 mapwrite 0x888f52 thru 0x889b8e (0xc3d bytes) 000374[0] 1711091661.277015 write 0x5c820e thru 0x5c8659 (0x44c bytes) 000376[1] 1711091661.288306 mapread 0x0ceeec thru 0x0d0171 (0x1286 bytes) 000378[0] 1711091661.300878 read 0x6e0011 thru 0x6e18ba (0x18aa bytes) 000380[1] 1711091661.311243 trunc from 0x889b8f to 0x9e1e0a (0x15827c bytes) 000381[0] 1711091661.322890 mapread 0x2090d7 thru 0x209d9c (0xcc6 bytes) 000384[1] 1711091661.357557 trunc from 0x9e1e0b to 0x096d23 (0x94b0e7 bytes) 000385[0] 1711091661.366688 read 0x022684 thru 0x02351d (0xe9a bytes) 000386[0] 1711091661.376036 mapread 0x08bd68 thru 0x08d8f6 (0x1b8f bytes) 000388[0] 1711091661.378110 mapwrite 0x364576 thru 0x365ee4 (0x196f bytes) 000389[1] 1711091661.418399 write 0x7b58fa thru 0x7b5b3e (0x245 bytes) 000390[1] 1711091661.425115 mapread 0x6e9193 thru 0x6ea718 (0x1586 bytes) 000392[1] 1711091661.435471 mapread 0x37e446 thru 0x37fc7c (0x1837 bytes) 000393[1] 1711091661.446601 mapread 0x0d4a88 thru 0x0d6a05 (0x1f7e bytes) 000394[1] 1711091661.453380 trunc from 0x7b5b3f to 0x949ff0 (0x1944b2 bytes) 000396[1] 1711091661.459659 mapread 0x4f75f3 thru 0x4f89b9 (0x13c7 bytes) 000398[0] 1711091661.464389 read 0x8b1f35 thru 0x8b28f9 (0x9c5 bytes) 000401[0] 1711091661.499825 read 0x4aef8e thru 0x4b05e8 (0x165b bytes) 000403[1] 1711091661.523799 mapwrite 0x0e54b0 thru 0x0e696b (0x14bc bytes) 000404[0] 1711091661.548471 mapwrite 0x7e77a8 thru 0x7e7f6a (0x7c3 bytes) 000405[0] 1711091661.582358 write 0x576346 thru 0x576575 (0x230 bytes) 000415[0] 1711091661.618941 read 0x2a133a thru 0x2a25a0 (0x1267 bytes) 000417[1] 1711091661.654778 mapwrite 0x75a6ae thru 0x75b62a (0xf7d bytes) 000418[1] 1711091661.681278 trunc from 0x949ff1 to 0x9bbd7b (0x71d8b bytes) 000419[0] 1711091661.693076 write 0x712ef6 thru 0x713fed (0x10f8 bytes) 000422[1] 1711091661.707314 trunc from 0x9bbd7c to 0x62c93e (0x38f43d bytes) 000423[0] 1711091661.714053 write 0x2130e7 thru 0x2144bc (0x13d6 bytes) 000426[1] 1711091661.727930 mapwrite 0x349549 thru 0x34a204 (0xcbc bytes) 000427[0] 1711091661.750811 trunc from 0x62c93f to 0x0808cb (0x5ac073 bytes) 000430[1] 1711091661.766885 write 0x734f29 thru 0x7356d5 (0x7ad bytes) 000431[1] 1711091661.774581 write 0x297b40 thru 0x299977 (0x1e38 bytes) 000434[0] 1711091661.780314 trunc from 0x7356d6 to 0x9272ce (0x1f1bf9 bytes) 000438[0] 1711091661.796655 mapread 0x017a73 thru 0x0194ae (0x1a3c bytes) 000442[0] 1711091661.817139 read 0x344a1a thru 0x3459d3 (0xfba bytes) 000444[1] 1711091661.854240 write 0x7e137a thru 0x7e2da6 (0x1a2d bytes) 000445[1] 1711091661.865715 mapread 0x609006 thru 0x609db1 (0xdac bytes) 000446[0] 1711091661.871872 write 0x058e2e thru 0x059422 (0x5f5 bytes) 000450[0] 1711091661.885993 write 0x894ea2 thru 0x89541c (0x57b bytes) 000451[0] 1711091661.889739 mapwrite 0x760312 thru 0x760749 (0x438 bytes) 000452[0] 1711091661.913026 mapwrite 0x140de8 thru 0x14286c (0x1a85 bytes) 000454[0] 1711091661.936969 mapread 0x321873 thru 0x322d92 (0x1520 bytes) 000458[1] 1711091661.945956 mapread 0x12392f thru 0x123c0b (0x2dd bytes) 000461[1] 1711091661.967904 mapread 0x32c591 thru 0x32c784 (0x1f4 bytes) 000462[0] 1711091661.972488 write 0x5f3d51 thru 0x5f59f4 (0x1ca4 bytes) 000465[0] 1711091661.991125 trunc from 0x9272cf to 0x2d471e (0x652bb0 bytes) 000467[0] 1711091662.016390 write 0x1cfbd0 thru 0x1d010b (0x53c bytes) 000468[0] 1711091662.022512 mapread 0x149f4f thru 0x14b6ce (0x1780 bytes) 000471[0] 1711091662.049823 mapread 0x0d9de0 thru 0x0da2fe (0x51f bytes) 000474[0] 1711091662.068682 mapwrite 0x208aac thru 0x20a5aa (0x1aff bytes) 000476[1] 1711091662.102828 mapwrite 0x76aaeb thru 0x76c1b9 (0x16cf bytes) 000477[1] 1711091662.152530 mapread 0x430dd1 thru 0x432580 (0x17b0 bytes) 000478[0] 1711091662.161522 mapread 0x4edfd0 thru 0x4ef473 (0x14a4 bytes) 000480[0] 1711091662.181560 trunc from 0x76c1ba to 0x8ab29a (0x13f0e1 bytes) 000486[1] 1711091662.213038 write 0x2fe2bf thru 0x2ff631 (0x1373 bytes) 000487[0] 1711091662.226033 mapread 0x6f37cc thru 0x6f3ae5 (0x31a bytes) 000489[0] 1711091662.243303 mapread 0x18138f thru 0x181c54 (0x8c6 bytes) 000492[0] 1711091662.266191 trunc from 0x8ab29b to 0x994c13 (0xe9979 bytes) 000495[0] 1711091662.288611 trunc from 0x994c14 to 0x73f053 (0x255bc0 bytes) 000499[1] 1711091662.307226 trunc from 0x73f054 to 0x1fe1f7 (0x540e5c bytes) 000500[0] 1711091662.327698 mapread 0x1ecad5 thru 0x1ed9f5 (0xf21 bytes) All operations completed A-OK! PASS 16b (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 03:14:25 (1711091665) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 03:14:28 (1711091668) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.121166 s, 69.2 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.122547 s, 68.5 MB/s ldlm.namespaces.MGC192.168.202.158@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58aa800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff88012a56e000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b58aa800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff88012a56e000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b58aa800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff88012a56e000.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.88414 s, 36.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.671691 s, 156 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.565964 s, 185 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.25056 s, 83.8 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.94453 s, 53.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.746089 s, 141 MB/s PASS 16d (11s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 03:14:41 (1711091681) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0701695 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0320728 s, 32.7 MB/s PASS 16e (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 03:14:45 (1711091685) TIMEOUT PASS 16f (22s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 03:15:08 (1711091708) TIMEOUT PASS 16g (42s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16h: mmap read after truncate file ======= 03:15:51 (1711091751) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16h (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16i: read after truncate file ============ 03:15:54 (1711091754) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16j: race dio with buffered i/o ========== 03:15:58 (1711091758) 1+0 records in 1+0 records out 26214400 bytes (26 MB) copied, 0.321169 s, 81.6 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.242792 s, 108 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.468991 s, 55.9 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.135415 s, 194 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.480555 s, 54.6 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.172748 s, 152 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.571205 s, 45.9 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.283826 s, 92.4 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.311159 s, 84.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.593629 s, 44.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.631831 s, 41.5 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.196797 s, 133 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.257741 s, 102 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.606664 s, 43.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.645693 s, 40.6 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.202146 s, 130 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.307501 s, 85.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.701663 s, 37.4 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.720636 s, 36.4 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 03:16:07 (1711091767) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x1a7062c skipping zero size read truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x32cce75 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x32cce75 skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x32cce75 truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x3c569db truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 Fri Mar 22 03:16:25 EDT 2024 + fsx 2 done: rc=0 Fri Mar 22 03:16:25 EDT 2024 + fsx 3 done: rc=0 Fri Mar 22 03:16:25 EDT 2024 + fsx 4 done: rc=0 Fri Mar 22 03:16:25 EDT 2024 + fsx 5 done: rc=0 Fri Mar 22 03:16:25 EDT 2024 PASS 16k (20s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 17: resource creation/LVB creation race ========================================================================= 03:16:29 (1711091789) fail_loc=0x8000030a PASS 17 (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 18: mmap sanity check =========================================================================================== 03:16:34 (1711091794) mmap test1: basic mmap operation (PASS, 0.019325s) mmap test2: MAP_PRIVATE not write back (PASS, 0.014807s) mmap test3: concurrent mmap ops on two nodes (PASS, 10.389s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.0817s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.043509s) mmap test6: check mmap write/read content on two nodes (PASS, 0.04263s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0118s) PASS 18 (16s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 19: test concurrent uncached read races ========================================================================= 03:16:52 (1711091812) oleg258-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg258-client: oleg258-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 20: test extra readahead page left in cache ============================================================== 03:16:54 (1711091814) PASS 20 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 03:16:57 (1711091817) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 23: others should see updated atime while another read============================================================== 03:17:00 (1711091820) now is 1711091821 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6749 new atime is 1711091882 PASS 23 (64s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 03:18:06 (1711091886) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3456 2205184 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 11264 3758080 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 43008 7495680 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3456 2205184 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 11264 3758080 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 43008 7495680 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.4K 386 490.1K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.1K 401 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 114.5K 402 114.1K 1% /mnt/lustre[OST:1] filesystem_summary: 229.3K 386 228.9K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.4K 386 490.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.1K 401 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 402 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.3K 386 228.9K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.4M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 11.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 31.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 42.0M 7.1G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502213 386 501827 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117905 401 117504 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117266 402 116864 1% /mnt/lustre2[OST:1] filesystem_summary: 234754 386 234368 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3456 2205184 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 11264 3758080 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 43008 7495680 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.4K 386 490.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.1K 401 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 402 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.3K 386 228.9K 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502213 386 501827 1% /mnt/lustre[MDT:0] filesystem_summary: 502213 386 501827 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502213 386 501827 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117905 401 117504 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117266 402 116864 1% /mnt/lustre2[OST:1] filesystem_summary: 234754 386 234368 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3456 2205184 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 11264 3758080 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 43008 7495680 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3456 2205184 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 11264 3758080 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 43008 7495680 1% /mnt/lustre2 PASS 24a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 24b: lfs df should show both filesystems ========================================================================= 03:18:10 (1711091890) PASS 24b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 03:18:14 (1711091894) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 03:18:18 (1711091898) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 26a: allow mtime to get older ============ 03:18:22 (1711091902) utime: good mknod times 1711091901 <= 1711091902 <= 1711091902 for /mnt/lustre/f26a utime: good mknod times 1711091901 <= 1711091902 <= 1711091902 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 26b: sync mtime between ost and mds ====== 03:18:26 (1711091906) PASS 26b (4s) debug_raw_pointers=0 debug_raw_pointers=0 SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 30: recreate file race =================== 03:18:33 (1711091913) 0 PASS 30 (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 03:18:39 (1711091919) fail_loc=0x314 PASS 31a (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 03:18:44 (1711091924) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg258-server mds-ost sync done. sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.202.158@tcp:/lustre 7542784 43008 7495680 1% /mnt/lustre2 PASS 31b (11s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 31r: open-rename(replace) race =========== 03:18:57 (1711091937) -rw-r--r-- 1 root root 0 Mar 22 03:18 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (5s) debug_raw_pointers=0 debug_raw_pointers=0 SKIP: sanityn test_33a skipping SLOW test 33a debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 03:19:05 (1711091945) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 03:19:08 (1711091948) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 33d: dependent transactions should trigger COS ========================================================== 03:19:12 (1711091952) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 03:19:15 (1711091955) SKIP: sanityn test_33e needs >= 2 MDTs SKIP 33e (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 34: no lock timeout under IO ============= 03:19:18 (1711091958) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg258-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b58aa800.ost_server_uuid,osc.lustre-OST0000-osc-ffff88012a56e000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800b58aa800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff88012a56e000.ost_server_uuid in IDLE state after 0 sec oleg258-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b58aa800.ost_server_uuid,osc.lustre-OST0001-osc-ffff88012a56e000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800b58aa800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff88012a56e000.ost_server_uuid in FULL state after 0 sec oleg258-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b58aa800.ost_server_uuid,osc.lustre-OST0000-osc-ffff88012a56e000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800b58aa800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff88012a56e000.ost_server_uuid in IDLE state after 0 sec oleg258-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b58aa800.ost_server_uuid,osc.lustre-OST0001-osc-ffff88012a56e000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800b58aa800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff88012a56e000.ost_server_uuid in FULL state after 0 sec oleg258-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b58aa800.ost_server_uuid,osc.lustre-OST0000-osc-ffff88012a56e000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800b58aa800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff88012a56e000.ost_server_uuid in IDLE state after 0 sec oleg258-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b58aa800.ost_server_uuid,osc.lustre-OST0001-osc-ffff88012a56e000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800b58aa800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff88012a56e000.ost_server_uuid in FULL state after 0 sec PASS 34 (88s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 03:20:48 (1711092048) fail_loc=0x80000317 Race attempt 0 Wait for 24066 24208 for 60 sec... - open/close 3102 (time 1711092060.23 total 10.00 last 310.18) total: 4000 open/close in 13.02 seconds: 307.15 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.202.158@tcp:/lustre 7.2G 43M 7.2G 1% /mnt/lustre 192.168.202.158@tcp:/lustre 7.2G 43M 7.2G 1% /mnt/lustre2 PASS 35 (64s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 36: handle ESTALE/open-unlink correctly == 03:21:54 (1711092114) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg258-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.290679 s, 180 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6749 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7494656) after_dd(7441408) after(7492608) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.231006 s, 227 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6749 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7492608) after_dd(7439360) after(7490560) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.257716 s, 203 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6749 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7490560) after_dd(7441408) after(7492608) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.277325 s, 189 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6749 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7492608) after_dd(7441408) after(7492608) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.298348 s, 176 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6749 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7492608) after_dd(7439360) after(7490560) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.344715 s, 152 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6749 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7490560) after_dd(7439360) after(7490560) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.376277 s, 139 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6749 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7490560) after_dd(7439360) after(7490560) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.355694 s, 147 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6749 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7490560) after_dd(7441408) after(7492608) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.311389 s, 168 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6749 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7492608) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.413797 s, 127 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6749 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7494656) after_dd(7437312) after(7488512) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.380064 s, 138 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6749 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7488512) after_dd(7443456) after(7494656) PASS 36 (225s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 03:25:41 (1711092341) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6749 - create 5334 (time 1711092352.90 total 10.00 last 533.35) total: 10000 create in 18.51 seconds: 540.16 ops/second PASS 37 (22s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 39a: file mtime does not change after rename ========================================================== 03:26:05 (1711092365) repeat after cancel_lru_locks PASS 39a (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 03:26:09 (1711092369) repeat after cancel_lru_locks PASS 39b (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 39c: check truncate mtime update ================================================================================ 03:26:15 (1711092375) repeat after cancel_lru_locks PASS 39c (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 39d: sync write should update mtime ====== 03:26:20 (1711092380) fail_loc=0x411 fail_loc=0 PASS 39d (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40a: pdirops: create vs others ======================================================================== 03:26:25 (1711092385) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40b: pdirops: open|create and others ======================================================================== 03:26:36 (1711092396) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40c: pdirops: link and others ======================================================================== 03:26:46 (1711092406) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40d: pdirops: unlink and others ======================================================================== 03:26:57 (1711092417) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 40e: pdirops: rename and others ======================================================================== 03:27:07 (1711092427) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 03:27:17 (1711092437) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41b: pdirops: create vs create ======================================================================== 03:27:24 (1711092444) open: File exists Conflict PASS 41b (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41c: pdirops: create vs link ======================================================================== 03:27:30 (1711092450) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41d: pdirops: create vs unlink ======================================================================== 03:27:38 (1711092458) Conflict PASS 41d (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 03:27:45 (1711092465) Conflict PASS 41e (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 03:27:53 (1711092473) Conflict PASS 41f (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41g: pdirops: create vs getattr ======================================================================== 03:28:01 (1711092481) Conflict PASS 41g (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41h: pdirops: create vs readdir ======================================================================== 03:28:08 (1711092488) Conflict PASS 41h (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 41i: reint_open: create vs create ======== 03:28:16 (1711092496) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (694s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 03:39:51 (1711093191) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 03:39:58 (1711093198) open: File exists Conflict PASS 42b (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 03:40:05 (1711093205) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 03:40:11 (1711093211) Conflict PASS 42d (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 03:40:18 (1711093218) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 03:40:24 (1711093224) Conflict PASS 42f (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 03:40:30 (1711093230) Conflict PASS 42g (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 03:40:37 (1711093237) Conflict PASS 42h (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 03:40:44 (1711093244) PASS 43a (16s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43b: pdirops: unlink vs create ======================================================================== 03:41:02 (1711093262) Conflict PASS 43b (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43c: pdirops: unlink vs link ======================================================================== 03:41:09 (1711093269) Conflict PASS 43c (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 03:41:15 (1711093275) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 03:41:21 (1711093281) Conflict PASS 43e (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 03:41:27 (1711093287) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 03:41:33 (1711093293) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 03:41:38 (1711093298) Conflict PASS 43h (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 03:41:45 (1711093305) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 03:41:49 (1711093309) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (36s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 43k: unlink vs create ==================== 03:42:26 (1711093346) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (482s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 03:50:33 (1711093833) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Fri Mar 22 03:50:38 EDT 2024 PASS 44a (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 03:50:44 (1711093844) open: File exists Conflict PASS 44b (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 03:50:57 (1711093857) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 03:51:09 (1711093869) Conflict PASS 44d (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 03:51:21 (1711093881) Conflict PASS 44e (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 03:51:31 (1711093891) Conflict PASS 44f (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 03:51:41 (1711093901) Conflict PASS 44g (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 03:51:51 (1711093911) Conflict PASS 44h (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 03:52:00 (1711093920) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 03:52:04 (1711093924) PASS 45a (76s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45b: pdirops: rename src vs create ======================================================================== 03:53:24 (1711094004) Conflict PASS 45b (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45c: pdirops: rename src vs link ======================================================================== 03:53:36 (1711094016) Conflict PASS 45c (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 03:53:48 (1711094028) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 03:54:00 (1711094040) Conflict PASS 45e (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 03:54:12 (1711094052) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (10s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 03:54:24 (1711094064) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 03:54:38 (1711094078) Conflict PASS 45h (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 03:54:49 (1711094089) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 45j: read vs rename ====================== 03:54:55 (1711094095) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (683s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 04:06:21 (1711094781) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46b: pdirops: link vs create ======================================================================== 04:06:32 (1711094792) open: File exists Conflict PASS 46b (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46c: pdirops: link vs link ======================================================================== 04:06:44 (1711094804) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46d: pdirops: link vs unlink ======================================================================== 04:06:56 (1711094816) Conflict PASS 46d (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 04:07:06 (1711094826) Conflict PASS 46e (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 04:07:17 (1711094837) Conflict PASS 46f (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46g: pdirops: link vs getattr ======================================================================== 04:07:27 (1711094847) Conflict PASS 46g (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46h: pdirops: link vs readdir ======================================================================== 04:07:38 (1711094858) Conflict PASS 46h (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 46i: pdirops: link vs remote mkdir ======= 04:07:48 (1711094868) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 04:07:52 (1711094872) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47b: pdirops: remote mkdir vs create ===== 04:07:56 (1711094876) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47c: pdirops: remote mkdir vs link ======= 04:08:01 (1711094881) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 04:08:06 (1711094886) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 04:08:11 (1711094891) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 04:08:15 (1711094895) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 04:08:20 (1711094900) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 04:08:24 (1711094904) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.123878 s, 82.7 kB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51a: layout lock: refresh layout should work ========================================================== 04:08:36 (1711094916) short read: 0 ->+ 159 -> 0 0 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.0228334 s, 7.0 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3445: kill: (11417) - No such process PASS 51a (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 04:08:44 (1711094924) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00718678 s, 142 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0428302 s, 24.5 MB/s 3145728 PASS 51b (15s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 04:09:02 (1711094942) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00942399 s, 109 kB/s PASS 51c (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 04:09:13 (1711094953) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285918 s, 36.7 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 04:09:21 (1711094961) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2449 0x991 0x280000400 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2449 0x991 0x280000400 read lease(1) has applied. PASS 51e (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 54: rename locking ======================= 04:09:30 (1711094970) llite.lustre-ffff8800b58aa800.xattr_cache=1 llite.lustre-ffff88012a56e000.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505002 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-03-22 04:09:30.000000000 -0400 Modify: 2024-03-22 04:09:30.000000000 -0400 Change: 2024-03-22 04:09:30.000000000 -0400 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505005 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-03-22 04:09:36.000000000 -0400 Modify: 2024-03-22 04:09:37.000000000 -0400 Change: 2024-03-22 04:09:37.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505008 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-03-22 04:09:42.000000000 -0400 Modify: 2024-03-22 04:09:42.000000000 -0400 Change: 2024-03-22 04:09:42.000000000 -0400 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505007 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-03-22 04:09:42.000000000 -0400 Modify: 2024-03-22 04:09:42.000000000 -0400 Change: 2024-03-22 04:09:42.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505010 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-03-22 04:09:47.000000000 -0400 Modify: 2024-03-22 04:09:48.000000000 -0400 Change: 2024-03-22 04:09:48.000000000 -0400 Birth: - File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505011 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-03-22 04:09:47.000000000 -0400 Modify: 2024-03-22 04:09:48.000000000 -0400 Change: 2024-03-22 04:09:48.000000000 -0400 Birth: - PASS 54 (25s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 55a: rename vs unlink target dir ========= 04:09:58 (1711094998) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 55b: rename vs unlink source dir ========= 04:10:10 (1711095010) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 55c: rename vs unlink orphan target dir == 04:10:21 (1711095021) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (14s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 55d: rename file vs link ================= 04:10:38 (1711095038) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (11s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 56a: test llverdev with single large stripe ========================================================== 04:10:52 (1711095052) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1711095053 chunksize: 4096 size: 16777216000000 write offset: 3783262208kB 0.469922 MB/s read offset: 3783262208kB inf MB/s llverdev: data verified successfully PASS 56a (33s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 04:11:29 (1711095089) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1711095093 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 2.14165 MB/s read offset: 32768000000kB -nan MB/s llverdev: data verified successfully PASS 56b (66s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 60: Verify data_version behaviour ======== 04:12:37 (1711095157) PASS 60 (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 70a: cd directory && rm directory ======== 04:12:44 (1711095164) PASS 70a (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 70b: remove files after calling rm_entry ========================================================== 04:12:51 (1711095171) PASS 70b (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 71a: correct file map just after write operation is finished ========================================================== 04:12:57 (1711095177) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00662009 s, 9.9 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 04:13:01 (1711095181) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.0059735 s, 11.0 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 04:13:05 (1711095185) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 04:13:09 (1711095189) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.0161253 s, 4.1 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 04:13:13 (1711095193) llite.lustre-ffff8800b58aa800.xattr_cache=1 llite.lustre-ffff88012a56e000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 04:13:19 (1711095199) llite.lustre-ffff8800b58aa800.xattr_cache=1 llite.lustre-ffff88012a56e000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 74: flock deadlock: different mounts ======================================================================== 04:13:24 (1711095204) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00479591 s, 214 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00655388 s, 156 kB/s 26125: get lock2 26126: get lock1 26125: done 26125: get lock1 26126: done 26126: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 26126: cannot get lock2: Resource deadlock avoided 26126: exit rc=0 26125: done 26125: exit rc=0 PASS 74 (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 75: osc: upcall after unuse lock============================================================================= 04:13:34 (1711095214) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0242789 s, 86.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170134 s, 61.6 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4031: 26872 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (8s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 76: Verify MDT open_files listing ======== 04:13:45 (1711095225) opening files: ........ closing files: ........ PASS 76 (43s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77a: check FIFO NRS policy =============== 04:14:30 (1711095270) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.190502 s, 88.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240314 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332626 s, 31.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378062 s, 27.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0258854 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111312 s, 94.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00889019 s, 118 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495808 s, 21.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0400293 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0545251 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0894923 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.133471 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.125576 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.112367 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146864 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148542 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.133343 s, 7.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0418167 s, 25.1 MB/s , 0.0385186 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0419968 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206494 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237478 s, 44.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308109 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.024467 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268793 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275241 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293864 s, 35.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0330798 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328888 s, 31.9 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0371158 s, 28.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0342879 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388046 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361963 s, 29.0 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg258-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77b: check CRR-N NRS policy ============== 04:14:38 (1711095278) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.157434 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013181 s, 79.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142362 s, 73.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131035 s, 80.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277316 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0831109 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0794855 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.083364 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0767305 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0936423 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0964335 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0816793 s, 12.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.10705 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106069 s, 9.9 MB/s 1048576 bytes (1.0 MB) copied, 0.112814 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.109974 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.123323 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037249 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.001725 s, 608 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0018043 s, 581 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0491939 s, 21.3 MB/s , 0.028939 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.038324 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0302284 s, 34.7 MB/s , 0.030556 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167437 s, 62.6 MB/s , 0.0291984 s, 35.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321882 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.031425 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025823 s, 40.6 MB/s , 0.0319382 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162685 s, 64.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345694 s, 30.3 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg258-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.13714 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229275 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293524 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0606895 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0693461 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0766831 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0920244 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0887368 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0883451 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0919855 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0881045 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0930995 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0956112 s, 11.0 MB/s , 0.0928838 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0921248 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0951979 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.107584 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00174645 s, 600 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217302 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256002 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0374164 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259713 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00153523 s, 683 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317165 s, 33.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0273558 s, 38.3 MB/s , 0.0199649 s, 52.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240069 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00271589 s, 386 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0305622 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00246975 s, 425 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258938 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00629434 s, 167 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.042638 s, 24.6 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg258-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77c: check ORR NRS policy ================ 04:14:46 (1711095286) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.153927 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135901 s, 77.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00914974 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181641 s, 57.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248688 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0436158 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0528677 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0685894 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0591052 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0779719 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0801258 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0655615 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0929502 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0699648 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0877748 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0826537 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106761 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207259 s, 50.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235388 s, 44.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0242157 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267282 s, 39.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328377 s, 31.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0330043 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333506 s, 31.4 MB/s , 0.0341136 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0455068 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.036467 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395608 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0403754 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0396762 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0380461 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389565 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423985 s, 24.7 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg258-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.137376 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163312 s, 64.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185436 s, 56.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0627983 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0609313 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0645533 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0773184 s, 13.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0751385 s, 14.0 MB/s , 0.075928 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0867219 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0744989 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.075713 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0882691 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0908472 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0816117 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0856192 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.101169 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00412227 s, 254 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185373 s, 56.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198993 s, 52.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231009 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181907 s, 57.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00165686 s, 633 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179036 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027662 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279341 s, 37.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00209652 s, 500 MB/s , 0.00218982 s, 479 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0016687 s, 628 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413774 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353261 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352464 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00290057 s, 362 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg258-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.181094 s, 92.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121634 s, 86.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025101 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148262 s, 70.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288217 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0311221 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0874769 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.100666 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0803218 s, 13.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.104726 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.111061 s, 9.4 MB/s , 0.0883249 s, 11.9 MB/s 1048576 bytes (1.0 MB) copied, 0.110512 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105109 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121491 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.133519 s, 7.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.034733 s, 30.2 MB/s , 0.037394 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019132 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218368 s, 48.0 MB/s , 0.0210587 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0244275 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245695 s, 42.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260215 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0252717 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0238579 s, 44.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0297032 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0329895 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275813 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292686 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308718 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371509 s, 28.2 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg258-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77d: check TRR nrs policy ================ 04:14:55 (1711095295) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.133793 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173937 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179249 s, 58.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0473884 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0512933 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.053854 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0518017 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.061889 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0555128 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.074526 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0676519 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0738436 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.075577 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0836092 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0845978 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0823454 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0891094 s, 11.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184125 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176915 s, 59.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166025 s, 63.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0174718 s, 60.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188314 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165327 s, 63.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166886 s, 62.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172239 s, 60.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220572 s, 47.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0243424 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168742 s, 62.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175376 s, 59.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190777 s, 55.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0189661 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259276 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222188 s, 47.2 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg258-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.145985 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324406 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304558 s, 34.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0826857 s, 12.7 MB/s , 0.0889574 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0892395 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0893974 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0926272 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.10196 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.101258 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.102518 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.101704 s, 10.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106934 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.111735 s, 9.4 MB/s 1048576 bytes (1.0 MB) copied, 0.110156 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.109255 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.116459 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0195641 s, 53.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0174894 s, 60.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00167236 s, 627 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00195561 s, 536 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222925 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00187122 s, 560 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00224144 s, 468 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132496 s, 79.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338313 s, 31.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297236 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228251 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0121462 s, 86.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0331626 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165823 s, 63.2 MB/s 1048576 bytes (1.0 MB) copied, 0.00503439 s, 208 MB/s 1048576 bytes (1.0 MB) copied, 0.0134718 s, 77.8 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg258-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.153074 s, 110 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183783 s, 57.1 MB/s 1048576 bytes (1.0 MB) copied, 0.020479 s, 51.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0609567 s, 17.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0736838 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0819297 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0853691 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0719548 s, 14.6 MB/s , 0.0790344 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0842062 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0778437 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0863011 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0926189 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0933024 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0870297 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0779286 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.111006 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304768 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242712 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243209 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289501 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0313776 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342181 s, 30.6 MB/s , 0.0341408 s, 30.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0381457 s, 27.5 MB/s , 0.0381456 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392469 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333667 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300328 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0350023 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0404579 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335141 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0500073 s, 21.0 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg258-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (6s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77e: check TBF NID nrs policy ============ 04:15:04 (1711095304) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.202.58@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.151948 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182152 s, 57.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188938 s, 55.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0759769 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0824324 s, 12.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0979631 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0988319 s, 10.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0975325 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.097753 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0975589 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.103898 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.100043 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.101629 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.102358 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106655 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.108025 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.110014 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00435546 s, 241 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245688 s, 42.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00452352 s, 232 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030687 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00670683 s, 156 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0336913 s, 31.1 MB/s , 0.0204886 s, 51.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0302596 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00817145 s, 128 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307682 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164979 s, 63.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0325443 s, 32.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0216446 s, 48.4 MB/s , 0.0318829 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338111 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034635 s, 30.3 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg258-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.131798 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016054 s, 65.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198629 s, 52.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0542502 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0606411 s, 17.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.061594 s, 17.0 MB/s , 0.0672656 s, 15.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0621707 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0674198 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0673883 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0789296 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0963314 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.100234 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0973232 s, 10.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.104432 s, 10.0 MB/s , 0.102652 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.101248 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194457 s, 53.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159042 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166444 s, 63.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189237 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236133 s, 44.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0232412 s, 45.1 MB/s , 0.0237951 s, 44.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.022168 s, 47.3 MB/s , 0.024412 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254875 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217676 s, 48.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0237746 s, 44.1 MB/s , 0.0239517 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242324 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255931 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267049 s, 39.3 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg258-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.142928 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0302485 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232401 s, 45.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.083316 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0853794 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0885411 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0969992 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.100415 s, 10.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.107572 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.116731 s, 9.0 MB/s 1048576 bytes (1.0 MB) copied, 0.117884 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.114863 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.118425 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.117538 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.128093 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.130458 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.139804 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00303216 s, 346 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0257444 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116102 s, 90.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017887 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00564091 s, 186 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143644 s, 73.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0127855 s, 82.0 MB/s , 0.0125482 s, 83.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105554 s, 99.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167806 s, 62.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192293 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208536 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0217996 s, 48.1 MB/s , 0.0219803 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237183 s, 44.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247317 s, 42.4 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg258-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.190892 s, 87.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189781 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0419496 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0424935 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0426908 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0663322 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.107954 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106565 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.107579 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.115916 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0979725 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.124122 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121549 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14547 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.140477 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142419 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.140942 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00206764 s, 507 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0308182 s, 34.0 MB/s , 0.0448158 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0409004 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0370269 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0510746 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00166907 s, 628 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271622 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232748 s, 45.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357506 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238935 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209556 s, 50.0 MB/s , 0.0168842 s, 62.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.013912 s, 75.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0104265 s, 101 MB/s , 0.0085518 s, 123 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg258-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (10s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77f: check TBF JobID nrs policy ========== 04:15:17 (1711095317) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg258-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.27397 s, 61.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03332 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0303089 s, 34.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304877 s, 34.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0141509 s, 74.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150012 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.124663 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.136207 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158368 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157148 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.176746 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.17254 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152388 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148558 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.175541 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.164984 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166424 s, 6.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0268962 s, 39.0 MB/s , 0.0171563 s, 61.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297691 s, 35.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021093 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0216496 s, 48.4 MB/s , 0.0220992 s, 47.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241922 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175626 s, 59.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270702 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0223557 s, 46.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0224971 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234651 s, 44.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260205 s, 40.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0271114 s, 38.7 MB/s , 0.0276504 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306511 s, 34.2 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg258-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg258-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.189843 s, 88.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193222 s, 54.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208444 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357213 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0926545 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105012 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0963523 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.102068 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106199 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.102508 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.125036 s, 8.4 MB/s , 0.112206 s, 9.3 MB/s 1048576 bytes (1.0 MB) copied, 0.108429 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1296 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.126885 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.128576 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.138739 s, 7.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0396724 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111622 s, 93.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186727 s, 56.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261761 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390676 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00244066 s, 430 MB/s 1048576 bytes (1.0 MB) copied, 0.0515902 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00203763 s, 515 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020135 s, 52.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0459582 s, 22.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0692962 s, 15.1 MB/s 1048576 bytes (1.0 MB) copied, 0.03426 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01543 s, 68.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0243523 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146544 s, 71.6 MB/s 1048576 bytes (1.0 MB) copied, 0.00985222 s, 106 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg258-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg258-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.157551 s, 106 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133154 s, 78.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161188 s, 65.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0299555 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0404327 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0849643 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0987721 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0923381 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.103914 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.128709 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.141191 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.138381 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.12671 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14316 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.122567 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.138166 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.17312 s, 6.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00472599 s, 222 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332595 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354626 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00900776 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356454 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146514 s, 71.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972023 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0401883 s, 26.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.045465 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275823 s, 38.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0344017 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308638 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338693 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345938 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.036421 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0349023 s, 30.0 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg258-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg258-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.188284 s, 89.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213553 s, 49.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0358557 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0757344 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0727492 s, 14.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0762675 s, 13.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0956886 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0754364 s, 13.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0890736 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0860952 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0896818 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.108611 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.138077 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.137749 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14133 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.138302 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.171569 s, 6.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144271 s, 72.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00320201 s, 327 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00191782 s, 547 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00172034 s, 610 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0021123 s, 496 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00229975 s, 456 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00187338 s, 560 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00342735 s, 306 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00175056 s, 599 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00256365 s, 409 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253561 s, 41.4 MB/s , 0.0337057 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0438516 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322534 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00738954 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00215824 s, 486 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg258-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (12s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77g: Change TBF type directly ============ 04:15:32 (1711095332) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 04:15:38 (1711095338) oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg258-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg258-client: oleg258-server: ssh exited with exit code 2 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 oleg258-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Unknown error 524 pdsh@oleg258-client: oleg258-server: ssh exited with exit code 12 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 oleg258-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Unknown error 524 pdsh@oleg258-client: oleg258-server: ssh exited with exit code 12 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 oleg258-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Unknown error 524 pdsh@oleg258-client: oleg258-server: ssh exited with exit code 12 PASS 77h (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77i: Change rank of TBF rule ============= 04:15:46 (1711095346) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77j: check TBF-OPCode NRS policy ========= 04:15:58 (1711095358) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.307398 s, 54.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00885052 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202686 s, 51.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258012 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0648783 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0583198 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0991146 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0994074 s, 10.5 MB/s , 0.0974952 s, 10.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145314 s, 7.2 MB/s 1048576 bytes (1.0 MB) copied, 0.150248 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152772 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149668 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.26422 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.254275 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.304109 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.309657 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173907 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00306363 s, 342 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123515 s, 84.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192417 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199782 s, 52.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00703058 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141067 s, 74.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129101 s, 81.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014358 s, 73.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023389 s, 44.8 MB/s , 0.0172513 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155829 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016212 s, 64.7 MB/s , 0.018442 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173745 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016555 s, 63.3 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg258-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.57441 s, 18.8 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4691 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (51s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 04:16:52 (1711095412) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg258-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg258-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg258-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg258-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg258-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.25209 s, 13.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108017 s, 97.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0204427 s, 51.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0530593 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.058334 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322385 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0536242 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0473222 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0820359 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.122584 s, 8.6 MB/s , 0.124237 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146455 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.139095 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142726 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142519 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.175935 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.187471 s, 5.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238001 s, 44.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202072 s, 51.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213177 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261815 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269774 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328927 s, 31.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0362361 s, 28.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0350011 s, 30.0 MB/s , 0.0326856 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0374949 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388807 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0351184 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0402224 s, 26.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0405992 s, 25.8 MB/s , 0.0414016 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0469893 s, 22.3 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg258-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.7956 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9262 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg258-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg258-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg258-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg258-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg258-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.24493 s, 13.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208656 s, 50.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0272489 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0554335 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0561356 s, 18.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0681095 s, 15.4 MB/s , 0.0642391 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0720808 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0908406 s, 11.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121269 s, 8.6 MB/s 1048576 bytes (1.0 MB) copied, 0.124018 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158424 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154834 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.16095 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15939 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.17628 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.180059 s, 5.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213583 s, 49.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187333 s, 56.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188932 s, 55.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243281 s, 43.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0193417 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195733 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199739 s, 52.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026556 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283851 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256265 s, 40.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0294399 s, 35.6 MB/s , 0.0297083 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0257805 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273187 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293364 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352849 s, 29.7 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg258-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.7809 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9626 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (134s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 04:19:08 (1711095548) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg258-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg258-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg258-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg258-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg258-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.23411 s, 13.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342032 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.035991 s, 29.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0638005 s, 16.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0620958 s, 16.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0660441 s, 15.9 MB/s , 0.0677288 s, 15.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0805366 s, 13.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0762699 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149793 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15153 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.174413 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.174893 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.175517 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.170002 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.186991 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.205878 s, 5.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199223 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201701 s, 52.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200395 s, 52.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210102 s, 49.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211626 s, 49.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0215248 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225119 s, 46.6 MB/s , 0.0225052 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238701 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212292 s, 49.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0227029 s, 46.2 MB/s , 0.0234522 s, 44.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222452 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218349 s, 48.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246515 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342859 s, 30.6 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg258-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.7308 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9728 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg258-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg258-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg258-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg258-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg258-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.24756 s, 13.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189308 s, 55.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0259136 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.054749 s, 19.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0533297 s, 19.7 MB/s , 0.0541947 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0517675 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0662102 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.076467 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.140868 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142827 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166135 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169981 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.180687 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.177557 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.187853 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.193762 s, 5.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194511 s, 53.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197149 s, 53.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206518 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212203 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240206 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023661 s, 44.3 MB/s , 0.0236724 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258918 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227583 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260408 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256873 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256263 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0302819 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0330286 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0336251 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390234 s, 26.9 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg258-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.795 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9226 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (133s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 04:21:24 (1711095684) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg258-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.265239 s, 63.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134597 s, 77.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226982 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0468053 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0405365 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451723 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0565428 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0496206 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0800015 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0831105 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0645816 s, 16.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0832392 s, 12.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0729482 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.101125 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0988082 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.100831 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.108431 s, 9.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00184588 s, 568 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00163981 s, 639 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00166412 s, 630 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00235684 s, 445 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00164665 s, 637 MB/s 1048576 bytes (1.0 MB) copied, 0.00170109 s, 616 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153596 s, 68.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0188442 s, 55.6 MB/s 1048576 bytes (1.0 MB) copied, 0.028596 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0323828 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0314291 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00244697 s, 429 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153795 s, 68.2 MB/s , 0.0109103 s, 96.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00638904 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00503684 s, 208 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg258-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.0464 s, 20.8 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.76615 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.58@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.58@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.209331 s, 80.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206298 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332312 s, 31.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0758843 s, 13.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0794923 s, 13.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0707835 s, 14.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0764629 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0988112 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106698 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0993686 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0849557 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0941215 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0868208 s, 12.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.103086 s, 10.2 MB/s , 0.124439 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.161226 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178233 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212176 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00330331 s, 317 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0483524 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0429651 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361648 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0366124 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308345 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137283 s, 76.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0369266 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284205 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0458951 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449214 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277008 s, 37.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0481967 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231471 s, 45.3 MB/s , 0.0246423 s, 42.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg258-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.24684 s, 20.0 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.78089 s, 10.7 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.58@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg258-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.216895 s, 77.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145173 s, 72.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173661 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0525109 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0561651 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0568195 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0639805 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0660671 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0628798 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0686925 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0703893 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.072213 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0757726 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0807673 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0813987 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.094777 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0961266 s, 10.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227793 s, 46.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177582 s, 59.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181983 s, 57.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018733 s, 56.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200764 s, 52.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0153349 s, 68.4 MB/s , 0.0159536 s, 65.7 MB/s , 0.0193202 s, 54.3 MB/s , 0.0155826 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247301 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167349 s, 62.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250441 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173918 s, 60.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0198012 s, 53.0 MB/s , 0.0196876 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219078 s, 47.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg258-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.55374 s, 18.9 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94474 s, 21.2 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg258-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.179686 s, 93.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159642 s, 65.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181141 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451322 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0416477 s, 25.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0380867 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0739753 s, 14.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0622407 s, 16.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0754753 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0784706 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0782328 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0630558 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0615291 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0872239 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0893535 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0872861 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0943835 s, 11.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00194892 s, 538 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277369 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0338661 s, 31.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0211016 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372404 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0467567 s, 22.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0258731 s, 40.5 MB/s , 0.0370003 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00167253 s, 627 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0471369 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0461245 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0477528 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0462134 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150096 s, 69.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103298 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00756905 s, 139 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg258-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.93767 s, 10.6 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.76797 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.35697 s, 19.6 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.78634 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg258-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.194726 s, 86.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140576 s, 74.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0402493 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447813 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0456139 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0492999 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0554493 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0595476 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0607115 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0596446 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0867741 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0878114 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0910624 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.132445 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.127117 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.136826 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146365 s, 7.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0169627 s, 61.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00235145 s, 446 MB/s , 0.0032912 s, 319 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120037 s, 87.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00481705 s, 218 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215912 s, 48.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00838095 s, 125 MB/s , 0.010253 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00647071 s, 162 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0123717 s, 84.8 MB/s , 0.0103647 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010562 s, 99.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249489 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117518 s, 89.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118486 s, 88.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129209 s, 81.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg258-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.7564 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4857 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg258-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.286475 s, 58.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190742 s, 55.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026928 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0555061 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0633084 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.06762 s, 15.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.08746 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0845389 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0914121 s, 11.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0812675 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0859851 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0896229 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0925876 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0978031 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0921826 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0977463 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.108461 s, 9.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141165 s, 74.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013456 s, 77.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00146724 s, 715 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192119 s, 54.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00395964 s, 265 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157546 s, 66.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120459 s, 87.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0159719 s, 65.7 MB/s 1048576 bytes (1.0 MB) copied, 0.00728961 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129431 s, 81.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0150653 s, 69.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0152536 s, 68.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138837 s, 75.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.014391 s, 72.9 MB/s , 0.014616 s, 71.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147818 s, 70.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg258-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.4966 s, 19.1 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.78732 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg258-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.199293 s, 84.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145921 s, 71.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192051 s, 54.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0370241 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449352 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0432525 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0433893 s, 24.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0408148 s, 25.7 MB/s , 0.0516634 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0443276 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0505028 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0551926 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0629084 s, 16.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.11021 s, 9.5 MB/s , 0.106629 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.110606 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.111403 s, 9.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0334571 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00225378 s, 465 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00153376 s, 684 MB/s , 0.0156517 s, 67.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00357267 s, 293 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00538828 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00182315 s, 575 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446846 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0016481 s, 636 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0325768 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316342 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0173899 s, 60.3 MB/s , 0.0237247 s, 44.2 MB/s , 0.0179575 s, 58.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0170307 s, 61.6 MB/s , 0.0112564 s, 93.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg258-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.94351 s, 10.5 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.76746 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.88377 s, 10.6 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.79077 s, 10.7 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (349s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 04:27:16 (1711096036) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 04:27:21 (1711096041) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg258-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.57075 s, 294 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg258-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0326 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (51s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 04:28:14 (1711096094) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.116013 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115002 s, 91.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014941 s, 70.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0255332 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026824 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0552006 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0690359 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0710799 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.046572 s, 22.5 MB/s , 0.0522352 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0477633 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0550046 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0811903 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.099533 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.104795 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.102494 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0900111 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021003 s, 49.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0180627 s, 58.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0159459 s, 65.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162052 s, 64.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182598 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196556 s, 53.3 MB/s , 0.0201536 s, 52.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0202955 s, 51.7 MB/s 1+0 records in 1+0 records out , 0.0200908 s, 52.2 MB/s 1048576 bytes (1.0 MB) copied, 0.020504 s, 51.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019362 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249894 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217527 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225014 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225064 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265547 s, 39.5 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg258-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.74635 s, 18.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92308 s, 21.3 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.30269 s, 55.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.016892 s, 62.1 MB/s , 0.0217837 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0544553 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0489232 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0576668 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0719289 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0742745 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0696534 s, 15.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0697367 s, 15.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0826006 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0779368 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.078823 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.23201 s, 4.5 MB/s , 0.234458 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.286655 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.283487 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200094 s, 52.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00329978 s, 318 MB/s , 0.016969 s, 61.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00478184 s, 219 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00197846 s, 530 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141506 s, 74.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163671 s, 64.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163348 s, 64.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00198966 s, 527 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124905 s, 83.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00154664 s, 678 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133977 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141554 s, 74.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00182748 s, 574 MB/s , 0.00182208 s, 575 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0021265 s, 493 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg258-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg258-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.17986 s, 20.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.98033 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (67s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77o: Changing rank should not panic ====== 04:29:24 (1711096164) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 04:29:31 (1711096171) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (40s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 04:30:14 (1711096214) mds.MDS.mdt.nrs_policies=tbf oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg258-client: oleg258-server: ssh exited with exit code 36 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg258-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg258-client: oleg258-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (15s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 77r: Change type of tbf policy at run time ========================================================== 04:30:32 (1711096232) mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 25411 kill md_thread xargs PASS 77r (40s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 78: Enable policy and specify tunings right away ========================================================== 04:31:14 (1711096274) oleg258-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device oleg258-server: error: set_param: setting 'ost/OSS/*/nrs_orr_quantum'='1': No such device pdsh@oleg258-client: oleg258-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (4s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 79: xattr: intent error ================== 04:31:21 (1711096281) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272506071 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-03-22 04:31:22.000000000 -0400 Modify: 2024-03-22 04:31:22.000000000 -0400 Change: 2024-03-22 04:31:22.000000000 -0400 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (14s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 80a: migrate directory when some children is being opened ========================================================== 04:31:37 (1711096297) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 80b: Accessing directory during migration ========================================================== 04:31:41 (1711096301) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 81a: rename and stat under striped directory ========================================================== 04:31:44 (1711096304) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 04:31:48 (1711096308) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 04:31:51 (1711096311) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 04:31:55 (1711096315) - open/close 1547 (time 1711096327.90 total 10.00 last 154.67) total: 2500 open/close in 16.86 seconds: 148.25 ops/second snapshot_time 1711096365.037086586 secs.nsecs start_time 1711096317.633595701 secs.nsecs elapsed_time 47.403490885 secs.nsecs open 2505 samples [usecs] 108 6287 2036623 2178303409 close 2504 samples [usecs] 94 3192 606538 235774108 mknod 2501 samples [usecs] 332 5263 1733116 1592323418 unlink 2500 samples [usecs] 382 33157 2527580 9854676902 mkdir 2 samples [usecs] 614 1657 2271 3122645 rmdir 2 samples [usecs] 3191 3678 6869 23710165 rename 2500 samples [usecs] 394 19579 6821903 31491461545 getattr 3764 samples [usecs] 7 2703 100441 31777811 parallel_rename_file 2500 samples [usecs] 394 19579 6821903 31491461545 crossdir_rename 2500 samples [usecs] 394 19579 6821903 31491461545 PASS 81d (50s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 04:32:47 (1711096367) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6749 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6749 PASS 82 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 04:32:51 (1711096371) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 84: 0-nlink race in lu_object_find() ===== 04:32:55 (1711096375) fail_loc=0x8000060b PASS 84 (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 85: Lustre API root cache race =========== 04:33:05 (1711096385) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1711096385 Finishing test test1 at 1711096387 PASS 85 (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 90: open/create and unlink striped directory ========================================================== 04:33:11 (1711096391) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 91: chmod and unlink striped directory === 04:33:14 (1711096394) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 92: create remote directory under orphan directory ========================================================== 04:33:19 (1711096399) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 04:33:23 (1711096403) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5438 0x153e 0x240000400 1 5424 0x1530 0x280000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5439 0x153f 0x240000400 1 5425 0x1531 0x280000400 rc1=0 and rc2=0 PASS 93 (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 94: signal vs CP callback race =========== 04:33:34 (1711096414) osc.lustre-OST0000-osc-ffff8800b58aa800.idle_timeout=0 osc.lustre-OST0000-osc-ffff88012a56e000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b58aa800.idle_timeout=0 osc.lustre-OST0001-osc-ffff88012a56e000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.13866 s, 59.1 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 3.00967 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5667: 11918 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0486527 s, 84.2 kB/s osc.lustre-OST0000-osc-ffff8800b58aa800.idle_timeout=debug osc.lustre-OST0000-osc-ffff88012a56e000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b58aa800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012a56e000.idle_timeout=debug PASS 94 (11s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 04:33:48 (1711096428) llite.lustre-ffff8800b58aa800.fast_read=0 llite.lustre-ffff88012a56e000.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0277942 s, 75.5 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff8800b58aa800.fast_read=1 llite.lustre-ffff88012a56e000.fast_read=1 PASS 95a (20s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 04:34:10 (1711096450) llite.lustre-ffff8800b58aa800.fast_read=0 llite.lustre-ffff88012a56e000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00764319 s, 1.6 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800b58aa800.fast_read=1 llite.lustre-ffff88012a56e000.fast_read=1 PASS 95b (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 04:34:22 (1711096462) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 04:34:26 (1711096466) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00222302 s, 1.8 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 04:34:31 (1711096471) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.045844 s, 45.7 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 04:34:37 (1711096477) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0347526 s, 60.3 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 100e: DoM: read on open and file size ==== 04:34:42 (1711096482) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00343369 s, 3.5 MB/s PASS 100e (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 101a: Discard DoM data on unlink ========= 04:34:48 (1711096488) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000816323 s, 5.0 MB/s PASS 101a (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 101b: Discard DoM data on rename ========= 04:34:54 (1711096494) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000931258 s, 4.4 MB/s PASS 101b (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 101c: Discard DoM data on close-unlink === 04:35:00 (1711096500) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000831307 s, 4.9 MB/s PASS 101c (4s) debug_raw_pointers=0 debug_raw_pointers=0 SKIP: sanityn test_102 skipping ALWAYS excluded test 102 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 103: Test size correctness with lockahead ========================================================== 04:35:07 (1711096507) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1711096509 lockahead_test: lockahead_test.c:1191: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1711096511 Finishing test test23 at 1711096512 PASS 103 (9s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 04:35:19 (1711096519) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 105: Glimpse and lock cancel race ======== 04:35:24 (1711096524) fail_loc=0x416 PASS 105 (23s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 106a: Verify the btime via statx() ======= 04:35:50 (1711096550) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 106b: Glimpse RPCs test for statx ======== 04:35:54 (1711096554) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 106c: Verify statx attributes mask ======= 04:35:58 (1711096558) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 107a: Basic grouplock conflict =========== 04:36:02 (1711096562) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.11014 s, 95.2 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 5430 0x1536 0x280000400 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6749 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6749 First grouplock blocks second one PASS 107a (5s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 04:36:09 (1711096569) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.120999 s, 86.7 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5447 0x1547 0x240000400 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6749 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (10s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 108a: lseek: parallel updates ============ 04:36:22 (1711096582) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00267362 s, 3.7 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.34732 s, 1.9 MB/s PASS 108a (7s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 109: Race with several mount instances on 1 node ========================================================== 04:36:32 (1711096592) 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 2 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 4 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 10 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 11 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 14 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 18 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 20 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 22 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 24 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 25 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 26 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 27 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 28 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 30 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 31 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 32 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 36 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_refcnt: could not read integer from '/sys/module/acpi_cpufreq/refcnt': 'No such device' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 38 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 40 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 41 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 42 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 43 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 44 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 45 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 46 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg258-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 47 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 192.168.202.158@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.202.158@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg258-client.virtnet /mnt/lustre (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg258-server' oleg258-server: oleg258-server.virtnet: executing load_modules_local oleg258-server: Loading modules from /home/green/git/lustre-release/lustre oleg258-server: detected 4 online CPUs by sysfs oleg258-server: Force libcfs to create 2 CPU partitions Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre Starting client: oleg258-client.virtnet: -o user_xattr,flock oleg258-server@tcp:/lustre /mnt/lustre2 PASS 109 (615s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 110: do not grant another lock on resend ========================================================== 04:46:50 (1711097210) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-03-22 04:46:51.000000000 -0400 Modify: 2024-03-22 04:46:51.000000000 -0400 Change: 2024-03-22 04:46:51.000000000 -0400 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg258-client.virtnet oleg258-client.virtnet PASS 110 (66s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 04:47:59 (1711097279) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 112: update max-inherit in default LMV === 04:48:03 (1711097283) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 113: check servers of specified fs ======= 04:48:06 (1711097286) lustre-OST0000-osc-ffff8800b610c000 active. lustre-OST0001-osc-ffff8800b610c000 active. lustre-OST0000-osc-ffff8800b610c000 active. lustre-OST0001-osc-ffff8800b610c000 active. MGC192.168.202.158@tcp active. PASS 113 (3s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 114: implicit default LMV inherit ======== 04:48:12 (1711097292) SKIP: sanityn test_114 We need at least 2 MDTs for this test SKIP 114 (2s) debug_raw_pointers=0 debug_raw_pointers=0 debug_raw_pointers=Y debug_raw_pointers=Y == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 04:48:16 (1711097296) SKIP: sanityn test_115 ldiskfs only test SKIP 115 (2s) debug_raw_pointers=0 debug_raw_pointers=0 cleanup: ====================================================== == sanityn test complete, duration 5924 sec ============== 04:48:19 (1711097299) === sanityn: start cleanup 04:48:20 (1711097300) === Stopping clients: oleg258-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg258-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 04:50:22 (1711097422) ===