-----============= acceptance-small: sanityn ============----- Fri May 16 08:45:09 EDT 2025 oleg311-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/sanityn.*ex: No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 Started clients oleg311-client.virtnet: 192.168.203.111@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt) oleg311-client.virtnet: executing check_config_client /mnt/lustre oleg311-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg311-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800a9c1c800.idle_timeout=debug osc.lustre-OST0000-osc-ffff88012aaf8800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800a9c1c800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012aaf8800.idle_timeout=debug disable quota as required oleg311-server: oleg311-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179909 s, 58.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6881] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 08:45:30 (1747399530) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (5s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 08:45:34 (1747399534) -rw-r--r-- 1 root root 0 May 16 08:45 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (4s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 08:45:38 (1747399538) -rw-r--r-- 1 root root 0 May 16 08:45 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (3s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 08:45:41 (1747399541) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (4s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 08:45:45 (1747399545) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (3s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 08:45:48 (1747399548) total 1 -rwxrwxrwx 1 root root 0 May 16 08:45 f2a -rwxrwxrwx 1 root root 0 May 16 08:45 f2b total 1 -rwxrwxrwx 1 root root 0 May 16 08:45 f2a -rwxrwxrwx 1 root root 0 May 16 08:45 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00555827 s, 92.1 kB/s PASS 2e (4s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 08:45:52 (1747399552) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 08:45:54 (1747399554) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.386105 s, 5.4 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (4s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 08:45:58 (1747399558) PASS 3 (4s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 08:46:02 (1747399562) PASS 4 (5s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 08:46:07 (1747399567) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (4s) == sanityn test 6: remove of open file on other node ============================================================================ 08:46:11 (1747399571) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (4s) == sanityn test 7: remove of open directory on other node ======================================================================= 08:46:15 (1747399575) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (5s) == sanityn test 8: remove of open special file on other node ==================================================================== 08:46:20 (1747399580) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (4s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 08:46:24 (1747399584) PASS 9a (5s) == sanityn test 9b: append to striped sparse file ======== 08:46:29 (1747399589) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390131 s, 26.9 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0613575 s, 0.0 kB/s Data read (expecting 'foo'): foo PASS 9b (4s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 08:46:33 (1747399593) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00616633 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00422235 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00471984 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00473061 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00808809 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00745353 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00597484 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0034293 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00542643 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00423405 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00663974 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00316478 s, 0.3 kB/s PASS 10a (4s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 08:46:37 (1747399597) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.0233605 s, 132 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0614152 s, 66.7 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000293004 s, 10.5 MB/s PASS 10b (4s) == sanityn test 11: execution of file opened for write should return error ============================================================== 08:46:41 (1747399601) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6881 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (4s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 08:46:45 (1747399605) warning: '-runtime' deprecated, use '-t runtime' instead - link 648 (time 1747399616.76 total 10.01 last 64.74) - link 1237 (time 1747399626.77 total 20.02 last 58.85) - link 1828 (time 1747399636.78 total 30.02 last 59.06) - link 2450 (time 1747399646.78 total 40.03 last 62.17) - link 3105 (time 1747399656.79 total 50.03 last 65.46) - link 3782 (time 1747399666.79 total 60.04 last 67.65) - link 4390 (time 1747399676.81 total 70.05 last 60.73) - link 5007 (time 1747399686.81 total 80.06 last 61.67) - link 5693 (time 1747399696.82 total 90.06 last 68.57) total: 6445 link in 100.00 seconds: 64.45 ops/second using seed 2256476539 running for 100 seconds - stat 10000 (time 1747399613 ; total 6 ; last 6) - stat 20000 (time 1747399619 ; total 12 ; last 6) - stat 30000 (time 1747399625 ; total 18 ; last 6) - stat 40000 (time 1747399631 ; total 24 ; last 6) - stat 50000 (time 1747399636 ; total 29 ; last 5) - stat 60000 (time 1747399642 ; total 35 ; last 6) - stat 70000 (time 1747399647 ; total 40 ; last 5) - stat 80000 (time 1747399652 ; total 45 ; last 5) - stat 90000 (time 1747399661 ; total 54 ; last 9) - stat 93552 (time 1747399672 ; total 65 ; last 11) - stat 97958 (time 1747399683 ; total 76 ; last 11) - stat 100000 (time 1747399689 ; total 82 ; last 6) - stat 104851 (time 1747399700 ; total 93 ; last 11) total: 108914 stats in 100 seconds: 1089.140015 stats/second - unlinked 0 (time 1747399657 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile6445) error: No such file or directory total: 6444 unlinks in 79 seconds: 81.569618 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (20620) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (20623) - No such process PASS 12 (134s) == sanityn test 13: test directory page revocation ======= 08:48:59 (1747399739) f13 f13 PASS 13 (4s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 08:49:04 (1747399744) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6881 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4279: 21997 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (4s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 08:49:08 (1747399748) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4279: 22628 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (3s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 08:49:12 (1747399752) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 23256 Terminated $DIR1/$tdir/sleep 60 PASS 14b (3s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 08:49:15 (1747399755) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 23886 Terminated $DIR1/$tdir/sleep 60 PASS 14c (4s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 08:49:19 (1747399759) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 24516 Terminated $DIR1/$tdir/sleep 60 PASS 14d (4s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 08:49:23 (1747399763) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7527424 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7527424kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (15s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 08:49:38 (1747399778) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.207923 s, 10.1 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.132275 s, 15.9 MB/s Chance of close/open is 1 in 50 Seed set to 9779 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000006[0] 1747399780.115089 mapwrite 0x7d900d thru 0x7e8bf1 (0xfbe5 bytes) 000007[1] 1747399780.255212 mapread 0x6d9d11 thru 0x6dd206 (0x34f6 bytes) truncating to largest ever: 0x1c468f 000010[1] 1747399780.304036 trunc from 0x7e8bf2 to 0x1c468e (0x624563 bytes) 000013[0] 1747399780.372336 mapread 0x0dda43 thru 0x0e57e7 (0x7da5 bytes) 000014[1] 1747399780.402019 mapwrite 0x8b3b69 thru 0x8bfe0c (0xc2a4 bytes) 000016[0] 1747399780.496193 mapread 0x1d9e62 thru 0x1de3bd (0x455c bytes) truncating to largest ever: 0x3f3fe1 000017[1] 1747399780.541024 trunc from 0x8bfe0d to 0x3f3fe0 (0x4cbe2c bytes) 000019[1] 1747399780.581544 write 0x6e96db thru 0x6f5c1f (0xc545 bytes) 000020[0] 1747399780.608983 trunc from 0x6f5c20 to 0x22aebb (0x4cad64 bytes) truncating to largest ever: 0x57d131 000021[1] 1747399780.642561 trunc from 0x22aebc to 0x57d130 (0x352275 bytes) 000027[0] 1747399780.749989 write 0x7dd708 thru 0x7e14dc (0x3dd5 bytes) 000029[1] 1747399780.779444 write 0x94da7f thru 0x9520ef (0x4671 bytes) 000031[0] 1747399780.807106 write 0x5a2ba9 thru 0x5a5d0e (0x3166 bytes) 000032[1] 1747399780.834668 write 0x0d20ec thru 0x0e148e (0xf3a3 bytes) 000033[0] 1747399780.850807 mapread 0x3e07da thru 0x3ed1ca (0xc9f1 bytes) 000035[0] 1747399780.903527 trunc from 0x9520f0 to 0x576915 (0x3db7da bytes) 000038[0] 1747399780.954157 trunc from 0x576916 to 0x386cc7 (0x1efc4e bytes) 000039[1] 1747399780.966643 read 0x1ba66a thru 0x1c7e94 (0xd82b bytes) 000040[0] 1747399780.984256 read 0x16e08a thru 0x1730e6 (0x505d bytes) 000044[1] 1747399781.012609 write 0x189710 thru 0x18ceb4 (0x37a5 bytes) 000045[0] 1747399781.034846 trunc from 0x386cc8 to 0x3196ac (0x6d61b bytes) 000046[0] 1747399781.059812 mapread 0x297e7a thru 0x2a3705 (0xb88c bytes) 000051[0] 1747399781.087548 mapwrite 0x81cde2 thru 0x828cbc (0xbedb bytes) 000052[0] 1747399781.171539 trunc from 0x828cbd to 0x42f0c4 (0x3f9bf8 bytes) 000053[0] 1747399781.206333 mapread 0x1369e9 thru 0x1433a2 (0xc9ba bytes) 000054[0] 1747399781.236707 read 0x3a59b9 thru 0x3a6d7f (0x13c7 bytes) 000055[0] 1747399781.260044 mapwrite 0x38233e thru 0x3853c7 (0x308a bytes) 000057[1] 1747399781.343908 mapread 0x1c3942 thru 0x1cffa8 (0xc667 bytes) 000059[0] 1747399781.383210 read 0x3454fd thru 0x349e5f (0x4963 bytes) 000062[0] 1747399781.426384 read 0x34658c thru 0x35485c (0xe2d1 bytes) 000063[1] 1747399781.437740 mapread 0x1d0e38 thru 0x1dc1d6 (0xb39f bytes) 000064[0] 1747399781.472789 read 0x0fed83 thru 0x10b8a8 (0xcb26 bytes) 000066[0] 1747399781.519886 write 0x9bfa58 thru 0x9c08d0 (0xe79 bytes) 000067[1] 1747399781.533419 write 0x7637ec thru 0x765234 (0x1a49 bytes) 000069[0] 1747399781.567458 trunc from 0x9c08d1 to 0x13a14e (0x886782 bytes) 000072[1] 1747399781.622456 read 0x0555b1 thru 0x065246 (0xfc96 bytes) 000079[0] 1747399781.683329 trunc from 0x13a14f to 0x2418c8 (0x10777a bytes) 000081[0] 1747399781.730599 mapwrite 0x4ff307 thru 0x50cbf4 (0xd8ee bytes) 000082[1] 1747399781.844295 mapwrite 0x29d5f6 thru 0x2a6de7 (0x97f2 bytes) 000083[1] 1747399781.951007 mapread 0x29d7c3 thru 0x2a9cdd (0xc51b bytes) 000089[0] 1747399782.041084 read 0x1e1256 thru 0x1e65ae (0x5359 bytes) 000092[0] 1747399782.077525 mapread 0x15d91e thru 0x16cf08 (0xf5eb bytes) 000093[1] 1747399782.148711 write 0x530e6f thru 0x53d6f5 (0xc887 bytes) 000095[0] 1747399782.172103 read 0x46486b thru 0x473ea3 (0xf639 bytes) 000096[1] 1747399782.209407 mapwrite 0x0ab800 thru 0x0af162 (0x3963 bytes) 000098[1] 1747399782.335100 trunc from 0x53d6f6 to 0x4c0824 (0x7ced1 bytes) 000099[0] 1747399782.352260 trunc from 0x4c0825 to 0x4f8c91 (0x3846d bytes) 000101[1] 1747399782.395616 mapwrite 0x314992 thru 0x315c52 (0x12c1 bytes) 000102[0] 1747399782.449851 read 0x06ca5a thru 0x06e92f (0x1ed6 bytes) truncating to largest ever: 0x79bd33 000105[0] 1747399782.492165 trunc from 0x4f8c92 to 0x79bd32 (0x2a30a1 bytes) 000106[1] 1747399782.518255 read 0x37857b thru 0x38217d (0x9c03 bytes) 000107[0] 1747399782.564066 mapread 0x047181 thru 0x048278 (0x10f8 bytes) 000108[1] 1747399782.628350 mapwrite 0x25e28a thru 0x260292 (0x2009 bytes) 000109[0] 1747399782.765786 write 0x8fceb8 thru 0x90a794 (0xd8dd bytes) 000110[1] 1747399782.793284 trunc from 0x90a795 to 0x74c0fc (0x1be698 bytes) 000111[1] 1747399782.836550 write 0x1b0ce8 thru 0x1be896 (0xdbaf bytes) 000112[0] 1747399782.850390 read 0x112493 thru 0x11e3a6 (0xbf14 bytes) 000114[1] 1747399782.907565 write 0x8d961d thru 0x8e6f39 (0xd91d bytes) 000118[1] 1747399782.935364 trunc from 0x8e6f3a to 0x6ff388 (0x1e7bb1 bytes) 000119[1] 1747399782.966638 trunc from 0x6ff389 to 0x6501c7 (0xaf1c1 bytes) 000122[1] 1747399782.997175 trunc from 0x6501c8 to 0x23abb5 (0x415612 bytes) 000124[1] 1747399783.015826 trunc from 0x23abb6 to 0x1429ee (0xf81c7 bytes) 000125[1] 1747399783.033534 mapwrite 0x262c64 thru 0x26e4f7 (0xb894 bytes) 000128[1] 1747399783.182747 mapwrite 0x6e3095 thru 0x6eaf50 (0x7ebc bytes) 000130[0] 1747399783.286629 mapread 0x0a7b2d thru 0x0b18c7 (0x9d9b bytes) 000131[1] 1747399783.366400 write 0x26e165 thru 0x27b9e2 (0xd87e bytes) 000132[1] 1747399783.390689 write 0x035bfe thru 0x03a00c (0x440f bytes) 000135[1] 1747399783.422474 write 0x608d8c thru 0x615d80 (0xcff5 bytes) truncating to largest ever: 0x9e43f0 000137[1] 1747399783.454921 trunc from 0x6eaf51 to 0x9e43ef (0x2f949f bytes) 000139[1] 1747399783.490373 trunc from 0x9e43f0 to 0x42273d (0x5c1cb2 bytes) 000141[1] 1747399783.524292 trunc from 0x42273e to 0x34233e (0xe03ff bytes) 000142[1] 1747399783.546646 read 0x23d1d2 thru 0x241bf7 (0x4a26 bytes) 000145[1] 1747399783.597521 read 0x0ece13 thru 0x0ef21f (0x240d bytes) 000146[0] 1747399783.629759 write 0x065049 thru 0x07226c (0xd224 bytes) 000152[0] 1747399783.721429 write 0x7ce904 thru 0x7d91ce (0xa8cb bytes) 000155[1] 1747399783.759909 trunc from 0x7d91cf to 0x4ee5af (0x2eac1f bytes) 000156[0] 1747399783.799786 mapwrite 0x94e9dc thru 0x95d85f (0xee84 bytes) 000157[1] 1747399783.935913 mapread 0x308946 thru 0x3185ae (0xfc69 bytes) 000158[1] 1747399784.026654 read 0x4db056 thru 0x4e9b71 (0xeb1c bytes) 000159[0] 1747399784.079720 trunc from 0x95d860 to 0x554688 (0x4091d7 bytes) 000161[0] 1747399784.139532 read 0x0f0e78 thru 0x0f821f (0x73a8 bytes) 000163[0] 1747399784.201613 write 0x83695e thru 0x83bf04 (0x55a7 bytes) 000165[0] 1747399784.210420 mapwrite 0x0b89d7 thru 0x0bd689 (0x4cb3 bytes) 000166[1] 1747399784.283847 trunc from 0x83bf05 to 0x1bc85f (0x67f6a5 bytes) 000170[1] 1747399784.340010 mapread 0x073adb thru 0x078ffe (0x5524 bytes) 000172[0] 1747399784.382382 read 0x189b3a thru 0x18f935 (0x5dfc bytes) 000173[0] 1747399784.399219 write 0x5fa3c6 thru 0x60703c (0xcc77 bytes) 000174[1] 1747399784.435364 mapread 0x49de81 thru 0x4a2415 (0x4595 bytes) 000176[1] 1747399784.493393 read 0x008873 thru 0x00e9cc (0x615a bytes) 000178[0] 1747399784.549403 write 0x631109 thru 0x640662 (0xf55a bytes) 000179[1] 1747399784.570502 write 0x02a211 thru 0x02ccf6 (0x2ae6 bytes) 000180[0] 1747399784.607215 write 0x12293c thru 0x12da85 (0xb14a bytes) 000182[0] 1747399784.650530 mapwrite 0x8de9c5 thru 0x8e6283 (0x78bf bytes) 000183[1] 1747399784.789760 mapread 0x3cc3b4 thru 0x3d4d97 (0x89e4 bytes) 000184[1] 1747399784.878697 read 0x0f0e23 thru 0x0f5a65 (0x4c43 bytes) 000186[1] 1747399784.937595 trunc from 0x8e6284 to 0x08a767 (0x85bb1c bytes) 000188[1] 1747399785.013469 write 0x942f3c thru 0x9467fb (0x38c0 bytes) 000189[0] 1747399785.041016 read 0x8050b6 thru 0x80e00e (0x8f59 bytes) 000191[0] 1747399785.083385 trunc from 0x9467fc to 0x6a5778 (0x2a1083 bytes) 000193[0] 1747399785.138997 trunc from 0x6a5779 to 0x328d61 (0x37ca17 bytes) 000194[0] 1747399785.173148 write 0x10aced thru 0x11148f (0x67a3 bytes) 000195[1] 1747399785.183241 trunc from 0x328d62 to 0x367dba (0x3f059 bytes) 000196[1] 1747399785.219764 trunc from 0x367dbb to 0x475499 (0x10d6df bytes) 000201[1] 1747399785.257811 trunc from 0x47549a to 0x43f472 (0x36027 bytes) 000202[0] 1747399785.287151 trunc from 0x43f473 to 0x9c957a (0x58a108 bytes) 000203[1] 1747399785.316215 mapread 0x8fa71a thru 0x9060f2 (0xb9d9 bytes) 000205[1] 1747399785.376119 write 0x8b238a thru 0x8b8ff2 (0x6c69 bytes) 000208[0] 1747399785.407452 write 0x19b38c thru 0x1a7bf7 (0xc86c bytes) 000209[0] 1747399785.440531 mapread 0x988f56 thru 0x991f11 (0x8fbc bytes) 000210[0] 1747399785.499147 read 0x6019aa thru 0x609dd0 (0x8427 bytes) 000211[0] 1747399785.545848 write 0x9e98ec thru 0x9f8e95 (0xf5aa bytes) 000217[1] 1747399785.571746 read 0x174d40 thru 0x176996 (0x1c57 bytes) 000218[0] 1747399785.666533 read 0x96d0ed thru 0x96ffbe (0x2ed2 bytes) 000219[0] 1747399785.687150 read 0x452538 thru 0x4556f5 (0x31be bytes) 000220[0] 1747399785.711230 mapread 0x4ce8c7 thru 0x4d11cb (0x2905 bytes) 000222[0] 1747399785.713656 mapread 0x6acfb5 thru 0x6b8957 (0xb9a3 bytes) 000223[1] 1747399785.714451 read 0x2fd96f thru 0x303b4d (0x61df bytes) 000224[0] 1747399785.714769 trunc from 0x9f8e96 to 0x5362d6 (0x4c2bbf bytes) 000225[0] 1747399785.784516 write 0x247115 thru 0x24f7cf (0x86bb bytes) 000226[1] 1747399785.795584 mapread 0x09f31b thru 0x0aca18 (0xd6fe bytes) 000228[0] 1747399785.883431 mapread 0x0f38d2 thru 0x100be6 (0xd315 bytes) 000232[1] 1747399785.971831 mapread 0x51abd9 thru 0x522817 (0x7c3f bytes) 000237[1] 1747399786.018270 trunc from 0x5362d7 to 0x1f012b (0x3461ab bytes) 000238[0] 1747399786.058340 write 0x03d28a thru 0x03e6f4 (0x146b bytes) 000239[0] 1747399786.088147 write 0x60d35a thru 0x610d2d (0x39d4 bytes) 000240[1] 1747399786.105218 write 0x93ec8e thru 0x947ba5 (0x8f18 bytes) 000241[1] 1747399786.118569 mapwrite 0x10c734 thru 0x116bd6 (0xa4a3 bytes) 000242[0] 1747399786.219168 mapwrite 0x103072 thru 0x1093d5 (0x6364 bytes) 000244[0] 1747399786.322836 write 0x1ca929 thru 0x1cb540 (0xc18 bytes) 000245[0] 1747399786.342177 trunc from 0x947ba6 to 0x9971c4 (0x4f61f bytes) 000246[0] 1747399786.356061 trunc from 0x9971c5 to 0x8aad8d (0xec437 bytes) 000248[0] 1747399786.382826 read 0x11341a thru 0x11f3ea (0xbfd1 bytes) 000249[0] 1747399786.424125 write 0x1c5c3b thru 0x1d2194 (0xc55a bytes) 000250[0] 1747399786.451503 mapread 0x304d23 thru 0x311311 (0xc5ef bytes) 000251[0] 1747399786.452427 trunc from 0x8aad8e to 0x948348 (0x9d5bb bytes) 000253[0] 1747399786.500555 mapwrite 0x8bcfd3 thru 0x8c59ab (0x89d9 bytes) 000256[1] 1747399786.606192 mapwrite 0x929b64 thru 0x9399c4 (0xfe61 bytes) 000257[0] 1747399786.743617 mapread 0x31e441 thru 0x323d1e (0x58de bytes) 000258[0] 1747399786.790113 mapwrite 0x79f754 thru 0x7af15a (0xfa07 bytes) 000259[1] 1747399786.892046 mapwrite 0x9aa5b8 thru 0x9b535a (0xada3 bytes) 000261[0] 1747399787.023814 read 0x14a2f3 thru 0x14e5a0 (0x42ae bytes) 000263[0] 1747399787.060780 write 0x03db9d thru 0x03e2f1 (0x755 bytes) 000264[1] 1747399787.143921 mapread 0x1e7d81 thru 0x1f499c (0xcc1c bytes) 000265[0] 1747399787.203911 write 0x44cab3 thru 0x44ee24 (0x2372 bytes) 000267[1] 1747399787.215353 read 0x61010d thru 0x611a4a (0x193e bytes) 000269[0] 1747399787.273185 mapwrite 0x4c4d60 thru 0x4cca5e (0x7cff bytes) 000271[1] 1747399787.358181 write 0x0b18e3 thru 0x0bd777 (0xbe95 bytes) 000275[1] 1747399787.404334 read 0x449730 thru 0x451298 (0x7b69 bytes) 000276[1] 1747399787.444651 mapwrite 0x72f96d thru 0x739265 (0x98f9 bytes) 000277[0] 1747399787.555101 mapread 0x697a86 thru 0x69bd6a (0x42e5 bytes) 000278[0] 1747399787.621307 read 0x34a409 thru 0x353e00 (0x99f8 bytes) 000279[0] 1747399787.720701 mapwrite 0x516794 thru 0x5246a0 (0xdf0d bytes) 000280[0] 1747399787.849245 mapread 0x19579a thru 0x1a241f (0xcc86 bytes) 000282[0] 1747399787.923778 trunc from 0x9b535b to 0x2721d7 (0x743183 bytes) 000284[1] 1747399787.955176 mapread 0x1682b9 thru 0x174a14 (0xc75c bytes) 000286[1] 1747399788.026794 mapwrite 0x887caf thru 0x88ba5a (0x3dac bytes) 000287[1] 1747399788.127157 trunc from 0x88ba5b to 0x0e4f5b (0x7a6aff bytes) 000288[1] 1747399788.152741 read 0x07a921 thru 0x07b828 (0xf08 bytes) 000290[1] 1747399788.185362 mapread 0x00ddb5 thru 0x0125a3 (0x47ef bytes) 000293[1] 1747399788.217064 mapread 0x006ec3 thru 0x00a1b4 (0x32f2 bytes) 000294[0] 1747399788.248688 trunc from 0x0e4f5c to 0x681b55 (0x59cbfa bytes) 000297[0] 1747399788.298487 mapread 0x58261b thru 0x58aef1 (0x88d7 bytes) 000298[0] 1747399788.349634 mapwrite 0x240a44 thru 0x247d94 (0x7351 bytes) 000299[1] 1747399788.428786 write 0x9ddbb7 thru 0x9ec9d8 (0xee22 bytes) 000300[1] 1747399788.454414 mapwrite 0x19cc10 thru 0x1a60c7 (0x94b8 bytes) 000301[1] 1747399788.578699 mapwrite 0x067ab0 thru 0x069311 (0x1862 bytes) 000302[1] 1747399788.704827 trunc from 0x9ec9d9 to 0x444ca7 (0x5a7d31 bytes) 000303[1] 1747399788.733467 mapwrite 0x4eddd5 thru 0x4fc42b (0xe657 bytes) 000308[0] 1747399788.918362 mapwrite 0x3de0c9 thru 0x3df03e (0xf76 bytes) 000312[1] 1747399789.047663 write 0x8a4a47 thru 0x8a6288 (0x1842 bytes) 000313[0] 1747399789.058118 mapwrite 0x31c7ea thru 0x31eace (0x22e5 bytes) 000314[1] 1747399789.128727 read 0x103118 thru 0x107e96 (0x4d7f bytes) 000315[0] 1747399789.220802 write 0x98574e thru 0x994566 (0xee19 bytes) 000318[0] 1747399789.293188 trunc from 0x994567 to 0x85718d (0x13d3d9 bytes) 000321[1] 1747399789.370023 mapread 0x0c89f9 thru 0x0d4842 (0xbe4a bytes) 000323[1] 1747399789.488394 read 0x3e5562 thru 0x3eb16d (0x5c0c bytes) 000324[0] 1747399789.530404 write 0x194c77 thru 0x19ca84 (0x7e0e bytes) 000326[0] 1747399789.587237 mapread 0x4bdbd6 thru 0x4c44e7 (0x6912 bytes) 000327[1] 1747399789.618202 write 0x71775c thru 0x724025 (0xc8ca bytes) 000329[0] 1747399789.671385 trunc from 0x85718e to 0x53ffa3 (0x3171ea bytes) 000330[1] 1747399789.722381 mapread 0x04ce35 thru 0x05c17b (0xf347 bytes) 000334[0] 1747399789.844465 read 0x2794c4 thru 0x2834b3 (0x9ff0 bytes) 000337[1] 1747399789.907335 write 0x6a0dc6 thru 0x6a7ee9 (0x7124 bytes) 000341[1] 1747399789.981931 trunc from 0x6a7eea to 0x6d3249 (0x2b360 bytes) 000342[0] 1747399790.008634 mapwrite 0x6d7fe9 thru 0x6d8477 (0x48f bytes) 000344[1] 1747399790.137020 write 0x3ba83f thru 0x3be432 (0x3bf4 bytes) 000345[0] 1747399790.174067 mapread 0x3641c5 thru 0x367ffa (0x3e36 bytes) 000347[1] 1747399790.233337 write 0x3877a6 thru 0x3894c1 (0x1d1c bytes) 000349[0] 1747399790.267510 trunc from 0x6d8478 to 0x57cc35 (0x15b842 bytes) 000352[0] 1747399790.330385 trunc from 0x57cc36 to 0x144abb (0x43817a bytes) 000354[0] 1747399790.361396 mapwrite 0x9fcfb2 thru 0x9fffff (0x304e bytes) 000356[0] 1747399790.467425 mapread 0x6a370f thru 0x6a5da7 (0x2699 bytes) 000361[0] 1747399790.516733 mapwrite 0x530c98 thru 0x533198 (0x2501 bytes) 000367[1] 1747399790.602599 write 0x6176ec thru 0x61ad64 (0x3679 bytes) 000368[0] 1747399790.637740 read 0x8ef7c7 thru 0x8f5a10 (0x624a bytes) 000369[1] 1747399790.677065 mapread 0x0f8a48 thru 0x1026ed (0x9ca6 bytes) 000372[1] 1747399790.743981 mapwrite 0x61d970 thru 0x62439d (0x6a2e bytes) 000375[1] 1747399790.850025 mapwrite 0x2e18ad thru 0x2e6d63 (0x54b7 bytes) 000376[0] 1747399790.912720 mapread 0x4ec747 thru 0x4f4270 (0x7b2a bytes) 000377[1] 1747399790.962274 trunc from 0xa00000 to 0x1680de (0x897f21 bytes) 000378[1] 1747399790.987140 mapwrite 0x9aaaca thru 0x9b33b4 (0x88eb bytes) 000381[0] 1747399791.090276 mapread 0x4fe08e thru 0x50125a (0x31cd bytes) 000382[0] 1747399791.126976 mapwrite 0x75317a thru 0x75f28c (0xc113 bytes) 000387[0] 1747399791.276328 write 0x99d673 thru 0x99e912 (0x12a0 bytes) 000392[0] 1747399791.323753 mapread 0x93156f thru 0x93836a (0x6dfc bytes) 000394[1] 1747399791.351413 read 0x57b280 thru 0x587b05 (0xc886 bytes) 000395[0] 1747399791.389066 mapread 0x1f94b0 thru 0x202041 (0x8b92 bytes) 000396[1] 1747399791.419490 mapwrite 0x5b70ab thru 0x5bc805 (0x575b bytes) 000398[0] 1747399791.522817 write 0x1571e3 thru 0x15db5b (0x6979 bytes) 000399[1] 1747399791.547209 mapread 0x17c577 thru 0x187dfc (0xb886 bytes) 000400[1] 1747399791.613415 mapwrite 0x3eb884 thru 0x3ec058 (0x7d5 bytes) 000402[0] 1747399791.694084 mapread 0x944d5e thru 0x9539b5 (0xec58 bytes) 000405[1] 1747399791.739481 trunc from 0x9b33b5 to 0x145256 (0x86e15e bytes) 000406[0] 1747399791.783327 read 0x039a49 thru 0x041810 (0x7dc8 bytes) 000407[0] 1747399791.811195 mapwrite 0x367514 thru 0x36b0d7 (0x3bc4 bytes) 000408[1] 1747399791.917797 trunc from 0x36b0d8 to 0x5e15d2 (0x2764fb bytes) 000410[0] 1747399791.961240 read 0x56dd6e thru 0x571718 (0x39ab bytes) 000411[0] 1747399791.996507 read 0x1a7067 thru 0x1af5d4 (0x856e bytes) 000412[0] 1747399792.006709 write 0x7fec68 thru 0x807c36 (0x8fcf bytes) 000415[0] 1747399792.039336 mapwrite 0x700f33 thru 0x70c8ba (0xb988 bytes) 000417[1] 1747399792.114669 read 0x024c7e thru 0x033166 (0xe4e9 bytes) 000418[0] 1747399792.155693 mapwrite 0x5355eb thru 0x54395a (0xe370 bytes) 000419[1] 1747399792.211780 write 0x0e1386 thru 0x0e9040 (0x7cbb bytes) 000420[0] 1747399792.247230 mapread 0x3ae5d2 thru 0x3b884c (0xa27b bytes) 000422[0] 1747399792.278970 trunc from 0x807c37 to 0x128f0f (0x6ded27 bytes) 000429[1] 1747399792.305068 read 0x01438e thru 0x01d40d (0x9080 bytes) 000430[1] 1747399792.337322 mapwrite 0x32f1ec thru 0x33b21c (0xc031 bytes) 000431[1] 1747399792.438478 mapwrite 0x0c344a thru 0x0c5601 (0x21b8 bytes) 000435[1] 1747399792.512356 mapwrite 0x03489f thru 0x03fcd7 (0xb439 bytes) 000436[1] 1747399792.607851 trunc from 0x33b21d to 0x10215a (0x2390c2 bytes) 000437[1] 1747399792.632612 read 0x08d270 thru 0x08df66 (0xcf7 bytes) 000439[1] 1747399792.667736 mapread 0x07b1f6 thru 0x086c12 (0xba1d bytes) 000440[1] 1747399792.733323 mapwrite 0x27158d thru 0x2811f8 (0xfc6c bytes) 000441[0] 1747399792.830735 mapread 0x25a92b thru 0x25ea5c (0x4132 bytes) 000444[0] 1747399792.887732 write 0x676dd2 thru 0x681ad7 (0xad06 bytes) 000446[0] 1747399792.923213 mapread 0x1abf77 thru 0x1b1467 (0x54f1 bytes) 000447[0] 1747399792.985429 trunc from 0x681ad8 to 0x422482 (0x25f655 bytes) 000448[1] 1747399793.012901 trunc from 0x422483 to 0x71dbe6 (0x2fb764 bytes) 000451[0] 1747399793.034280 read 0x6da0a7 thru 0x6ddc8c (0x3be6 bytes) 000453[1] 1747399793.063376 trunc from 0x71dbe7 to 0x66546f (0xb8777 bytes) 000454[0] 1747399793.094124 mapread 0x2e5e2e thru 0x2ec181 (0x6354 bytes) 000458[1] 1747399793.176534 trunc from 0x665470 to 0x4684ec (0x1fcf83 bytes) 000460[0] 1747399793.198274 mapread 0x0a2e1b thru 0x0a7a4d (0x4c33 bytes) 000461[1] 1747399793.235915 mapread 0x17eab8 thru 0x185fe4 (0x752d bytes) 000463[1] 1747399793.288673 write 0x269fff thru 0x278bef (0xebf1 bytes) 000464[1] 1747399793.317452 write 0x0c78ba thru 0x0d1802 (0x9f49 bytes) 000470[0] 1747399793.388222 mapread 0x1448cb thru 0x152375 (0xdaab bytes) 000471[1] 1747399793.450645 mapread 0x18b224 thru 0x199383 (0xe160 bytes) 000472[1] 1747399793.498767 read 0x1ff3c8 thru 0x2072d5 (0x7f0e bytes) 000473[1] 1747399793.517174 mapread 0x138838 thru 0x145b6f (0xd338 bytes) 000474[0] 1747399793.574031 write 0x9d7221 thru 0x9de678 (0x7458 bytes) 000476[1] 1747399793.613679 read 0x48c71a thru 0x496a47 (0xa32e bytes) 000478[1] 1747399793.697196 mapwrite 0x69a84e thru 0x6a849b (0xdc4e bytes) 000479[0] 1747399793.986006 mapwrite 0x2a559d thru 0x2a8fa4 (0x3a08 bytes) 000480[0] 1747399794.090755 read 0x8740a4 thru 0x87bd1f (0x7c7c bytes) 000482[1] 1747399794.118412 write 0x274b77 thru 0x27d11a (0x85a4 bytes) 000483[1] 1747399794.141592 trunc from 0x9de679 to 0x16e8c3 (0x86fdb5 bytes) 000485[1] 1747399794.188740 mapwrite 0x364cf3 thru 0x3667b5 (0x1ac3 bytes) 000487[0] 1747399794.275805 mapwrite 0x81be34 thru 0x81cdf6 (0xfc3 bytes) 000489[1] 1747399794.366134 read 0x7e7aa1 thru 0x7ecb8c (0x50ec bytes) 000491[1] 1747399794.379723 mapwrite 0x5d3037 thru 0x5d65e7 (0x35b1 bytes) 000492[0] 1747399794.451338 trunc from 0x81cdf7 to 0x4732fb (0x3a9afb bytes) 000493[1] 1747399794.486345 mapwrite 0x7cbda1 thru 0x7d5293 (0x94f3 bytes) 000495[1] 1747399794.603078 write 0x6481ad thru 0x64f765 (0x75b9 bytes) 000498[0] 1747399794.639274 write 0x83c792 thru 0x84a3bd (0xdc2c bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 9794 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read 000009[0] 1747399795.076370 mapwrite 0x503000 thru 0x50935a (0x635b bytes) truncating to largest ever: 0x48d35c 000011[1] 1747399795.178935 trunc from 0x50935b to 0x48d35b (0x7bfff bytes) 000012[0] 1747399795.211138 mapread 0x145000 thru 0x152968 (0xd969 bytes) 000013[0] 1747399795.241956 write 0x2f2000 thru 0x2f6436 (0x4437 bytes) 000014[0] 1747399795.267664 mapwrite 0x4d6000 thru 0x4d8fd2 (0x2fd3 bytes) 000015[1] 1747399795.357654 write_OD 0x993000 thru 0x9a0fff (0xe000 bytes) 000017[1] 1747399795.402543 read_OD 0x88d000 thru 0x894fff (0x8000 bytes) 000018[1] 1747399795.415544 mapwrite 0x00a000 thru 0x00d538 (0x3539 bytes) 000019[0] 1747399795.470870 trunc from 0x9a1000 to 0x29a85a (0x7067a5 bytes) 000020[0] 1747399795.486182 write 0x15e000 thru 0x1672f6 (0x92f7 bytes) 000022[0] 1747399795.489093 mapread 0x149000 thru 0x152d21 (0x9d22 bytes) 000023[1] 1747399795.500207 mapread 0x157000 thru 0x15805b (0x105c bytes) 000024[1] 1747399795.520234 mapread 0x1b6000 thru 0x1b7e66 (0x1e67 bytes) truncating to largest ever: 0x7cc79d 000025[0] 1747399795.528276 trunc from 0x29a85b to 0x7cc79c (0x531f42 bytes) 000026[1] 1747399795.554100 trunc from 0x7cc79d to 0x5efb2b (0x1dcc71 bytes) 000028[0] 1747399795.566232 trunc from 0x5efb2c to 0x023db0 (0x5cbd7b bytes) 000029[0] 1747399795.584797 read 0x011000 thru 0x01bc9e (0xac9f bytes) 000031[1] 1747399795.600595 mapwrite 0x660000 thru 0x66c830 (0xc831 bytes) 000032[0] 1747399795.673611 write 0x0a7000 thru 0x0a85fc (0x15fd bytes) 000034[0] 1747399795.702842 mapread 0x0a4000 thru 0x0a573e (0x173f bytes) 000035[1] 1747399795.703987 read_OD 0x0c8000 thru 0x0cbfff (0x4000 bytes) 000036[1] 1747399795.723611 mapread 0x3f7000 thru 0x3fa706 (0x3707 bytes) 000037[1] 1747399795.781233 mapread 0x30d000 thru 0x3107c0 (0x37c1 bytes) 000038[0] 1747399795.781563 mapwrite 0x2c7000 thru 0x2d50de (0xe0df bytes) 000040[0] 1747399795.847692 read 0x1e6000 thru 0x1eea1e (0x8a1f bytes) 000041[1] 1747399795.891217 read_OD 0x097000 thru 0x0a0fff (0xa000 bytes) 000042[1] 1747399795.915081 mapread 0x464000 thru 0x469ea0 (0x5ea1 bytes) 000043[1] 1747399795.967773 write_OD 0x103000 thru 0x107fff (0x5000 bytes) 000044[0] 1747399796.016095 trunc from 0x66c831 to 0x0f8b02 (0x573d2e bytes) 000047[1] 1747399796.062564 write_OD 0x8af000 thru 0x8bafff (0xc000 bytes) 000049[1] 1747399796.104541 read_OD 0x00f000 thru 0x010fff (0x2000 bytes) 000050[0] 1747399796.116585 mapwrite 0x6bc000 thru 0x6c97f5 (0xd7f6 bytes) 000051[0] 1747399796.219268 read 0x783000 thru 0x784c50 (0x1c51 bytes) 000052[0] 1747399796.220132 trunc from 0x8bb000 to 0x5e8e12 (0x2d21ed bytes) 000053[1] 1747399796.245865 read_OD 0x4f2000 thru 0x4f3fff (0x2000 bytes) 000054[1] 1747399796.267910 read_OD 0x52f000 thru 0x532fff (0x4000 bytes) truncating to largest ever: 0x86cbd2 000055[1] 1747399796.274821 trunc from 0x5e8e13 to 0x86cbd1 (0x283dbf bytes) 000056[0] 1747399796.292135 read 0x3ed000 thru 0x3f9146 (0xc147 bytes) 000057[0] 1747399796.340050 read 0x7a6000 thru 0x7ad5c2 (0x75c3 bytes) 000058[1] 1747399796.340412 read_OD 0x5b3000 thru 0x5b7fff (0x5000 bytes) 000059[1] 1747399796.356432 write_OD 0x818000 thru 0x819fff (0x2000 bytes) 000060[1] 1747399796.406424 write_OD 0x03a000 thru 0x03afff (0x1000 bytes) 000063[0] 1747399796.459738 mapread 0x682000 thru 0x6883a1 (0x63a2 bytes) 000067[1] 1747399796.494256 write 0x412000 thru 0x418f4e (0x6f4f bytes) 000068[1] 1747399796.515932 read 0x316000 thru 0x31b337 (0x5338 bytes) 000069[1] 1747399796.532826 read 0x3df000 thru 0x3e8a81 (0x9a82 bytes) 000072[0] 1747399796.563716 read 0x4b9000 thru 0x4c0da5 (0x7da6 bytes) 000073[1] 1747399796.621254 mapread 0x504000 thru 0x50902b (0x502c bytes) 000074[1] 1747399796.633931 trunc from 0x86cbd2 to 0x387b82 (0x4e504f bytes) 000078[0] 1747399796.666170 write 0x250000 thru 0x25dab7 (0xdab8 bytes) 000082[1] 1747399796.701560 mapwrite 0x006000 thru 0x007ddb (0x1ddc bytes) 000083[0] 1747399796.795423 mapwrite 0x083000 thru 0x08c062 (0x9063 bytes) 000085[0] 1747399796.876455 write_OD 0x901000 thru 0x906fff (0x6000 bytes) 000086[0] 1747399796.918250 read_OD 0x0d2000 thru 0x0d6fff (0x5000 bytes) 000090[0] 1747399796.930324 write_OD 0x7ea000 thru 0x7f5fff (0xc000 bytes) 000091[0] 1747399796.973272 read_OD 0x29f000 thru 0x2a1fff (0x3000 bytes) 000093[1] 1747399796.980344 read 0x630000 thru 0x632e5e (0x2e5f bytes) 000094[0] 1747399796.999487 trunc from 0x907000 to 0x6cdc53 (0x2393ac bytes) 000098[0] 1747399797.040779 mapwrite 0x3f0000 thru 0x3f3f4c (0x3f4d bytes) 000099[0] 1747399797.121131 write_OD 0x51c000 thru 0x523fff (0x8000 bytes) 000101[1] 1747399797.188584 read 0x650000 thru 0x654238 (0x4239 bytes) 000102[1] 1747399797.197892 mapwrite 0x810000 thru 0x812bd5 (0x2bd6 bytes) 000105[0] 1747399797.270867 mapread 0x30a000 thru 0x3135e5 (0x95e6 bytes) 000109[1] 1747399797.310722 write 0x82c000 thru 0x83087f (0x4880 bytes) 000110[1] 1747399797.328190 write 0x33e000 thru 0x34b180 (0xd181 bytes) 000112[1] 1747399797.347677 read 0x105000 thru 0x10b21d (0x621e bytes) 000113[0] 1747399797.414774 write_OD 0x456000 thru 0x456fff (0x1000 bytes) 000114[0] 1747399797.473365 write_OD 0x873000 thru 0x87dfff (0xb000 bytes) 000116[0] 1747399797.514430 mapread 0x42a000 thru 0x43231e (0x831f bytes) 000121[1] 1747399797.556848 write 0x333000 thru 0x33d9f9 (0xa9fa bytes) 000122[0] 1747399797.583897 mapwrite 0x6cd000 thru 0x6d72ac (0xa2ad bytes) 000124[0] 1747399797.706107 mapwrite 0x33b000 thru 0x33c181 (0x1182 bytes) 000126[0] 1747399797.770481 mapwrite 0x423000 thru 0x42cc72 (0x9c73 bytes) 000127[0] 1747399797.856017 write_OD 0x06c000 thru 0x076fff (0xb000 bytes) 000128[1] 1747399797.893186 write 0x3e7000 thru 0x3e899b (0x199c bytes) 000129[1] 1747399797.912477 write 0x6b8000 thru 0x6c23c4 (0xa3c5 bytes) 000130[1] 1747399797.916670 mapwrite 0x0de000 thru 0x0edd94 (0xfd95 bytes) 000131[0] 1747399798.048379 trunc from 0x87e000 to 0x700917 (0x17d6e8 bytes) 000135[1] 1747399798.082755 write 0x8a9000 thru 0x8ac554 (0x3555 bytes) 000140[1] 1747399798.129645 mapread 0x621000 thru 0x623373 (0x2374 bytes) 000143[0] 1747399798.155610 read_OD 0x128000 thru 0x131fff (0xa000 bytes) 000145[0] 1747399798.178304 read_OD 0x574000 thru 0x577fff (0x4000 bytes) 000146[1] 1747399798.180915 read 0x1dc000 thru 0x1eac7d (0xec7e bytes) 000147[0] 1747399798.229102 mapread 0x83e000 thru 0x84c621 (0xe622 bytes) 000152[1] 1747399798.268034 read 0x34e000 thru 0x3558ab (0x78ac bytes) 000153[0] 1747399798.268336 mapwrite 0x43a000 thru 0x441c7c (0x7c7d bytes) 000154[1] 1747399798.340998 mapread 0x299000 thru 0x2a6c65 (0xdc66 bytes) 000155[0] 1747399798.408184 write_OD 0x6b2000 thru 0x6bffff (0xe000 bytes) 000158[1] 1747399798.444038 mapread 0x3c3000 thru 0x3c9814 (0x6815 bytes) 000159[1] 1747399798.464564 mapwrite 0x555000 thru 0x55f201 (0xa202 bytes) 000163[1] 1747399798.566316 read 0x732000 thru 0x732d52 (0xd53 bytes) 000166[1] 1747399798.611578 mapread 0x7a8000 thru 0x7b6981 (0xe982 bytes) 000170[1] 1747399798.623783 trunc from 0x8ac555 to 0x250cbb (0x65b899 bytes) 000172[0] 1747399798.658674 mapwrite 0x1f2000 thru 0x1f76c1 (0x56c2 bytes) 000173[1] 1747399798.740311 trunc from 0x250cbc to 0x42505d (0x1d43a2 bytes) 000175[0] 1747399798.777130 mapread 0x347000 thru 0x34da1f (0x6a20 bytes) 000176[1] 1747399798.799064 mapread 0x01b000 thru 0x02ac1d (0xfc1e bytes) 000181[0] 1747399798.845105 mapread 0x18f000 thru 0x191c76 (0x2c77 bytes) 000182[0] 1747399798.854040 mapwrite 0x37e000 thru 0x37e69e (0x69f bytes) 000185[1] 1747399798.893395 mapread 0x2b0000 thru 0x2bc013 (0xc014 bytes) 000186[0] 1747399798.946002 read_OD 0x33b000 thru 0x33efff (0x4000 bytes) 000187[1] 1747399798.949102 mapwrite 0x881000 thru 0x8826dd (0x16de bytes) 000188[1] 1747399799.025753 read 0x22f000 thru 0x2321de (0x31df bytes) 000189[0] 1747399799.100747 mapread 0x360000 thru 0x366289 (0x628a bytes) 000190[1] 1747399799.125051 mapwrite 0x64b000 thru 0x65af01 (0xff02 bytes) 000192[1] 1747399799.224471 read 0x5ae000 thru 0x5b4c7e (0x6c7f bytes) 000198[1] 1747399799.305145 trunc from 0x8826de to 0x4a025a (0x3e2483 bytes) 000199[1] 1747399799.325699 write 0x27b000 thru 0x27e049 (0x304a bytes) 000200[0] 1747399799.339211 mapwrite 0x419000 thru 0x41ed7d (0x5d7e bytes) 000201[1] 1747399799.397181 trunc from 0x4a025b to 0x2dde25 (0x1c2435 bytes) 000202[0] 1747399799.416253 trunc from 0x2dde26 to 0x1a93d6 (0x134a4f bytes) 000203[0] 1747399799.437283 trunc from 0x1a93d7 to 0x7f306d (0x649c97 bytes) 000206[0] 1747399799.482347 mapread 0x4e6000 thru 0x4e8fea (0x2feb bytes) 000207[0] 1747399799.500474 read_OD 0x2bc000 thru 0x2c9fff (0xe000 bytes) 000208[1] 1747399799.512804 mapread 0x503000 thru 0x50a3ff (0x7400 bytes) 000209[0] 1747399799.533669 mapwrite 0x755000 thru 0x761e6e (0xce6f bytes) 000211[0] 1747399799.637448 read_OD 0x78d000 thru 0x796fff (0xa000 bytes) 000213[1] 1747399799.661567 read 0x63f000 thru 0x64bed5 (0xced6 bytes) 000214[1] 1747399799.677752 mapwrite 0x4b0000 thru 0x4b71de (0x71df bytes) 000218[0] 1747399799.811515 read_OD 0x375000 thru 0x37bfff (0x7000 bytes) 000220[0] 1747399799.833519 read_OD 0x349000 thru 0x34cfff (0x4000 bytes) 000221[0] 1747399799.836864 write_OD 0x168000 thru 0x169fff (0x2000 bytes) 000222[0] 1747399799.873414 write_OD 0x6dc000 thru 0x6e2fff (0x7000 bytes) 000224[0] 1747399799.912810 mapread 0x385000 thru 0x390a60 (0xba61 bytes) 000225[1] 1747399799.954193 write 0x4f6000 thru 0x5031b0 (0xd1b1 bytes) 000226[1] 1747399799.972769 mapwrite 0x47e000 thru 0x4830ef (0x50f0 bytes) 000228[1] 1747399800.023213 write 0x292000 thru 0x29be59 (0x9e5a bytes) 000230[0] 1747399800.039356 mapwrite 0x22b000 thru 0x239471 (0xe472 bytes) 000232[1] 1747399800.166857 mapread 0x621000 thru 0x62d2e1 (0xc2e2 bytes) 000234[0] 1747399800.209278 trunc from 0x7f306e to 0x3f56b6 (0x3fd9b7 bytes) 000237[0] 1747399800.236873 trunc from 0x3f56b7 to 0x680b83 (0x28b4cd bytes) 000241[0] 1747399800.272790 trunc from 0x680b84 to 0x5de004 (0xa2b7f bytes) 000242[0] 1747399800.282182 mapwrite 0x27b000 thru 0x27cdce (0x1dcf bytes) 000245[0] 1747399800.344801 trunc from 0x5de005 to 0x6dea31 (0x100a2d bytes) 000246[1] 1747399800.358102 trunc from 0x6dea32 to 0x6bb1ae (0x23883 bytes) 000247[0] 1747399800.373923 trunc from 0x6bb1af to 0x3c6121 (0x2f508d bytes) 000249[0] 1747399800.389672 mapwrite 0x966000 thru 0x96dced (0x7cee bytes) 000250[1] 1747399800.454258 mapread 0x7d7000 thru 0x7dbd0e (0x4d0f bytes) 000251[0] 1747399800.505649 mapwrite 0x533000 thru 0x542e4b (0xfe4c bytes) 000252[1] 1747399800.603613 mapread 0x462000 thru 0x465bd3 (0x3bd4 bytes) 000253[1] 1747399800.629829 read 0x5ed000 thru 0x5f63fc (0x93fd bytes) 000254[0] 1747399800.657436 write_OD 0x24d000 thru 0x256fff (0xa000 bytes) 000256[0] 1747399800.691021 mapwrite 0x0e1000 thru 0x0e5593 (0x4594 bytes) 000257[1] 1747399800.755245 read 0x4a5000 thru 0x4a8850 (0x3851 bytes) 000258[0] 1747399800.793088 mapwrite 0x28f000 thru 0x2938ff (0x4900 bytes) 000259[1] 1747399800.868677 read 0x90f000 thru 0x918ca4 (0x9ca5 bytes) 000260[0] 1747399800.884852 mapread 0x16b000 thru 0x1756ef (0xa6f0 bytes) truncating to largest ever: 0x9994ec 000263[1] 1747399800.923380 trunc from 0x96dcee to 0x9994eb (0x2b7fe bytes) 000264[1] 1747399800.958315 read 0x495000 thru 0x4a009f (0xb0a0 bytes) 000266[1] 1747399801.008497 mapread 0x17e000 thru 0x18a0a6 (0xc0a7 bytes) 000267[0] 1747399801.066250 read_OD 0x6a3000 thru 0x6aefff (0xc000 bytes) 000268[1] 1747399801.086310 trunc from 0x9994ec to 0x281966 (0x717b85 bytes) 000269[1] 1747399801.112634 write 0x9d1000 thru 0x9d2908 (0x1909 bytes) 000270[0] 1747399801.119345 trunc from 0x9d2909 to 0x88d4a6 (0x145462 bytes) 000271[0] 1747399801.132619 read_OD 0x827000 thru 0x82dfff (0x7000 bytes) 000273[1] 1747399801.149608 mapwrite 0x647000 thru 0x64d0d4 (0x60d5 bytes) 000274[1] 1747399801.222921 read 0x1ce000 thru 0x1d81aa (0xa1ab bytes) 000276[0] 1747399801.297491 write_OD 0x953000 thru 0x958fff (0x6000 bytes) 000279[1] 1747399801.347161 write 0x8e0000 thru 0x8ec3e2 (0xc3e3 bytes) 000283[1] 1747399801.392741 mapwrite 0x35d000 thru 0x368eca (0xbecb bytes) 000285[1] 1747399801.488192 write 0x622000 thru 0x62a5f3 (0x85f4 bytes) 000289[0] 1747399801.529675 trunc from 0x959000 to 0x73ba4e (0x21d5b1 bytes) 000290[0] 1747399801.569007 mapread 0x45c000 thru 0x463a6f (0x7a70 bytes) 000291[1] 1747399801.601357 mapread 0x382000 thru 0x388424 (0x6425 bytes) 000295[0] 1747399801.638330 mapread 0x730000 thru 0x73a1be (0xa1bf bytes) 000297[0] 1747399801.676538 trunc from 0x73ba4f to 0x1bb41a (0x580634 bytes) 000299[1] 1747399801.708779 trunc from 0x1bb41b to 0x66923f (0x4ade25 bytes) 000300[0] 1747399801.740252 write_OD 0x3fa000 thru 0x406fff (0xd000 bytes) 000301[0] 1747399801.776836 read_OD 0x4eb000 thru 0x4f7fff (0xd000 bytes) 000302[0] 1747399801.786481 trunc from 0x669240 to 0x1a1097 (0x4c81a8 bytes) 000303[0] 1747399801.817922 write_OD 0x828000 thru 0x82cfff (0x5000 bytes) 000304[1] 1747399801.861129 write 0x30e000 thru 0x30f464 (0x1465 bytes) 000305[1] 1747399801.893240 mapread 0x319000 thru 0x326d1c (0xdd1d bytes) 000306[1] 1747399801.966132 mapwrite 0x69e000 thru 0x6a210b (0x410c bytes) 000307[0] 1747399802.032256 trunc from 0x82d000 to 0x42a39d (0x402c62 bytes) 000308[0] 1747399802.061129 mapwrite 0x771000 thru 0x77ec2c (0xdc2d bytes) 000309[1] 1747399802.144905 read 0x38b000 thru 0x38ded1 (0x2ed2 bytes) 000311[1] 1747399802.216355 trunc from 0x77ec2d to 0x78895c (0x9d30 bytes) 000315[0] 1747399802.293371 mapread 0x51c000 thru 0x52bcd3 (0xfcd4 bytes) 000316[0] 1747399802.386042 trunc from 0x78895d to 0x7d642b (0x4dacf bytes) 000317[0] 1747399802.409543 write_OD 0x378000 thru 0x384fff (0xd000 bytes) 000320[1] 1747399802.465171 mapread 0x70b000 thru 0x70f8c7 (0x48c8 bytes) 000321[0] 1747399802.488590 trunc from 0x7d642c to 0x4a7ed9 (0x32e552 bytes) 000322[1] 1747399802.510664 read 0x2c0000 thru 0x2c74fb (0x74fc bytes) 000323[0] 1747399802.543317 mapread 0x1fc000 thru 0x20923b (0xd23c bytes) 000325[0] 1747399802.584774 write_OD 0x47e000 thru 0x480fff (0x3000 bytes) 000326[1] 1747399802.628832 read 0x355000 thru 0x3642bf (0xf2c0 bytes) 000330[1] 1747399802.651918 trunc from 0x4a7eda to 0x259e16 (0x24e0c3 bytes) 000332[1] 1747399802.686767 mapread 0x057000 thru 0x0602d3 (0x92d4 bytes) 000335[1] 1747399802.774879 write 0x53b000 thru 0x548642 (0xd643 bytes) 000337[0] 1747399802.796544 mapread 0x41a000 thru 0x42989f (0xf8a0 bytes) 000340[0] 1747399802.869478 mapread 0x528000 thru 0x5317a6 (0x97a7 bytes) 000343[0] 1747399802.896829 trunc from 0x548643 to 0x5ce859 (0x86217 bytes) 000345[0] 1747399802.923314 write_OD 0x37f000 thru 0x380fff (0x2000 bytes) 000346[0] 1747399802.975171 mapwrite 0x535000 thru 0x536c6c (0x1c6d bytes) 000348[0] 1747399803.067199 trunc from 0x5ce85a to 0x4246bc (0x1aa19d bytes) 000351[1] 1747399803.095706 mapread 0x165000 thru 0x1749b2 (0xf9b3 bytes) 000353[0] 1747399803.180733 mapwrite 0x860000 thru 0x8632d1 (0x32d2 bytes) 000355[1] 1747399803.290438 mapread 0x690000 thru 0x69755d (0x755e bytes) 000356[0] 1747399803.339492 mapread 0x6ca000 thru 0x6d0aa9 (0x6aaa bytes) 000357[1] 1747399803.368493 read 0x129000 thru 0x131657 (0x8658 bytes) 000360[0] 1747399803.404537 mapread 0x1c4000 thru 0x1c5343 (0x1344 bytes) 000362[1] 1747399803.421163 write 0x2d5000 thru 0x2dc8ec (0x78ed bytes) 000363[1] 1747399803.443423 read 0x54a000 thru 0x550e99 (0x6e9a bytes) 000364[1] 1747399803.486640 trunc from 0x8632d2 to 0x585c90 (0x2dd641 bytes) 000365[1] 1747399803.509413 trunc from 0x585c91 to 0x3e2fd1 (0x1a2cbf bytes) 000367[0] 1747399803.547739 write_OD 0x28e000 thru 0x291fff (0x4000 bytes) 000369[0] 1747399803.617210 mapwrite 0x260000 thru 0x26a5e5 (0xa5e6 bytes) 000370[1] 1747399803.717312 mapwrite 0x5b0000 thru 0x5baef5 (0xaef6 bytes) 000371[0] 1747399803.845498 mapwrite 0x6ed000 thru 0x6fa082 (0xd083 bytes) 000372[0] 1747399803.957039 trunc from 0x6fa083 to 0x06f569 (0x68ab19 bytes) 000374[1] 1747399803.975374 mapread 0x041000 thru 0x050e87 (0xfe88 bytes) 000375[0] 1747399804.041212 mapwrite 0x56c000 thru 0x57a90f (0xe910 bytes) 000378[0] 1747399804.211871 mapwrite 0x8ca000 thru 0x8d138b (0x738c bytes) 000379[0] 1747399804.295480 mapwrite 0x1a6000 thru 0x1afad4 (0x9ad5 bytes) 000381[1] 1747399804.406507 mapread 0x3f8000 thru 0x401748 (0x9749 bytes) 000382[0] 1747399804.457560 mapread 0x5ae000 thru 0x5bb22d (0xd22e bytes) 000387[1] 1747399804.513035 write 0x985000 thru 0x986f9f (0x1fa0 bytes) 000388[1] 1747399804.521629 write 0x2d8000 thru 0x2dfc12 (0x7c13 bytes) 000392[0] 1747399804.563017 read_OD 0x926000 thru 0x927fff (0x2000 bytes) 000393[1] 1747399804.582352 write 0x2c6000 thru 0x2d19ae (0xb9af bytes) 000394[1] 1747399804.592116 write 0x0f8000 thru 0x0f980a (0x180b bytes) 000395[0] 1747399804.613842 trunc from 0x986fa0 to 0x4cd58c (0x4b9a13 bytes) 000396[0] 1747399804.657350 read_OD 0x2fa000 thru 0x303fff (0xa000 bytes) 000399[0] 1747399804.678195 mapwrite 0x730000 thru 0x73e981 (0xe982 bytes) 000400[0] 1747399804.808911 mapwrite 0x812000 thru 0x820564 (0xe565 bytes) 000402[1] 1747399804.900443 read 0x2ff000 thru 0x304295 (0x5296 bytes) 000404[0] 1747399804.958316 trunc from 0x820565 to 0x908c1f (0xe86bb bytes) 000405[0] 1747399804.982225 read_OD 0x31c000 thru 0x31efff (0x3000 bytes) 000406[1] 1747399804.991109 mapwrite 0x0df000 thru 0x0e6a77 (0x7a78 bytes) 000409[1] 1747399805.105720 mapread 0x4ce000 thru 0x4ddece (0xfecf bytes) 000411[0] 1747399805.164513 write_OD 0x948000 thru 0x94bfff (0x4000 bytes) 000412[0] 1747399805.212352 mapread 0x8d9000 thru 0x8df7a4 (0x67a5 bytes) 000413[1] 1747399805.285356 write 0x4f3000 thru 0x4f50a8 (0x20a9 bytes) 000415[1] 1747399805.320570 trunc from 0x94c000 to 0x5b9374 (0x392c8b bytes) 000416[0] 1747399805.339398 read_OD 0x529000 thru 0x52cfff (0x4000 bytes) 000417[1] 1747399805.360278 mapread 0x289000 thru 0x295064 (0xc065 bytes) 000419[1] 1747399805.394883 mapread 0x402000 thru 0x403c7c (0x1c7d bytes) 000421[1] 1747399805.401504 mapwrite 0x4d4000 thru 0x4e0dad (0xcdae bytes) 000424[0] 1747399805.529025 trunc from 0x5b9375 to 0x30b219 (0x2ae15b bytes) 000425[0] 1747399805.544883 trunc from 0x30b21a to 0x262021 (0xa91f8 bytes) 000426[1] 1747399805.554772 trunc from 0x262022 to 0x78dd60 (0x52bd3f bytes) 000429[1] 1747399805.631070 read 0x0fa000 thru 0x102977 (0x8978 bytes) 000432[0] 1747399805.698163 read_OD 0x394000 thru 0x396fff (0x3000 bytes) 000434[0] 1747399805.734120 trunc from 0x78dd61 to 0x9433d9 (0x1b5679 bytes) 000436[0] 1747399805.760815 write_OD 0x6e6000 thru 0x6effff (0xa000 bytes) 000437[0] 1747399805.809881 trunc from 0x9433da to 0x551788 (0x3f1c51 bytes) 000439[0] 1747399805.834410 read_OD 0x0c0000 thru 0x0c3fff (0x4000 bytes) 000441[0] 1747399805.844557 mapread 0x1fc000 thru 0x20bc2a (0xfc2b bytes) 000445[0] 1747399805.908636 read_OD 0x40e000 thru 0x412fff (0x5000 bytes) 000446[1] 1747399805.921116 write 0x5ad000 thru 0x5ae867 (0x1868 bytes) 000448[1] 1747399805.961860 write 0x9ac000 thru 0x9ba43a (0xe43b bytes) 000449[1] 1747399805.964472 write 0x23b000 thru 0x24384f (0x8850 bytes) 000450[1] 1747399805.979569 read 0x036000 thru 0x040a62 (0xaa63 bytes) 000451[0] 1747399806.034369 mapread 0x226000 thru 0x22b125 (0x5126 bytes) 000453[1] 1747399806.118224 mapread 0x814000 thru 0x821056 (0xd057 bytes) 000455[1] 1747399806.162536 write 0x6f5000 thru 0x700dd9 (0xbdda bytes) 000456[1] 1747399806.181057 mapwrite 0x779000 thru 0x784421 (0xb422 bytes) 000457[1] 1747399806.265647 mapwrite 0x68e000 thru 0x68f969 (0x196a bytes) 000460[1] 1747399806.341751 mapread 0x953000 thru 0x95bc7a (0x8c7b bytes) 000461[0] 1747399806.393819 trunc from 0x9ba43b to 0x97affb (0x3f43f bytes) 000463[0] 1747399806.419303 trunc from 0x97affc to 0x6752b3 (0x305d48 bytes) 000467[0] 1747399806.468495 mapread 0x1ec000 thru 0x1ef3a5 (0x33a6 bytes) 000469[1] 1747399806.485246 mapread 0x3b9000 thru 0x3be8f4 (0x58f5 bytes) 000471[1] 1747399806.509665 mapread 0x0ba000 thru 0x0ba51b (0x51c bytes) 000475[0] 1747399806.515284 read_OD 0x16c000 thru 0x16efff (0x3000 bytes) 000476[0] 1747399806.520125 read_OD 0x0f5000 thru 0x0fbfff (0x7000 bytes) 000477[0] 1747399806.527425 mapread 0x5c8000 thru 0x5cae65 (0x2e66 bytes) 000481[0] 1747399806.574180 write_OD 0x600000 thru 0x604fff (0x5000 bytes) 000482[1] 1747399806.627065 mapread 0x1ca000 thru 0x1d0288 (0x6289 bytes) 000483[1] 1747399806.652430 write 0x09d000 thru 0x09f24e (0x224f bytes) 000487[0] 1747399806.663168 mapwrite 0x13a000 thru 0x1404ac (0x64ad bytes) truncating to largest ever: 0x9f7121 000490[1] 1747399806.747970 trunc from 0x6752b4 to 0x9f7120 (0x381e6d bytes) 000497[0] 1747399806.805819 mapwrite 0x8a5000 thru 0x8abd1a (0x6d1b bytes) 000498[0] 1747399806.893866 trunc from 0x9f7121 to 0x4d6658 (0x520ac8 bytes) 000500[1] 1747399806.928586 write 0x4ed000 thru 0x4ee21e (0x121f bytes) All operations completed A-OK! PASS 16a (30s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 08:50:08 (1747399808) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.229542 s, 9.1 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.164307 s, 12.8 MB/s Chance of close/open is 1 in 50 Seed set to 9810 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1272ac 000002[1] 1747399810.676599 trunc from 00000000 to 0x1272ab (0x1272ac bytes) 000005[0] 1747399810.718576 read 0x0b020a thru 0x0b0f96 (0xd8d bytes) 000006[0] 1747399810.728384 write 0x708d5c thru 0x70a255 (0x14fa bytes) 000007[0] 1747399810.742543 mapread 0x458e1e thru 0x459f77 (0x115a bytes) 000008[1] 1747399810.756035 read 0x498dde thru 0x49ac0c (0x1e2f bytes) truncating to largest ever: 0x7d7a56 000011[0] 1747399810.802665 trunc from 0x70a256 to 0x7d7a55 (0xcd800 bytes) 000012[0] 1747399810.830481 write 0x687bea thru 0x689960 (0x1d77 bytes) 000014[0] 1747399810.867079 trunc from 0x7d7a56 to 0x3ce4d3 (0x409582 bytes) 000016[1] 1747399810.902490 write 0x66910c thru 0x669a4b (0x940 bytes) 000018[0] 1747399810.929732 mapwrite 0x94f14f thru 0x95032a (0x11dc bytes) 000019[0] 1747399811.035601 mapread 0x029731 thru 0x02ac88 (0x1558 bytes) 000020[1] 1747399811.043907 write 0x79a957 thru 0x79c7b8 (0x1e62 bytes) 000021[0] 1747399811.073597 read 0x1c705e thru 0x1c73dd (0x380 bytes) 000022[1] 1747399811.150749 mapread 0x247ecc thru 0x248a17 (0xb4c bytes) truncating to largest ever: 0x81f01b 000026[0] 1747399811.179715 trunc from 0x95032b to 0x81f01a (0x131310 bytes) 000027[0] 1747399811.226472 mapread 0x5683d4 thru 0x56840a (0x37 bytes) 000028[1] 1747399811.232571 mapwrite 0x36f34b thru 0x370ed2 (0x1b88 bytes) 000030[0] 1747399811.315037 mapwrite 0x94b4da thru 0x94c08c (0xbb3 bytes) 000033[1] 1747399811.399901 mapwrite 0x86ed58 thru 0x86efd3 (0x27c bytes) 000036[0] 1747399811.504051 trunc from 0x94c08d to 0x7b78d7 (0x1947b5 bytes) 000037[1] 1747399811.554486 mapwrite 0x1b0527 thru 0x1b1226 (0xd00 bytes) 000038[1] 1747399811.646974 trunc from 0x7b78d8 to 0x7f415d (0x3c886 bytes) truncating to largest ever: 0x8a80da 000040[1] 1747399811.678073 trunc from 0x7f415e to 0x8a80d9 (0xb3f7c bytes) 000042[0] 1747399811.689715 mapwrite 0x29b38e thru 0x29c58a (0x11fd bytes) 000043[1] 1747399811.774601 mapread 0x1315dd thru 0x13204a (0xa6e bytes) 000044[1] 1747399811.792865 write 0x3449c0 thru 0x3457fd (0xe3e bytes) 000049[0] 1747399811.863631 write 0x0e73d6 thru 0x0e7ef0 (0xb1b bytes) 000051[0] 1747399811.885458 mapread 0x0ce8b2 thru 0x0cfcde (0x142d bytes) 000052[1] 1747399811.914408 mapread 0x71c6b0 thru 0x71de34 (0x1785 bytes) 000053[1] 1747399811.948252 read 0x0d3446 thru 0x0d4ff4 (0x1baf bytes) 000055[1] 1747399812.045916 mapread 0x09df4e thru 0x09dfc0 (0x73 bytes) 000056[0] 1747399812.052471 write 0x220eb2 thru 0x222953 (0x1aa2 bytes) 000057[1] 1747399812.102252 trunc from 0x8a80da to 0x7de850 (0xc9889 bytes) 000058[1] 1747399812.150697 trunc from 0x7de851 to 0x763d07 (0x7ab49 bytes) truncating to largest ever: 0x8b2af5 000059[0] 1747399812.159842 trunc from 0x763d08 to 0x8b2af4 (0x14eded bytes) 000060[0] 1747399812.199547 mapwrite 0x68f808 thru 0x690fc2 (0x17bb bytes) 000061[1] 1747399812.265022 mapwrite 0x878984 thru 0x878a2c (0xa9 bytes) 000063[1] 1747399812.384858 trunc from 0x8b2af5 to 0x816b7f (0x9bf75 bytes) 000065[0] 1747399812.414964 read 0x0f4533 thru 0x0f61aa (0x1c78 bytes) 000067[1] 1747399812.435380 write 0x1a7201 thru 0x1a8e89 (0x1c89 bytes) 000068[0] 1747399812.441786 mapwrite 0x7be84a thru 0x7bfb6b (0x1322 bytes) 000073[0] 1747399812.598121 read 0x699f0f thru 0x69ae55 (0xf47 bytes) 000074[1] 1747399812.598665 mapwrite 0x73a950 thru 0x73b88b (0xf3c bytes) 000077[0] 1747399812.664296 write 0x662742 thru 0x6634a1 (0xd60 bytes) 000078[1] 1747399812.696619 read 0x221f31 thru 0x2233cb (0x149b bytes) 000080[0] 1747399812.768031 write 0x6ba815 thru 0x6bb645 (0xe31 bytes) 000081[1] 1747399812.793423 read 0x443d02 thru 0x445ac7 (0x1dc6 bytes) 000085[0] 1747399812.835561 trunc from 0x816b80 to 0x650353 (0x1c682c bytes) 000086[0] 1747399812.875556 mapread 0x0e2299 thru 0x0e3ebb (0x1c23 bytes) 000087[0] 1747399812.901356 read 0x4ce224 thru 0x4cec36 (0xa13 bytes) 000090[0] 1747399812.946768 trunc from 0x650354 to 0x679eb9 (0x29b66 bytes) 000091[1] 1747399812.957539 mapwrite 0x7fc803 thru 0x7fc96c (0x16a bytes) 000092[1] 1747399813.064433 read 0x0bdb50 thru 0x0bde34 (0x2e5 bytes) 000097[0] 1747399813.144598 read 0x79ef0b thru 0x79f0b2 (0x1a8 bytes) 000099[1] 1747399813.185047 mapread 0x31e609 thru 0x3202a7 (0x1c9f bytes) 000101[0] 1747399813.214193 write 0x38611b thru 0x38668e (0x574 bytes) 000102[1] 1747399813.233229 write 0x20a0c8 thru 0x20a78c (0x6c5 bytes) 000103[0] 1747399813.241545 read 0x42dc3e thru 0x42ea47 (0xe0a bytes) 000107[1] 1747399813.324239 write 0x7414ec thru 0x741c31 (0x746 bytes) truncating to largest ever: 0x8de9de 000111[0] 1747399813.349552 trunc from 0x7fc96d to 0x8de9dd (0xe2071 bytes) 000112[1] 1747399813.401799 mapwrite 0x6db64e thru 0x6dcdd2 (0x1785 bytes) 000116[0] 1747399813.529370 mapread 0x21f72d thru 0x21f8b0 (0x184 bytes) 000119[0] 1747399813.563495 trunc from 0x8de9de to 0x55ac9d (0x383d40 bytes) 000120[1] 1747399813.580103 mapread 0x04dc2b thru 0x04e64b (0xa21 bytes) 000126[0] 1747399813.629013 trunc from 0x55ac9e to 0x56684f (0xbbb2 bytes) 000129[1] 1747399813.660222 write 0x0ef6a3 thru 0x0f0fa2 (0x1900 bytes) truncating to largest ever: 0x9f612d 000130[0] 1747399813.682608 trunc from 0x566850 to 0x9f612c (0x48f8dd bytes) 000131[0] 1747399813.719383 write 0x9965ad thru 0x997332 (0xd86 bytes) 000132[0] 1747399813.747877 mapwrite 0x04d76a thru 0x04e265 (0xafc bytes) 000133[0] 1747399813.820152 read 0x89bd48 thru 0x89d3c4 (0x167d bytes) 000135[0] 1747399813.841515 mapwrite 0x783404 thru 0x783c36 (0x833 bytes) 000136[0] 1747399813.892753 write 0x629be8 thru 0x62afd9 (0x13f2 bytes) 000139[0] 1747399813.917305 trunc from 0x9f612d to 0x7235cd (0x2d2b5f bytes) 000140[0] 1747399813.933297 read 0x033907 thru 0x0351e4 (0x18de bytes) 000143[1] 1747399813.996077 read 0x6f1cb0 thru 0x6f2de9 (0x113a bytes) 000152[1] 1747399814.019207 mapwrite 0x9734e4 thru 0x974be5 (0x1702 bytes) 000153[1] 1747399814.111850 mapread 0x0c85c3 thru 0x0c9e39 (0x1877 bytes) 000154[0] 1747399814.120147 mapwrite 0x5971b7 thru 0x5989a1 (0x17eb bytes) 000155[0] 1747399814.206401 trunc from 0x974be6 to 0x0e3b8c (0x891059 bytes) 000156[0] 1747399814.262821 mapread 0x0b4b97 thru 0x0b6ad3 (0x1f3d bytes) 000157[1] 1747399814.299734 mapwrite 0x661dc2 thru 0x663778 (0x19b7 bytes) 000158[0] 1747399814.384121 read 0x0196f5 thru 0x01a96c (0x1278 bytes) 000159[0] 1747399814.426591 read 0x0f4d81 thru 0x0f612c (0x13ac bytes) 000161[0] 1747399814.440299 read 0x2fe1ed thru 0x3000b9 (0x1ecd bytes) 000164[1] 1747399814.484810 mapwrite 0x3172be thru 0x318b9f (0x18e2 bytes) 000167[1] 1747399814.550727 trunc from 0x663779 to 0x31abd0 (0x348ba8 bytes) 000168[0] 1747399814.576990 mapread 0x0686ef thru 0x069b5d (0x146f bytes) 000169[0] 1747399814.610614 write 0x079870 thru 0x07ab49 (0x12da bytes) 000170[0] 1747399814.648262 mapwrite 0x2e6b49 thru 0x2e712d (0x5e5 bytes) 000171[1] 1747399814.713554 trunc from 0x31abd1 to 0x895c0c (0x57b03c bytes) 000172[0] 1747399814.738513 read 0x40bfd5 thru 0x40c9cb (0x9f7 bytes) 000173[1] 1747399814.769342 mapread 0x5bddf0 thru 0x5be29e (0x4af bytes) 000175[0] 1747399814.786336 read 0x1f864f thru 0x1f9fa4 (0x1956 bytes) 000178[0] 1747399814.831844 read 0x49501e thru 0x496e40 (0x1e23 bytes) 000180[0] 1747399814.837401 mapwrite 0x26aa08 thru 0x26c23b (0x1834 bytes) 000181[1] 1747399814.909900 mapwrite 0x1beffc thru 0x1c0e4b (0x1e50 bytes) 000182[0] 1747399814.999149 write 0x16e747 thru 0x170516 (0x1dd0 bytes) 000186[0] 1747399815.021630 mapwrite 0x1f3cb6 thru 0x1f5776 (0x1ac1 bytes) 000188[1] 1747399815.071481 mapwrite 0x4fd034 thru 0x4fd9f6 (0x9c3 bytes) 000189[1] 1747399815.121887 read 0x1208a9 thru 0x122188 (0x18e0 bytes) 000191[0] 1747399815.203310 mapread 0x3aec0f thru 0x3af079 (0x46b bytes) 000193[1] 1747399815.230301 mapwrite 0x5329c3 thru 0x533e0b (0x1449 bytes) 000198[1] 1747399815.320580 write 0x0b495e thru 0x0b5c86 (0x1329 bytes) 000202[1] 1747399815.354855 read 0x05386d thru 0x05492c (0x10c0 bytes) 000208[0] 1747399815.438400 mapread 0x3cbe07 thru 0x3ccc77 (0xe71 bytes) 000211[1] 1747399815.490249 read 0x44194c thru 0x443145 (0x17fa bytes) 000212[1] 1747399815.515441 read 0x13ecf8 thru 0x14071a (0x1a23 bytes) 000214[1] 1747399815.539803 mapread 0x2fe755 thru 0x2ff58b (0xe37 bytes) 000215[0] 1747399815.553040 mapwrite 0x02b261 thru 0x02d12b (0x1ecb bytes) 000216[0] 1747399815.640832 trunc from 0x895c0d to 0x1cab16 (0x6cb0f6 bytes) 000221[1] 1747399815.689821 mapread 0x03e42a thru 0x03f5f6 (0x11cd bytes) 000222[1] 1747399815.719581 mapread 0x0988b9 thru 0x098b4b (0x293 bytes) 000224[1] 1747399815.747563 write 0x9651db thru 0x9666d9 (0x14ff bytes) 000225[0] 1747399815.759768 write 0x64aa7a thru 0x64c445 (0x19cc bytes) 000228[1] 1747399815.799119 trunc from 0x9666da to 0x35fa0e (0x606ccb bytes) 000229[1] 1747399815.821166 read 0x22ec76 thru 0x2300b3 (0x143e bytes) 000230[1] 1747399815.843000 read 0x011cd8 thru 0x012d36 (0x105f bytes) 000232[1] 1747399815.886463 write 0x038913 thru 0x039c8e (0x137c bytes) 000233[1] 1747399815.897136 trunc from 0x35fa0f to 0x5171ae (0x1b77a0 bytes) 000234[0] 1747399815.929561 mapwrite 0x69b739 thru 0x69c7f9 (0x10c1 bytes) 000235[1] 1747399816.036678 mapwrite 0x9ca872 thru 0x9cab93 (0x322 bytes) 000240[1] 1747399816.228390 trunc from 0x9cab94 to 0x737640 (0x293553 bytes) 000243[1] 1747399816.266856 mapwrite 0x4ae417 thru 0x4ae7f5 (0x3df bytes) 000244[1] 1747399816.321643 trunc from 0x737641 to 0x61e25e (0x1193e2 bytes) 000245[1] 1747399816.336698 trunc from 0x61e25f to 0x159649 (0x4c4c15 bytes) 000247[0] 1747399816.373423 mapwrite 0x2774ee thru 0x2786be (0x11d1 bytes) 000250[0] 1747399816.522174 read 0x2046f4 thru 0x204706 (0x13 bytes) 000251[0] 1747399816.537828 trunc from 0x2786bf to 0x3809c3 (0x108305 bytes) 000252[0] 1747399816.568141 trunc from 0x3809c4 to 0x3de28e (0x5d8cb bytes) 000253[0] 1747399816.588324 write 0x6077db thru 0x60863c (0xe62 bytes) 000257[1] 1747399816.620238 mapread 0x06006b thru 0x0607b1 (0x747 bytes) 000259[0] 1747399816.663607 mapread 0x3a39d7 thru 0x3a3b1c (0x146 bytes) 000260[1] 1747399816.708438 write 0x55b8c3 thru 0x55c3ba (0xaf8 bytes) 000261[1] 1747399816.737601 mapread 0x3c1d97 thru 0x3c2eed (0x1157 bytes) 000262[0] 1747399816.753196 mapwrite 0x125d7d thru 0x126f79 (0x11fd bytes) 000263[0] 1747399816.828964 trunc from 0x60863d to 0x512f6b (0xf56d1 bytes) 000264[1] 1747399816.868164 read 0x08371a thru 0x08397d (0x264 bytes) 000266[0] 1747399816.893442 trunc from 0x512f6c to 0x0e6d3c (0x42c22f bytes) 000267[1] 1747399816.915269 read 0x058d43 thru 0x059532 (0x7f0 bytes) 000268[1] 1747399816.934720 write 0x24780a thru 0x24950f (0x1d06 bytes) 000269[1] 1747399816.944553 mapread 0x0572d0 thru 0x057b85 (0x8b6 bytes) 000271[1] 1747399816.984126 trunc from 0x249510 to 0x1ae211 (0x9b2fe bytes) 000272[0] 1747399817.026165 write 0x30317c thru 0x305057 (0x1edc bytes) 000273[1] 1747399817.050504 mapwrite 0x709f36 thru 0x70b0eb (0x11b6 bytes) 000274[0] 1747399817.182592 write 0x3d6603 thru 0x3d8439 (0x1e37 bytes) 000276[1] 1747399817.240921 read 0x4b556d thru 0x4b56a6 (0x13a bytes) 000277[0] 1747399817.252638 write 0x86a655 thru 0x86b993 (0x133f bytes) 000278[1] 1747399817.265287 mapwrite 0x7a69c5 thru 0x7a75c8 (0xc04 bytes) 000279[1] 1747399817.368599 trunc from 0x86b994 to 0x1bf86f (0x6ac124 bytes) 000280[0] 1747399817.404194 mapwrite 0x3b7b68 thru 0x3b97b1 (0x1c4a bytes) 000282[1] 1747399817.519402 write 0x6ed8ee thru 0x6ee0f4 (0x807 bytes) 000283[1] 1747399817.542539 trunc from 0x6ee0f5 to 0x8e09ab (0x1f28b7 bytes) 000285[1] 1747399817.573599 trunc from 0x8e09ac to 0x133422 (0x7ad589 bytes) 000286[1] 1747399817.607806 mapread 0x01f1f1 thru 0x02068c (0x149c bytes) 000287[0] 1747399817.625285 write 0x075e04 thru 0x0767c0 (0x9bd bytes) 000289[1] 1747399817.646823 trunc from 0x133423 to 0x95a7c9 (0x8273a7 bytes) 000290[1] 1747399817.668526 mapwrite 0x16d59d thru 0x16e52c (0xf90 bytes) 000293[0] 1747399817.753341 trunc from 0x95a7ca to 0x3695a0 (0x5f1229 bytes) 000294[1] 1747399817.791543 mapread 0x282178 thru 0x282b51 (0x9da bytes) 000295[1] 1747399817.820874 write 0x199083 thru 0x19ae07 (0x1d85 bytes) 000296[1] 1747399817.854488 read 0x0deb4a thru 0x0e0259 (0x1710 bytes) 000297[0] 1747399817.889094 mapread 0x0ceb7a thru 0x0cec6a (0xf1 bytes) 000299[0] 1747399817.899604 write 0x45926e thru 0x459c5e (0x9f1 bytes) 000300[1] 1747399817.922746 write 0x2b1406 thru 0x2b188c (0x487 bytes) 000302[1] 1747399817.952528 mapread 0x2db34f thru 0x2dbf69 (0xc1b bytes) 000303[1] 1747399817.960597 read 0x057e51 thru 0x0586f1 (0x8a1 bytes) 000304[1] 1747399817.995349 read 0x18f455 thru 0x190b45 (0x16f1 bytes) 000305[1] 1747399817.996268 read 0x06ab99 thru 0x06bb8b (0xff3 bytes) 000306[0] 1747399817.996577 trunc from 0x459c5f to 0x7cee05 (0x3751a7 bytes) 000309[0] 1747399818.053134 write 0x9d92ea thru 0x9da0af (0xdc6 bytes) 000312[0] 1747399818.078598 write 0x6e67bc thru 0x6e6ef6 (0x73b bytes) 000315[1] 1747399818.131527 write 0x9005ef thru 0x9016d2 (0x10e4 bytes) 000316[0] 1747399818.159151 write 0x2f6ea5 thru 0x2f7d26 (0xe82 bytes) 000322[1] 1747399818.222022 write 0x5404d7 thru 0x541c7b (0x17a5 bytes) 000323[1] 1747399818.238922 mapwrite 0x442500 thru 0x44286c (0x36d bytes) 000324[0] 1747399818.324563 write 0x4c4638 thru 0x4c6581 (0x1f4a bytes) 000326[0] 1747399818.359827 read 0x35655a thru 0x357a8f (0x1536 bytes) 000327[0] 1747399818.433748 write 0x8cc64f thru 0x8cde2b (0x17dd bytes) 000329[1] 1747399818.434515 mapread 0x2430b5 thru 0x24330b (0x257 bytes) 000330[1] 1747399818.463225 read 0x622a15 thru 0x6240bc (0x16a8 bytes) 000331[0] 1747399818.497762 trunc from 0x9da0b0 to 0x03b37b (0x99ed34 bytes) 000332[1] 1747399818.555388 read 0x0019e0 thru 0x0029aa (0xfcb bytes) 000333[0] 1747399818.585641 write 0x5194f3 thru 0x51a57e (0x108c bytes) 000336[0] 1747399818.595745 mapwrite 0x3ed4f6 thru 0x3ee5df (0x10ea bytes) 000337[0] 1747399818.662168 write 0x77b731 thru 0x77c493 (0xd63 bytes) 000338[1] 1747399818.664717 trunc from 0x77c494 to 0x222bca (0x5598c9 bytes) 000340[1] 1747399818.739595 trunc from 0x222bcb to 0x43484e (0x211c84 bytes) 000345[0] 1747399818.760608 read 0x14dd91 thru 0x14de36 (0xa6 bytes) 000346[1] 1747399818.788344 mapread 0x2b93d7 thru 0x2ba232 (0xe5c bytes) 000348[1] 1747399818.837887 mapwrite 0x1042c0 thru 0x104cfa (0xa3b bytes) 000355[0] 1747399818.963887 mapread 0x235079 thru 0x236cbb (0x1c43 bytes) 000362[0] 1747399819.045841 write 0x446a4a thru 0x447f64 (0x151b bytes) 000363[0] 1747399819.068357 write 0x6433d3 thru 0x643fbf (0xbed bytes) 000365[1] 1747399819.092694 mapread 0x0bc6b7 thru 0x0bd81e (0x1168 bytes) 000366[0] 1747399819.134307 mapwrite 0x1e0831 thru 0x1e200a (0x17da bytes) 000367[1] 1747399819.208820 mapwrite 0x3f2528 thru 0x3f451d (0x1ff6 bytes) 000368[1] 1747399819.311434 mapwrite 0x28fc47 thru 0x291098 (0x1452 bytes) 000370[1] 1747399819.378441 trunc from 0x643fc0 to 0x52cf26 (0x117099 bytes) 000371[0] 1747399819.391768 mapread 0x28fa5b thru 0x291834 (0x1dda bytes) 000372[1] 1747399819.430042 mapwrite 0x8b4888 thru 0x8b5664 (0xddd bytes) 000374[0] 1747399819.522038 read 0x0411c0 thru 0x0419fa (0x83b bytes) 000375[1] 1747399819.551385 write 0x1f3ded thru 0x1f4002 (0x216 bytes) 000376[1] 1747399819.558874 trunc from 0x8b5665 to 0x658cad (0x25c9b7 bytes) 000380[1] 1747399819.610746 trunc from 0x658cae to 0x537735 (0x121578 bytes) 000381[0] 1747399819.636264 mapread 0x47637a thru 0x477264 (0xeeb bytes) 000384[0] 1747399819.686294 read 0x247a8a thru 0x2490a3 (0x161a bytes) 000393[1] 1747399819.764459 read 0x2e2782 thru 0x2e325b (0xada bytes) 000395[0] 1747399819.797673 trunc from 0x537736 to 0x714756 (0x1dd021 bytes) 000396[0] 1747399819.863434 write 0x79e81c thru 0x79f253 (0xa38 bytes) 000397[0] 1747399819.867764 mapread 0x562ba5 thru 0x563707 (0xb63 bytes) 000398[1] 1747399819.906563 mapread 0x0ecb5a thru 0x0edef4 (0x139b bytes) 000399[0] 1747399819.935144 mapwrite 0x024765 thru 0x024e64 (0x700 bytes) 000401[0] 1747399820.029820 read 0x2f7601 thru 0x2f9320 (0x1d20 bytes) 000402[1] 1747399820.070243 write 0x853b46 thru 0x854e72 (0x132d bytes) 000403[1] 1747399820.094412 trunc from 0x854e73 to 0x0b704d (0x79de25 bytes) 000408[1] 1747399820.145896 mapread 0x01da36 thru 0x01df19 (0x4e4 bytes) 000409[1] 1747399820.152968 trunc from 0x0b704e to 0x9e0f1a (0x929ecd bytes) 000410[1] 1747399820.190471 trunc from 0x9e0f1b to 0x1ad9f2 (0x833528 bytes) 000412[1] 1747399820.217556 read 0x08e1fe thru 0x08e4fe (0x301 bytes) 000416[1] 1747399820.251661 write 0x90b47e thru 0x90d045 (0x1bc8 bytes) 000419[0] 1747399820.270699 mapwrite 0x34954b thru 0x349bdf (0x695 bytes) 000420[0] 1747399820.353820 trunc from 0x90d046 to 0x71a196 (0x1f2eaf bytes) 000422[1] 1747399820.387527 trunc from 0x71a197 to 0x08ac68 (0x68f52e bytes) 000423[1] 1747399820.440358 mapread 0x027962 thru 0x0290f5 (0x1794 bytes) 000424[0] 1747399820.466126 write 0x22449f thru 0x2260be (0x1c20 bytes) 000426[0] 1747399820.512714 trunc from 0x2260bf to 0x9c9fba (0x7a3efc bytes) 000433[0] 1747399820.564862 mapread 0x18af58 thru 0x18b0e2 (0x18b bytes) 000435[1] 1747399820.601729 trunc from 0x9c9fbb to 0x997b2e (0x3248c bytes) 000437[1] 1747399820.652806 write 0x471e67 thru 0x472a13 (0xbad bytes) 000440[0] 1747399820.724431 mapread 0x6d07cd thru 0x6d0965 (0x199 bytes) 000442[1] 1747399820.759027 trunc from 0x997b2f to 0x97b985 (0x1c1a9 bytes) 000445[0] 1747399820.800492 trunc from 0x97b986 to 0x3e808f (0x5938f6 bytes) 000446[1] 1747399820.850272 mapwrite 0x619205 thru 0x61ad5c (0x1b58 bytes) 000447[0] 1747399820.960176 read 0x157b63 thru 0x159245 (0x16e3 bytes) 000448[1] 1747399821.003553 trunc from 0x61ad5d to 0x613c6b (0x70f1 bytes) 000449[1] 1747399821.043937 mapread 0x18393f thru 0x184942 (0x1004 bytes) 000451[1] 1747399821.076677 read 0x0e736b thru 0x0e7464 (0xfa bytes) 000453[1] 1747399821.177015 mapread 0x5ac602 thru 0x5ae055 (0x1a54 bytes) 000454[0] 1747399821.187661 read 0x5b0cc2 thru 0x5b1e9e (0x11dd bytes) 000456[0] 1747399821.230352 mapread 0x4b7fc6 thru 0x4b9595 (0x15d0 bytes) 000457[1] 1747399821.264677 read 0x31b928 thru 0x31bfd7 (0x6b0 bytes) 000458[1] 1747399821.287927 read 0x42da75 thru 0x42dad8 (0x64 bytes) 000459[1] 1747399821.314895 mapread 0x156720 thru 0x157a66 (0x1347 bytes) 000461[0] 1747399821.325039 trunc from 0x613c6c to 0x7419e9 (0x12dd7e bytes) 000462[0] 1747399821.368628 trunc from 0x7419ea to 0x22c4a1 (0x515548 bytes) 000463[1] 1747399821.403576 read 0x00a4ad thru 0x00be6a (0x19be bytes) 000464[1] 1747399821.451890 mapwrite 0x615d45 thru 0x616748 (0xa04 bytes) 000468[0] 1747399821.609497 mapwrite 0x34b13a thru 0x34b412 (0x2d9 bytes) 000469[1] 1747399821.699112 mapread 0x57bebe thru 0x57d21a (0x135d bytes) 000470[1] 1747399821.737518 write 0x8ef9a8 thru 0x8f138c (0x19e5 bytes) 000472[0] 1747399821.743807 write 0x2eb0ac thru 0x2eb6dd (0x632 bytes) 000474[0] 1747399821.793651 mapwrite 0x5854ca thru 0x585dda (0x911 bytes) 000475[0] 1747399821.866834 read 0x1e645f thru 0x1e804d (0x1bef bytes) 000476[1] 1747399821.918340 mapread 0x5e7e8f thru 0x5e8b5e (0xcd0 bytes) 000478[1] 1747399821.973011 read 0x447261 thru 0x447d86 (0xb26 bytes) 000480[1] 1747399822.029995 trunc from 0x8f138d to 0x3d7e9f (0x5194ed bytes) 000482[0] 1747399822.075703 write 0x5dd6e1 thru 0x5df058 (0x1978 bytes) 000483[0] 1747399822.096653 mapwrite 0x1dec5c thru 0x1e0655 (0x19fa bytes) 000484[1] 1747399822.173365 write 0x245c36 thru 0x246701 (0xacc bytes) 000485[1] 1747399822.193447 write 0x9cff58 thru 0x9d160b (0x16b4 bytes) 000487[0] 1747399822.222110 trunc from 0x9d160c to 0x83ef6c (0x19269f bytes) 000488[1] 1747399822.247485 trunc from 0x83ef6d to 0x8cbe19 (0x8cead bytes) 000490[0] 1747399822.288702 trunc from 0x8cbe1a to 0x1a084e (0x72b5cb bytes) 000491[1] 1747399822.317005 mapread 0x0b6730 thru 0x0b7981 (0x1252 bytes) 000494[1] 1747399822.382099 write 0x7638b9 thru 0x763a81 (0x1c9 bytes) 000498[1] 1747399822.437297 read 0x68c4cb thru 0x68e277 (0x1dad bytes) 000499[0] 1747399822.468998 write 0x969c87 thru 0x96a322 (0x69c bytes) 000500[1] 1747399822.491379 mapwrite 0x4adb39 thru 0x4af971 (0x1e39 bytes) All operations completed A-OK! PASS 16b (16s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 08:50:24 (1747399824) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (2s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 08:50:26 (1747399826) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.197042 s, 10.6 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.183417 s, 11.4 MB/s ldlm.namespaces.MGC192.168.203.111@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800a9c1c800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff88012aaf8800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800a9c1c800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff88012aaf8800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800a9c1c800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff88012aaf8800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 8.05388 s, 13.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.73913 s, 60.3 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.16085 s, 90.3 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 3.64991 s, 28.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 6.0605 s, 17.3 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.64704 s, 63.7 MB/s PASS 16d (29s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 08:50:56 (1747399856) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.102246 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0604833 s, 17.3 MB/s PASS 16e (3s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 08:50:59 (1747399859) TIMEOUT PASS 16f (24s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 08:51:23 (1747399883) TIMEOUT PASS 16g (22s) == sanityn test 16i: read after truncate file ============ 08:51:46 (1747399906) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (2s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 08:51:48 (1747399908) fail_loc=0x8000030a PASS 17 (4s) == sanityn test 18: mmap sanity check =========================================================================================== 08:51:53 (1747399913) mmap test1: basic mmap operation (PASS, 0.048843s) mmap test2: MAP_PRIVATE not write back (PASS, 0.03773s) mmap test3: concurrent mmap ops on two nodes (PASS, 8.6892s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.2299s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.067925s) mmap test6: check mmap write/read content on two nodes (PASS, 0.069003s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0314s) PASS 18 (16s) == sanityn test 19: test concurrent uncached read races ========================================================================= 08:52:08 (1747399928) oleg311-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg311-client: oleg311-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 08:52:09 (1747399929) PASS 20 (3s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 08:52:12 (1747399932) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) == sanityn test 23: others should see updated atime while another read============================================================== 08:52:14 (1747399934) now is 1747399935 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6881 new atime is 1747399996 PASS 23 (63s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 08:53:18 (1747399998) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 376 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.1K 368 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 376 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 368 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 7.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 13.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118008 376 117632 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235664 368 235296 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 376 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 368 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118008 376 117632 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre[OST:1] filesystem_summary: 235664 368 235296 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118008 376 117632 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235664 368 235296 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre2 PASS 24a (2s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 08:53:20 (1747400000) PASS 24b (2s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 08:53:22 (1747400002) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (2s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 08:53:24 (1747400004) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 08:53:25 (1747400005) utime: good mknod times 1747400005 <= 1747400006 <= 1747400006 for /mnt/lustre/f26a utime: good mknod times 1747400005 <= 1747400006 <= 1747400006 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (4s) == sanityn test 26b: sync mtime between ost and mds ====== 08:53:29 (1747400009) PASS 26b (4s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 08:53:35 (1747400015) 0 PASS 30 (5s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 08:53:40 (1747400020) fail_loc=0x314 PASS 31a (3s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 08:53:43 (1747400023) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg311-server mds-ost sync done. sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.203.111@tcp:/lustre 7542784 13312 7525376 1% /mnt/lustre2 PASS 31b (13s) == sanityn test 31r: open-rename(replace) race =========== 08:53:56 (1747400036) -rw-r--r-- 1 root root 0 May 16 08:53 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (6s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 08:54:03 (1747400043) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 08:54:04 (1747400044) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (2s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 08:54:06 (1747400046) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (2s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 08:54:08 (1747400048) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 08:54:09 (1747400049) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg311-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a9c1c800.ost_server_uuid,osc.lustre-OST0000-osc-ffff88012aaf8800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800a9c1c800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff88012aaf8800.ost_server_uuid in FULL state after 0 sec oleg311-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800a9c1c800.ost_server_uuid,osc.lustre-OST0001-osc-ffff88012aaf8800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800a9c1c800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff88012aaf8800.ost_server_uuid in FULL state after 0 sec oleg311-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a9c1c800.ost_server_uuid,osc.lustre-OST0000-osc-ffff88012aaf8800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800a9c1c800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff88012aaf8800.ost_server_uuid in IDLE state after 0 sec oleg311-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800a9c1c800.ost_server_uuid,osc.lustre-OST0001-osc-ffff88012aaf8800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800a9c1c800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff88012aaf8800.ost_server_uuid in IDLE state after 0 sec oleg311-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a9c1c800.ost_server_uuid,osc.lustre-OST0000-osc-ffff88012aaf8800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800a9c1c800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff88012aaf8800.ost_server_uuid in IDLE state after 0 sec oleg311-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800a9c1c800.ost_server_uuid,osc.lustre-OST0001-osc-ffff88012aaf8800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800a9c1c800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff88012aaf8800.ost_server_uuid in IDLE state after 0 sec PASS 34 (82s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 08:55:31 (1747400131) fail_loc=0x80000317 Race attempt 0 Wait for 15695 15834 for 60 sec... - open/close 1618 (time 1747400143.81 total 10.00 last 161.72) - open/close 3519 (time 1747400153.81 total 20.01 last 190.01) total: 4000 open/close in 22.83 seconds: 175.23 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.203.111@tcp:/lustre 7.2G 13M 7.2G 1% /mnt/lustre 192.168.203.111@tcp:/lustre 7.2G 13M 7.2G 1% /mnt/lustre2 PASS 35 (64s) == sanityn test 36: handle ESTALE/open-unlink correctly == 08:56:36 (1747400196) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg311-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.303638 s, 173 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6881 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.305406 s, 172 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6881 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.340057 s, 154 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6881 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.312796 s, 168 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6881 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.3442 s, 152 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6881 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7525376) after_dd(7470080) after(7521280) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.337693 s, 155 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6881 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7521280) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.29894 s, 175 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6881 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7523328) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.311173 s, 168 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6881 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.308773 s, 170 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6881 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.287524 s, 182 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6881 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.293423 s, 179 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6881 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7525376) after_dd(7474176) after(7525376) PASS 36 (229s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 09:00:24 (1747400424) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6881 - create 6633 (time 1747400436.05 total 10.00 last 663.23) total: 10000 create in 15.13 seconds: 660.89 ops/second PASS 37 (18s) == sanityn test 39a: test from 11063 ============================================================================================ 09:00:42 (1747400442) repeat after cancel_lru_locks PASS 39a (2s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 09:00:44 (1747400444) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 09:00:47 (1747400447) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 09:00:50 (1747400450) fail_loc=0x411 fail_loc=0 PASS 39d (1s) == sanityn test 40a: pdirops: create vs others ======================================================================== 09:00:51 (1747400451) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (8s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 09:00:59 (1747400459) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (7s) == sanityn test 40c: pdirops: link and others ======================================================================== 09:01:06 (1747400466) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (7s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 09:01:13 (1747400473) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (7s) == sanityn test 40e: pdirops: rename and others ======================================================================== 09:01:20 (1747400480) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (7s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 09:01:27 (1747400487) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (5s) == sanityn test 41b: pdirops: create vs create ======================================================================== 09:01:32 (1747400492) open: File exists Conflict PASS 41b (4s) == sanityn test 41c: pdirops: create vs link ======================================================================== 09:01:36 (1747400496) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (5s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 09:01:41 (1747400501) Conflict PASS 41d (5s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 09:01:46 (1747400506) Conflict PASS 41e (4s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 09:01:50 (1747400510) Conflict PASS 41f (5s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 09:01:55 (1747400515) Conflict PASS 41g (4s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 09:02:00 (1747400520) Conflict PASS 41h (4s) == sanityn test 41i: reint_open: create vs create ======== 09:02:04 (1747400524) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (666s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 09:13:10 (1747401190) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (4s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 09:13:14 (1747401194) open: File exists Conflict PASS 42b (5s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 09:13:19 (1747401199) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (4s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 09:13:23 (1747401203) Conflict PASS 42d (4s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 09:13:27 (1747401207) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (4s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 09:13:31 (1747401211) Conflict PASS 42f (5s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 09:13:36 (1747401216) Conflict PASS 42g (4s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 09:13:40 (1747401220) Conflict PASS 42h (4s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 09:13:44 (1747401224) PASS 43a (11s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 09:13:55 (1747401235) Conflict PASS 43b (4s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 09:13:59 (1747401239) Conflict PASS 43c (4s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 09:14:03 (1747401243) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (4s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 09:14:08 (1747401248) Conflict PASS 43e (4s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 09:14:12 (1747401252) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (4s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 09:14:16 (1747401256) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (4s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 09:14:20 (1747401260) Conflict PASS 43h (5s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 09:14:25 (1747401265) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (0s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 09:14:25 (1747401265) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (33s) == sanityn test 43k: unlink vs create ==================== 09:14:58 (1747401298) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (354s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 09:20:52 (1747401652) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Fri May 16 09:20:55 EDT 2025 PASS 44a (4s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 09:20:56 (1747401656) open: File exists Conflict PASS 44b (4s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 09:21:00 (1747401660) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (5s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 09:21:05 (1747401665) Conflict PASS 44d (4s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 09:21:09 (1747401669) Conflict PASS 44e (4s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 09:21:13 (1747401673) Conflict PASS 44f (4s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 09:21:17 (1747401677) Conflict PASS 44g (4s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 09:21:21 (1747401681) Conflict PASS 44h (4s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 09:21:26 (1747401686) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (0s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 09:21:26 (1747401686) PASS 45a (19s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 09:21:45 (1747401705) Conflict PASS 45b (4s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 09:21:49 (1747401709) Conflict PASS 45c (5s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 09:21:54 (1747401714) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (4s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 09:21:58 (1747401718) Conflict PASS 45e (5s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 09:22:03 (1747401723) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (4s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 09:22:07 (1747401727) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (4s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 09:22:11 (1747401731) Conflict PASS 45h (4s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 09:22:15 (1747401735) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 09:22:16 (1747401736) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (352s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 09:28:08 (1747402088) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (4s) == sanityn test 46b: pdirops: link vs create ======================================================================== 09:28:12 (1747402092) open: File exists Conflict PASS 46b (5s) == sanityn test 46c: pdirops: link vs link ======================================================================== 09:28:17 (1747402097) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (4s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 09:28:21 (1747402101) Conflict PASS 46d (4s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 09:28:25 (1747402105) Conflict PASS 46e (4s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 09:28:29 (1747402109) Conflict PASS 46f (4s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 09:28:33 (1747402113) Conflict PASS 46g (5s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 09:28:38 (1747402118) Conflict PASS 46h (4s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 09:28:42 (1747402122) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 09:28:43 (1747402123) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (0s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 09:28:43 (1747402123) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 09:28:44 (1747402124) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 09:28:45 (1747402125) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (0s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 09:28:46 (1747402126) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 09:28:46 (1747402126) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 09:28:47 (1747402127) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 09:28:48 (1747402128) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.00442982 s, 2.3 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (6s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 09:28:54 (1747402134) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00337971 s, 47.0 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (24496) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 09:28:58 (1747402138) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00263015 s, 389 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115309 s, 90.9 MB/s 3145728 PASS 51b (13s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 09:29:11 (1747402151) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00321069 s, 319 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 09:29:17 (1747402157) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00792128 s, 132 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (3s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 09:29:20 (1747402160) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2438 0x986 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2438 0x986 0 read lease(1) has applied. PASS 51e (3s) == sanityn test 54: rename locking ======================= 09:29:23 (1747402163) llite.lustre-ffff8800a9c1c800.xattr_cache=1 llite.lustre-ffff88012aaf8800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504963 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:29:23.000000000 -0400 Modify: 2025-05-16 09:29:23.000000000 -0400 Change: 2025-05-16 09:29:23.000000000 -0400 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504966 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:29:29.000000000 -0400 Modify: 2025-05-16 09:29:29.000000000 -0400 Change: 2025-05-16 09:29:29.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504969 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:29:34.000000000 -0400 Modify: 2025-05-16 09:29:34.000000000 -0400 Change: 2025-05-16 09:29:34.000000000 -0400 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:29:34.000000000 -0400 Modify: 2025-05-16 09:29:34.000000000 -0400 Change: 2025-05-16 09:29:34.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:29:40.000000000 -0400 Modify: 2025-05-16 09:29:40.000000000 -0400 Change: 2025-05-16 09:29:40.000000000 -0400 Birth: - File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504972 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:29:40.000000000 -0400 Modify: 2025-05-16 09:29:40.000000000 -0400 Change: 2025-05-16 09:29:40.000000000 -0400 Birth: - PASS 54 (23s) == sanityn test 55a: rename vs unlink target dir ========= 09:29:46 (1747402186) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 09:29:53 (1747402193) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 09:30:00 (1747402200) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 09:30:12 (1747402212) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) == sanityn test 60: Verify data_version behaviour ======== 09:30:21 (1747402221) PASS 60 (1s) == sanityn test 70a: cd directory && rm directory ======== 09:30:22 (1747402222) PASS 70a (2s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 09:30:24 (1747402224) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 09:30:26 (1747402226) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.0023767 s, 27.6 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (0s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 09:30:26 (1747402226) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00340247 s, 19.3 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 09:30:27 (1747402227) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 09:30:28 (1747402228) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00292749 s, 22.4 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 09:30:29 (1747402229) llite.lustre-ffff8800a9c1c800.xattr_cache=1 llite.lustre-ffff88012aaf8800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (1s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 09:30:30 (1747402230) llite.lustre-ffff8800a9c1c800.xattr_cache=1 llite.lustre-ffff88012aaf8800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (2s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 09:30:32 (1747402232) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00211131 s, 485 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00221024 s, 463 kB/s 3156: get lock2 3157: get lock1 3156: done 3156: get lock1 3157: done 3157: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 3157: cannot get lock2: Resource deadlock avoided 3157: exit rc=0 fcntl cmd 7 failed: Resource deadlock avoided 3156: cannot get lock1: Resource deadlock avoided 3156: done 3156: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 09:30:38 (1747402238) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0126481 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0074936 s, 140 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 3791 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (6s) == sanityn test 76: Verify MDT open_files listing ======== 09:30:44 (1747402244) opening files: ........ closing files: ........ PASS 76 (13s) == sanityn test 77a: check FIFO NRS policy =============== 09:30:57 (1747402257) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0730508 s, 230 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00936206 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121274 s, 86.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010283 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112807 s, 93.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278911 s, 37.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281487 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331159 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239266 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235064 s, 44.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207059 s, 50.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023186 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240566 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285101 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030821 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286467 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0424276 s, 24.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012aaf8800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff88012aaf8800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00885305 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00159899 s, 656 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00938422 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00938131 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109974 s, 95.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00903598 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102184 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00677168 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112952 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00190658 s, 550 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00881014 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00888273 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143211 s, 73.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0101009 s, 104 MB/s , 0.0106993 s, 98.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0112275 s, 93.4 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg311-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) == sanityn test 77b: check CRR-N NRS policy ============== 09:30:59 (1747402259) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0897286 s, 187 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109012 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114671 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122589 s, 85.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168112 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01602 s, 65.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122861 s, 85.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250862 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327796 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00507072 s, 207 MB/s 1048576 bytes (1.0 MB) copied, 0.0224796 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284268 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135801 s, 77.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255389 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182669 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150701 s, 69.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012aaf8800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201128 s, 52.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00926088 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00660309 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848251 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00710812 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00791511 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100685 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00860398 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00654656 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848734 s, 124 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00800534 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00971643 s, 108 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012aaf8800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00940462 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00995452 s, 105 MB/s 1048576 bytes (1.0 MB) copied, 0.0101545 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106449 s, 98.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106945 s, 98.0 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg311-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0775731 s, 216 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00684305 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101518 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156745 s, 66.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0230596 s, 45.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.02487 s, 42.2 MB/s , 0.0280624 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0287459 s, 36.5 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff88012aaf8800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff88012aaf8800.lock_unused_count=3 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274863 s, 38.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0231346 s, 45.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315745 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360058 s, 29.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0405404 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0373817 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342185 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0362829 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0383223 s, 27.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0070882 s, 148 MB/s , 0.00901501 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107793 s, 97.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972396 s, 108 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113063 s, 92.7 MB/s 1048576 bytes (1.0 MB) copied, 0.00880404 s, 119 MB/s 1048576 bytes (1.0 MB) copied, 0.00933058 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00923868 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00932715 s, 112 MB/s 1048576 bytes (1.0 MB) copied, 0.011396 s, 92.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104071 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111362 s, 94.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117196 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119531 s, 87.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112161 s, 93.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012aaf8800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127198 s, 82.4 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg311-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) == sanityn test 77c: check ORR NRS policy ================ 09:31:02 (1747402262) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.116203 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00606859 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170989 s, 61.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.024842 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138054 s, 76.0 MB/s , 0.0182219 s, 57.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019541 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122821 s, 85.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229153 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0298761 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255105 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016466 s, 63.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255552 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209466 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015876 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164906 s, 63.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012aaf8800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178723 s, 58.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00656558 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00665444 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00692615 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126823 s, 82.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00867091 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00912459 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012736 s, 82.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119559 s, 87.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124317 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150794 s, 69.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133156 s, 78.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141962 s, 73.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014628 s, 71.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copiedldlm.namespaces.lustre-OST0000-osc-ffff88012aaf8800.lock_unused_count=2 , 0.0153333 s, 68.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147222 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149374 s, 70.2 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg311-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.110168 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119386 s, 87.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123015 s, 85.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170907 s, 61.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0303292 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324666 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277539 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256987 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0262342 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261475 s, 40.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0274584 s, 38.2 MB/s , 0.0140284 s, 74.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229663 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016687 s, 62.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213291 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151906 s, 69.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012aaf8800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352482 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00342746 s, 306 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00483638 s, 217 MB/s 1048576 bytes (1.0 MB) copied, 0.00265631 s, 395 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0094279 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00763603 s, 137 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00979608 s, 107 MB/s , 0.00896483 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00570853 s, 184 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00631769 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00664329 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00742494 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00772553 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00821248 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00869106 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983229 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00993696 s, 106 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg311-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0942566 s, 178 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00912658 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948139 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261273 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267988 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0329524 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326627 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266673 s, 39.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0347593 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0376589 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0407738 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034192 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0347913 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372561 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0412813 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353764 s, 29.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012aaf8800.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff88012aaf8800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0417827 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00852926 s, 123 MB/s , 0.00845717 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00756342 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00745502 s, 141 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0104644 s, 100 MB/s , 0.0104222 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127751 s, 82.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0089568 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120104 s, 87.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111614 s, 93.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112079 s, 93.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112232 s, 93.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0120477 s, 87.0 MB/s , 0.01216 s, 86.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112483 s, 93.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134021 s, 78.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012aaf8800.lock_unused_count=1 removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg311-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (4s) == sanityn test 77d: check TRR nrs policy ================ 09:31:06 (1747402266) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.104383 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00690335 s, 152 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0152656 s, 68.7 MB/s , 0.0166266 s, 63.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220342 s, 47.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238099 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216361 s, 48.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013173 s, 79.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258588 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290983 s, 36.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251937 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242297 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210618 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263273 s, 39.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0285553 s, 36.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0231711 s, 45.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292311 s, 35.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012aaf8800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00700575 s, 150 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00699073 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00708504 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00888463 s, 118 MB/s 1048576 bytes (1.0 MB) copied, 0.0112751 s, 93.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00937535 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105211 s, 99.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00747473 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102749 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00884697 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119354 s, 87.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00901245 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00921206 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111173 s, 94.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010004 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108769 s, 96.4 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg311-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0845052 s, 199 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111149 s, 94.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116378 s, 90.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142175 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152261 s, 68.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148183 s, 70.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0112468 s, 93.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00638154 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134013 s, 78.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326957 s, 32.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306167 s, 34.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0302784 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180194 s, 58.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137197 s, 76.4 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012aaf8800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210561 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0299947 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229274 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0084917 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00282217 s, 372 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00149738 s, 700 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00852161 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00347033 s, 302 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00473195 s, 222 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101684 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00137976 s, 760 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00997778 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116297 s, 90.2 MB/s , 0.0112156 s, 93.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100252 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00658307 s, 159 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00211696 s, 495 MB/s , 0.0013865 s, 756 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00136671 s, 767 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012aaf8800.lock_unused_count=1 removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg311-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0731771 s, 229 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00647581 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00760923 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124551 s, 84.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023105 s, 45.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0310924 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322876 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228078 s, 46.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0268746 s, 39.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0253956 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0358836 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328122 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246062 s, 42.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012aaf8800.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff88012aaf8800.lock_unused_count=4 1048576 bytes (1.0 MB) copied, 0.0356325 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034844 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034054 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326728 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0076522 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00725163 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00716956 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00835476 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00747127 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0014164 s, 740 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00766305 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00454504 s, 231 MB/s 1048576 bytes (1.0 MB) copied, 0.00709056 s, 148 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00270363 s, 388 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102105 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00346169 s, 303 MB/s , 0.00285934 s, 367 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00998967 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00127309 s, 824 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012aaf8800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119483 s, 87.8 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg311-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (3s) == sanityn test 77e: check TBF NID nrs policy ============ 09:31:09 (1747402269) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.203.11@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.114724 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248134 s, 42.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00775237 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318148 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034447 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00656029 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411211 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324298 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00796266 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207196 s, 50.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449079 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0559599 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316821 s, 33.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0594545 s, 17.6 MB/s , 0.0361903 s, 29.0 MB/s , 0.0617656 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0620118 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103347 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008518 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00912112 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00933424 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0106012 s, 98.9 MB/s 1048576 bytes (1.0 MB) copied, 0.00963961 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134873 s, 77.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106346 s, 98.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110304 s, 95.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114514 s, 91.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0117272 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113146 s, 92.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115498 s, 90.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011866 s, 88.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123451 s, 84.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132108 s, 79.4 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg311-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0755219 s, 222 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00771108 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126653 s, 82.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156652 s, 66.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191201 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253739 s, 41.3 MB/s , 0.0192323 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189727 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228044 s, 46.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321001 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00878342 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289598 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240382 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258224 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106714 s, 98.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00553829 s, 189 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316837 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00801608 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00811187 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00838855 s, 125 MB/s 1048576 bytes (1.0 MB) copied, 0.00833809 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0108668 s, 96.5 MB/s 1048576 bytes (1.0 MB) copied, 0.00848018 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123279 s, 85.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00994164 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00960388 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108132 s, 97.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00979961 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00773536 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105254 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115903 s, 90.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105525 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109857 s, 95.4 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg311-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.11118 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101392 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125228 s, 83.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120319 s, 87.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214111 s, 49.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123739 s, 84.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189347 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194233 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0257296 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315166 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0358419 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395598 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.038453 s, 27.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0427943 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400752 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0426967 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0409711 s, 25.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012aaf8800.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff88012aaf8800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00944752 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00235277 s, 446 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00814189 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00957519 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00950338 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00946946 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00993557 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101705 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103245 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00350088 s, 300 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00552968 s, 190 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115937 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011707 s, 89.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118681 s, 88.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111029 s, 94.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012aaf8800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138587 s, 75.7 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg311-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0894038 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00647385 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00681817 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101614 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0130866 s, 80.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0108168 s, 96.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00923989 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104197 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156362 s, 67.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293204 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129768 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208581 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154007 s, 68.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202449 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270756 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191064 s, 54.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012aaf8800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88012aaf8800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322714 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00822185 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0069495 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00711312 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00696724 s, 151 MB/s , 0.00681286 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00860333 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00203925 s, 514 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00833414 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00495119 s, 212 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00826701 s, 127 MB/s , 0.00794369 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00165343 s, 634 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00959788 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0012813 s, 818 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00179816 s, 583 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0012229 s, 857 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012aaf8800.lock_unused_count=1 removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg311-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (6s) == sanityn test 77f: check TBF JobID nrs policy ========== 09:31:15 (1747402275) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg311-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.130007 s, 129 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00629833 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00761379 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0084072 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106537 s, 98.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0223881 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238574 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026092 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100658 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198189 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206236 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226917 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266236 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0257542 s, 40.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0257475 s, 40.7 MB/s , 0.0270584 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0221514 s, 47.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00969071 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00865329 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00997495 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00913342 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100468 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00935303 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00940499 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121062 s, 86.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00999875 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00960602 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105408 s, 99.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010219 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01018 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00976997 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110985 s, 94.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130072 s, 80.6 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg311-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg311-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0862487 s, 195 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00714977 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115233 s, 91.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162124 s, 64.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208731 s, 50.2 MB/s , 0.0222016 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161754 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184713 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247143 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343744 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.038542 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0337202 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0412011 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0380435 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0412192 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0355438 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389939 s, 26.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00803249 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0107085 s, 97.9 MB/s 1048576 bytes (1.0 MB) copied, 0.00899757 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00893181 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00910171 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116907 s, 89.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00547789 s, 191 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0092879 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113348 s, 92.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116783 s, 89.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00160373 s, 654 MB/s 1048576 bytes (1.0 MB) copied, 0.00957558 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00942682 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106776 s, 98.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106002 s, 98.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108867 s, 96.3 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg311-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg311-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.129865 s, 129 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00740421 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00717664 s, 146 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235667 s, 44.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0239692 s, 43.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171295 s, 61.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217592 s, 48.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0248186 s, 42.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0198129 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148266 s, 70.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0299793 s, 35.0 MB/s , 0.0336704 s, 31.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0406025 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275894 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027181 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324085 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0301982 s, 34.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012aaf8800.lock_unused_count=1 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0079005 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00874105 s, 120 MB/s , 0.00967283 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983673 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115882 s, 90.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102628 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00938214 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00880549 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00884194 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106522 s, 98.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112732 s, 93.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115137 s, 91.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108491 s, 96.7 MB/s , 0.0121436 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120895 s, 86.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137905 s, 76.0 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg311-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg311-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0842455 s, 199 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00962072 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189312 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211399 s, 49.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166002 s, 63.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0221932 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00875306 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279687 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160264 s, 65.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00796564 s, 132 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012aaf8800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327347 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0310689 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0439525 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0469165 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365057 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395578 s, 26.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0582432 s, 18.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0100806 s, 104 MB/s , 0.00853822 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00881137 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00188259 s, 557 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011832 s, 88.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107582 s, 97.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106504 s, 98.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138997 s, 75.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106139 s, 98.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00693375 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016015 s, 65.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011811 s, 88.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00189939 s, 552 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102629 s, 102 MB/s 1048576 bytes (1.0 MB) copied, 0.0128555 s, 81.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127684 s, 82.1 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg311-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (6s) == sanityn test 77g: Change TBF type directly ============ 09:31:21 (1747402281) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (3s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 09:31:24 (1747402284) oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg311-client: oleg311-server: ssh exited with exit code 2 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg311-client: oleg311-server: ssh exited with exit code 12 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg311-client: oleg311-server: ssh exited with exit code 12 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg311-client: oleg311-server: ssh exited with exit code 12 PASS 77h (2s) == sanityn test 77i: Change rank of TBF rule ============= 09:31:26 (1747402286) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (6s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 09:31:32 (1747402292) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0797804 s, 210 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010544 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168751 s, 62.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167839 s, 62.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158597 s, 66.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022776 s, 46.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151509 s, 69.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.013263 s, 79.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157878 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274091 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215816 s, 48.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275894 s, 38.0 MB/s , 0.0246256 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156551 s, 67.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0369385 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0394955 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00715232 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00834639 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00504243 s, 208 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105069 s, 99.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00680274 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00828357 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118477 s, 88.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126933 s, 82.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0079147 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00885301 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00909647 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00937939 s, 112 MB/s , 0.00947688 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138098 s, 75.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0131728 s, 79.6 MB/s , 0.013223 s, 79.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133302 s, 78.7 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg311-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.9189 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4321 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (49s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 09:32:21 (1747402341) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg311-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.642037 s, 26.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166014 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.168816 s, 6.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.164254 s, 6.4 MB/s 1048576 bytes (1.0 MB) copied, 0.163536 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.171005 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.177131 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.172316 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.179021 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.368879 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.37181 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.383794 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.378795 s, 2.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.392502 s, 2.7 MB/s , 0.386432 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.392978 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.398146 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156194 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158407 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156244 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.156375 s, 6.7 MB/s 1048576 bytes (1.0 MB) copied, 0.157406 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159079 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158312 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158038 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158076 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157597 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157874 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157654 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158208 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158104 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1583 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.160038 s, 6.6 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg311-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8301 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9636 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg311-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.63894 s, 26.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.170536 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.168301 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.188499 s, 5.6 MB/s 1048576 bytes (1.0 MB) copied, 0.181509 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.186831 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.192152 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.184803 s, 5.7 MB/s 1048576 bytes (1.0 MB) copied, 0.188705 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.369304 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.374913 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.383055 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.381509 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.387908 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.385023 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.389971 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.39324 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152074 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151084 s, 6.9 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151258 s, 6.9 MB/s , 0.148391 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.149004 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152592 s, 6.9 MB/s , 0.148668 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150364 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15026 s, 7.0 MB/s , 0.150676 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151408 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150569 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152531 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152037 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152586 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153907 s, 6.8 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg311-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8312 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9652 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (130s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 09:34:31 (1747402471) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg311-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.199566 s, 84.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103815 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269696 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273611 s, 38.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107369 s, 97.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249226 s, 42.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0335165 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306795 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206108 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0594629 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0568602 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0693109 s, 15.1 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0497451 s, 21.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0472611 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0682133 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0592691 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0801839 s, 13.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00946616 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983295 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126322 s, 83.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00855802 s, 123 MB/s 1048576 bytes (1.0 MB) copied, 0.0120791 s, 86.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130943 s, 80.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105207 s, 99.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0102726 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112505 s, 93.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122675 s, 85.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117222 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116133 s, 90.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136224 s, 77.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131091 s, 80.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013604 s, 77.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014878 s, 70.5 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg311-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94574 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73569 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.11@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.11@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.197317 s, 85.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157368 s, 66.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177586 s, 59.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0242752 s, 43.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0241818 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250719 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130657 s, 80.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125943 s, 83.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145826 s, 71.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0602982 s, 17.4 MB/s , 0.0689433 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150291 s, 7.0 MB/s , 0.167154 s, 6.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.251253 s, 4.2 MB/s , 0.258174 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.341928 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.364892 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136539 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136314 s, 76.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209136 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164828 s, 63.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171553 s, 61.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175994 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180023 s, 58.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170191 s, 61.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022604 s, 46.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0190506 s, 55.0 MB/s , 0.0193594 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184747 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196113 s, 53.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195683 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195297 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202312 s, 51.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg311-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94354 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74305 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.11@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg311-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.185915 s, 90.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130109 s, 80.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283564 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124728 s, 84.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229927 s, 45.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315097 s, 33.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199426 s, 52.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0343468 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333556 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0707387 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0675805 s, 15.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0811607 s, 12.9 MB/s , 0.0684986 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0709862 s, 14.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0740366 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0844725 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0712902 s, 14.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00818568 s, 128 MB/s 1048576 bytes (1.0 MB) copied, 0.00852461 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00845634 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00632954 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106073 s, 98.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00967011 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0047215 s, 222 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00707385 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00920105 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00859913 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00861305 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00920616 s, 114 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105436 s, 99.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107706 s, 97.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0110551 s, 94.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142727 s, 73.5 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg311-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93265 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.89001 s, 21.4 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg311-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.36145 s, 46.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222633 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188363 s, 55.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236585 s, 44.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0280887 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0362452 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0472307 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0313273 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0516711 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14989 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.17056 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.162981 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159413 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.258512 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.242521 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.284908 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.287986 s, 3.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153066 s, 68.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014449 s, 72.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0158036 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0192898 s, 54.4 MB/s , 0.0161519 s, 64.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214105 s, 49.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173633 s, 60.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0139216 s, 75.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0196267 s, 53.4 MB/s , 0.00302697 s, 346 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157614 s, 66.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178398 s, 58.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188155 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186992 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00529454 s, 198 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197184 s, 53.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg311-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.82057 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73831 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.9217 s, 21.3 MB/s Write runtime is 5 s, speed is 20.000000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7441 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg311-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.656771 s, 25.5 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193378 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187934 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174423 s, 60.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297357 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220034 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212312 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141503 s, 74.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00944184 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.345428 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.35007 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.364714 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.354019 s, 3.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.36029 s, 2.9 MB/s , 0.379322 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.389637 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.394709 s, 2.7 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00935665 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00663671 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106441 s, 98.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0094138 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125028 s, 83.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100651 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00842489 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107806 s, 97.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0093278 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968641 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104381 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107097 s, 97.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114767 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116994 s, 89.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121221 s, 86.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136666 s, 76.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg311-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8407 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4424 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg311-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.188081 s, 89.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00646464 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0081822 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109909 s, 95.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193351 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0195331 s, 53.7 MB/s 1048576 bytes (1.0 MB) copied, 0.00692777 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202349 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00837031 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0696026 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.078295 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0651857 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0870214 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0733012 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0762602 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0806335 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0798537 s, 13.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00872018 s, 120 MB/s , 0.00801299 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114319 s, 91.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106489 s, 98.5 MB/s 1048576 bytes (1.0 MB) copied, 0.00975845 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00834985 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.007344 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111335 s, 94.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00158807 s, 660 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00938915 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00190805 s, 550 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00982183 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114357 s, 91.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108481 s, 96.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125959 s, 83.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00163719 s, 640 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg311-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94088 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73056 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg311-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.356004 s, 47.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105619 s, 99.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134674 s, 77.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216886 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217531 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00817087 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151454 s, 69.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179794 s, 58.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00710816 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148892 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15981 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.167711 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149722 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.329695 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.342464 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.353066 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.353185 s, 3.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112636 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0021138 s, 496 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111562 s, 94.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107898 s, 97.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00310163 s, 338 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115142 s, 91.1 MB/s 1+0 records in 1+0 records out , 0.0115157 s, 91.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0114079 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128679 s, 81.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136429 s, 76.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154021 s, 68.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122966 s, 85.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123045 s, 85.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126422 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0138736 s, 75.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0154337 s, 67.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg311-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77874 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7317 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7608 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72923 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (357s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 09:40:28 (1747402828) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (3s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 09:40:31 (1747402831) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg311-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.35578 s, 312 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg311-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.015 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 09:41:21 (1747402881) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0879872 s, 191 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121877 s, 86.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139271 s, 75.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0276941 s, 37.9 MB/s , 0.0259944 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0336963 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364384 s, 28.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378791 s, 27.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0338493 s, 31.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392551 s, 26.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0430977 s, 24.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0313013 s, 33.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0463692 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0462344 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457172 s, 22.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012aaf8800.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff88012aaf8800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398687 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0456652 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102404 s, 102 MB/s , 0.0103301 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140284 s, 74.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105911 s, 99.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0098525 s, 106 MB/s 1048576 bytes (1.0 MB) copied, 0.00960855 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125126 s, 83.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100183 s, 105 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012aaf8800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117107 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150958 s, 69.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125901 s, 83.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127314 s, 82.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134747 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122729 s, 85.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147368 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138362 s, 75.8 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg311-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93569 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96787 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.153953 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197381 s, 53.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154331 s, 67.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0180055 s, 58.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202761 s, 51.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227452 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017462 s, 60.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136684 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125932 s, 83.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0539299 s, 19.4 MB/s , 0.063098 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142289 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.141189 s, 7.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.25898 s, 4.0 MB/s , 0.262116 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.338845 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.340071 s, 3.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.010387 s, 101 MB/s 1+0 records in 1+0 records out , 0.0105128 s, 99.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0106522 s, 98.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180454 s, 58.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157345 s, 66.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149584 s, 70.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116271 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169789 s, 61.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125324 s, 83.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106479 s, 98.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191976 s, 54.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155023 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151525 s, 69.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132845 s, 78.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00229869 s, 456 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184819 s, 56.7 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg311-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg311-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92946 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.9671 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (67s) == sanityn test 77o: Changing rank should not panic ====== 09:42:28 (1747402948) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (4s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 09:42:32 (1747402952) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (26s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 09:42:58 (1747402978) mds.MDS.mdt.nrs_policies=tbf oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg311-client: oleg311-server: ssh exited with exit code 36 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg311-client: oleg311-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (9s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 09:43:07 (1747402987) oleg311-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device pdsh@oleg311-client: oleg311-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (3s) == sanityn test 79: xattr: intent error ================== 09:43:10 (1747402990) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505431 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:43:10.000000000 -0400 Modify: 2025-05-16 09:43:10.000000000 -0400 Change: 2025-05-16 09:43:11.000000000 -0400 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (13s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 09:43:23 (1747403003) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (0s) == sanityn test 80b: Accessing directory during migration ========================================================== 09:43:23 (1747403003) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 09:43:24 (1747403004) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 09:43:25 (1747403005) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 09:43:26 (1747403006) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 09:43:27 (1747403007) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6881 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6881 PASS 82 (2s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 09:43:29 (1747403009) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 09:43:30 (1747403010) fail_loc=0x8000060b PASS 84 (7s) == sanityn test 90: open/create and unlink striped directory ========================================================== 09:43:37 (1747403017) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 09:43:38 (1747403018) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 09:43:39 (1747403019) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 09:43:40 (1747403020) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2647 0xa57 0 1 2627 0xa43 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2628 0xa44 0 0 2648 0xa58 0 rc1=0 and rc2=0 PASS 93 (8s) == sanityn test 94: signal vs CP callback race =========== 09:43:48 (1747403028) osc.lustre-OST0000-osc-ffff8800a9c1c800.idle_timeout=0 osc.lustre-OST0000-osc-ffff88012aaf8800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800a9c1c800.idle_timeout=0 osc.lustre-OST0001-osc-ffff88012aaf8800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0272555 s, 301 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 3.00245 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 4513 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0210053 s, 195 kB/s osc.lustre-OST0000-osc-ffff8800a9c1c800.idle_timeout=debug osc.lustre-OST0000-osc-ffff88012aaf8800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800a9c1c800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012aaf8800.idle_timeout=debug PASS 94 (10s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 09:43:58 (1747403038) llite.lustre-ffff8800a9c1c800.fast_read=0 llite.lustre-ffff88012aaf8800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00184004 s, 6.7 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800a9c1c800.fast_read=1 llite.lustre-ffff88012aaf8800.fast_read=1 PASS 95b (8s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 09:44:06 (1747403046) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 09:44:07 (1747403047) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00278816 s, 1.5 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 09:44:09 (1747403049) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0192672 s, 109 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 09:44:11 (1747403051) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0215013 s, 97.5 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (3s) == sanityn test 100e: DoM: read on open and file size ==== 09:44:14 (1747403054) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00142873 s, 8.4 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 09:44:16 (1747403056) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000864133 s, 4.7 MB/s PASS 101a (2s) == sanityn test 101b: Discard DoM data on rename ========= 09:44:18 (1747403058) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00055454 s, 7.4 MB/s PASS 101b (1s) == sanityn test 101c: Discard DoM data on close-unlink === 09:44:20 (1747403060) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000540059 s, 7.6 MB/s PASS 101c (3s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 09:44:23 (1747403063) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1747403063 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1747403066 Finishing test test23 at 1747403066 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 09:44:30 (1747403070) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 09:44:31 (1747403071) fail_loc=0x416 PASS 105 (47s) == sanityn test 106a: Verify the btime via statx() ======= 09:45:19 (1747403119) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 09:45:20 (1747403120) SKIP: sanityn test_106b statx() only test SKIP 106b (0s) == sanityn test 106c: Verify statx attributes mask ======= 09:45:20 (1747403120) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) == sanityn test 107a: Basic grouplock conflict =========== 09:45:21 (1747403121) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0514581 s, 204 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2633 0xa49 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6881 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6881 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 09:45:25 (1747403125) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0534155 s, 196 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2654 0xa5e 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6881 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) == sanityn test 108a: lseek: parallel updates ============ 09:45:33 (1747403133) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00138686 s, 7.2 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.30816 s, 1.9 MB/s PASS 108a (6s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 09:45:39 (1747403139) 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions oleg311-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 2 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions oleg311-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 3 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions oleg311-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 7 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions oleg311-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 10 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions oleg311-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 21 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions oleg311-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 30 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions oleg311-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 33 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions oleg311-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 41 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions oleg311-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 46 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions oleg311-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg311-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 48 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions oleg311-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 49 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions oleg311-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 50 Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 192.168.203.111@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.111@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg311-client.virtnet /mnt/lustre (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg311-server' oleg311-server: oleg311-server.virtnet: executing load_modules_local oleg311-server: Loading modules from /home/green/git/lustre-release/lustre oleg311-server: detected 4 online CPUs by sysfs oleg311-server: Force libcfs to create 2 CPU partitions Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre Starting client: oleg311-client.virtnet: -o user_xattr,flock oleg311-server@tcp:/lustre /mnt/lustre2 PASS 109 (434s) == sanityn test 110: do not grant another lock on resend ========================================================== 09:52:53 (1747403573) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-16 09:52:54.000000000 -0400 Modify: 2025-05-16 09:52:54.000000000 -0400 Change: 2025-05-16 09:52:54.000000000 -0400 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg311-client.virtnet oleg311-client.virtnet PASS 110 (36s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 09:53:29 (1747403609) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 09:53:30 (1747403610) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (2s) == sanityn test 113: check servers of specified fs ======= 09:53:32 (1747403612) lustre-OST0000-osc-ffff88009f29a800 active. lustre-OST0001-osc-ffff88009f29a800 active. lustre-OST0000-osc-ffff88009f29a800 active. lustre-OST0001-osc-ffff88009f29a800 active. MGC192.168.203.111@tcp active. PASS 113 (2s) cleanup: ====================================================== == sanityn test complete, duration 4106 sec ============== 09:53:35 (1747403615) Stopping clients: oleg311-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg311-client.virtnet /mnt/lustre2 opts: