-----============= acceptance-small: sanityn ============----- Sun Apr 13 05:07:26 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg331-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/sanityn.*ex: No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a === sanityn: start setup 05:07:30 (1744535250) === Starting client oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 Started clients oleg331-client.virtnet: 192.168.203.131@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) oleg331-client.virtnet: executing check_config_client /mnt/lustre oleg331-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg331-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff88012a612800.idle_timeout=debug osc.lustre-OST0000-osc-ffff88012e275800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012a612800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012e275800.idle_timeout=debug disable quota as required oleg331-server: oleg331-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 05:07:38 (1744535258) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119402 s, 87.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7032] == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 05:07:38 (1744535258) PASS 0 (2s) == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 05:07:40 (1744535260) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (2s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 05:07:42 (1744535262) -rw-r--r-- 1 root root 0 Apr 13 05:07 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (1s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 05:07:43 (1744535263) -rw-r--r-- 1 root root 0 Apr 13 05:07 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (2s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 05:07:45 (1744535265) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (1s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 05:07:46 (1744535266) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (2s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 05:07:48 (1744535268) total 1 -rwxrwxrwx 1 root root 0 Apr 13 05:07 f2a -rwxrwxrwx 1 root root 0 Apr 13 05:07 f2b total 1 -rwxrwxrwx 1 root root 0 Apr 13 05:07 f2a -rwxrwxrwx 1 root root 0 Apr 13 05:07 f2b running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00369813 s, 138 kB/s PASS 2e (1s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 05:07:49 (1744535269) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 05:07:50 (1744535270) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0782931 s, 26.8 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (2s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 05:07:52 (1744535272) PASS 3 (1s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 05:07:53 (1744535273) PASS 4 (2s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 05:07:55 (1744535275) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (2s) == sanityn test 6: remove of open file on other node ============================================================================ 05:07:57 (1744535277) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (1s) == sanityn test 7: remove of open directory on other node ======================================================================= 05:07:58 (1744535278) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (2s) == sanityn test 8: remove of open special file on other node ==================================================================== 05:08:00 (1744535280) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (1s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 05:08:01 (1744535281) PASS 9a (2s) == sanityn test 9b: append to striped sparse file ======== 05:08:03 (1744535283) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112363 s, 93.3 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.010417 s, 0.3 kB/s Data read (expecting 'foo'): foo PASS 9b (1s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 05:08:04 (1744535284) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00145664 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00247948 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00216171 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00351363 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0028569 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0036456 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00208433 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00211347 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00243401 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00238225 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0023915 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00377005 s, 0.3 kB/s PASS 10a (2s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 05:08:06 (1744535286) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00218714 s, 1.4 MB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00612938 s, 668 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000201138 s, 15.3 MB/s PASS 10b (1s) == sanityn test 11: execution of file opened for write should return error ============================================================== 05:08:07 (1744535287) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 301: /mnt/lustre2/d11/f: Text file busy PASS 11 (2s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 05:08:09 (1744535289) warning: '-runtime' deprecated, use '-t runtime' instead - link 3053 (time 1744535299.77 total 10.00 last 305.25) - link 5948 (time 1744535309.77 total 20.00 last 289.43) - link 9100 (time 1744535319.78 total 30.00 last 315.19) - link 10000 (time 1744535322.74 total 32.97 last 303.21) - link 13250 (time 1744535332.74 total 42.97 last 324.98) - link 16280 (time 1744535342.75 total 52.97 last 302.95) - link 19448 (time 1744535352.75 total 62.98 last 316.76) - link 22403 (time 1744535362.75 total 72.98 last 295.41) - link 25307 (time 1744535372.75 total 82.98 last 290.32) - link 28385 (time 1744535382.75 total 92.98 last 307.76) - link 30000 (time 1744535388.71 total 98.94 last 271.02) total: 30332 link in 100.00 seconds: 303.31 ops/second using seed 3403660744 running for 100 seconds - stat 10000 (time 1744535295 ; total 5 ; last 5) - stat 20000 (time 1744535300 ; total 10 ; last 5) - stat 30000 (time 1744535305 ; total 15 ; last 5) - stat 40000 (time 1744535310 ; total 20 ; last 5) - stat 50000 (time 1744535314 ; total 24 ; last 4) - stat 60000 (time 1744535318 ; total 28 ; last 4) - stat 70000 (time 1744535323 ; total 33 ; last 5) - stat 80000 (time 1744535327 ; total 37 ; last 4) - stat 90000 (time 1744535332 ; total 42 ; last 5) - stat 100000 (time 1744535336 ; total 46 ; last 4) - stat 110000 (time 1744535342 ; total 52 ; last 6) - stat 119221 (time 1744535353 ; total 63 ; last 11) - stat 120000 (time 1744535353 ; total 63 ; last 0) - stat 128621 (time 1744535364 ; total 74 ; last 11) - stat 130000 (time 1744535365 ; total 75 ; last 1) - stat 138627 (time 1744535376 ; total 86 ; last 11) - stat 140000 (time 1744535377 ; total 87 ; last 1) - stat 148710 (time 1744535388 ; total 98 ; last 11) - stat 150000 (time 1744535389 ; total 99 ; last 1) total: 150769 stats in 100 seconds: 1507.689941 stats/second - unlinked 0 (time 1744535340 ; total 0 ; last 0) - unlinked 10000 (time 1744535391 ; total 51 ; last 51) - unlinked 20000 (time 1744535416 ; total 76 ; last 25) - unlinked 30000 (time 1744535440 ; total 100 ; last 24) unlink(/mnt/lustre2/lockdir/lockfile30332) error: No such file or directory total: 30331 unlinks in 100 seconds: 303.309998 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (22039) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (22042) - No such process PASS 12 (153s) == sanityn test 13: test directory page revocation ======= 05:10:42 (1744535442) f13 f13 PASS 13 (1s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 05:10:43 (1744535443) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 336: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4880: 23439 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 05:10:45 (1744535445) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4880: 24073 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (1s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 05:10:46 (1744535446) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 351: 24705 Terminated $DIR1/$tdir/sleep 60 PASS 14b (2s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 05:10:48 (1744535448) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 363: 25340 Terminated $DIR1/$tdir/sleep 60 PASS 14c (1s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 05:10:49 (1744535449) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 374: 25974 Terminated $DIR1/$tdir/sleep 60 PASS 14d (2s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 05:10:51 (1744535451) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_CODE=117440512 CLIENT_OS_VERSION_ID=7 STRIPECOUNT=2 ORIGFREE=7523328 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7523328kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (12s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 05:11:03 (1744535463) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.114809 s, 73.1 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.0909428 s, 92.2 MB/s Chance of close/open is 1 in 50 Seed set to 5464 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000004[0] 1744535464.250068 mapwrite 0x001377 thru 0x00be6a (0xaaf4 bytes) 000007[1] 1744535464.291982 mapwrite 0x43bc9f thru 0x4491d0 (0xd532 bytes) 000009[1] 1744535464.324997 write 0x06f4e9 thru 0x07ad19 (0xb831 bytes) 000010[0] 1744535464.327868 read 0x01acf1 thru 0x022a0d (0x7d1d bytes) truncating to largest ever: 0x118c8 000014[0] 1744535464.347201 trunc from 0x4491d1 to 0x0118c7 (0x437909 bytes) truncating to largest ever: 0x3612a0 000015[1] 1744535464.355992 trunc from 0x0118c8 to 0x36129f (0x34f9d8 bytes) 000016[0] 1744535464.368021 read 0x2875ed thru 0x295e97 (0xe8ab bytes) truncating to largest ever: 0x4a6ba9 000017[0] 1744535464.394600 trunc from 0x3612a0 to 0x4a6ba8 (0x145909 bytes) truncating to largest ever: 0x5385ee 000019[1] 1744535464.407993 trunc from 0x4a6ba9 to 0x5385ed (0x91a45 bytes) 000020[1] 1744535464.419668 mapread 0x29e2e2 thru 0x29f793 (0x14b2 bytes) 000021[0] 1744535464.425728 write 0x2a4211 thru 0x2b1315 (0xd105 bytes) 000027[0] 1744535464.456166 write 0x0c884d thru 0x0cb3a8 (0x2b5c bytes) truncating to largest ever: 0x76db53 000029[1] 1744535464.464670 trunc from 0x5385ee to 0x76db52 (0x235565 bytes) 000031[1] 1744535464.478037 trunc from 0x76db53 to 0x43edbe (0x32ed94 bytes) 000032[0] 1744535464.482115 write 0x1d21e7 thru 0x1dad71 (0x8b8b bytes) 000035[1] 1744535464.497184 write 0x377a06 thru 0x380518 (0x8b13 bytes) 000038[0] 1744535464.511582 mapread 0x39f8d2 thru 0x3acd1f (0xd44e bytes) 000041[1] 1744535464.542618 mapwrite 0x56b6a1 thru 0x57730f (0xbc6f bytes) 000043[0] 1744535464.576669 mapread 0x08c170 thru 0x096a72 (0xa903 bytes) 000044[0] 1744535464.591825 write 0x0a7e3f thru 0x0b3bd6 (0xbd98 bytes) 000046[0] 1744535464.602971 mapwrite 0x3d4646 thru 0x3d5af8 (0x14b3 bytes) 000047[1] 1744535464.625287 write 0x853343 thru 0x85c1c0 (0x8e7e bytes) 000048[1] 1744535464.628509 write 0x809fa2 thru 0x80de97 (0x3ef6 bytes) 000053[1] 1744535464.636617 write 0x918715 thru 0x921217 (0x8b03 bytes) 000055[0] 1744535464.638661 mapwrite 0x3e9c5e thru 0x3f000b (0x63ae bytes) 000057[1] 1744535464.672736 write 0x2a62f9 thru 0x2a8209 (0x1f11 bytes) 000060[1] 1744535464.686035 write 0x659bdc thru 0x660d7b (0x71a0 bytes) 000061[1] 1744535464.688586 write 0x5924af thru 0x5954e2 (0x3034 bytes) 000065[0] 1744535464.700815 mapread 0x224d81 thru 0x22d367 (0x85e7 bytes) 000066[1] 1744535464.719969 mapwrite 0x683ad1 thru 0x68a384 (0x68b4 bytes) 000068[0] 1744535464.749719 mapread 0x916db3 thru 0x921217 (0xa465 bytes) 000070[0] 1744535464.769650 mapread 0x652adb thru 0x653e86 (0x13ac bytes) 000072[0] 1744535464.790247 read 0x8b7434 thru 0x8c0b3a (0x9707 bytes) 000074[1] 1744535464.795080 mapread 0x89e7c3 thru 0x8a4522 (0x5d60 bytes) 000076[1] 1744535464.809068 write 0x56b89f thru 0x56f6a5 (0x3e07 bytes) 000080[0] 1744535464.819708 mapwrite 0x216f55 thru 0x21c0d1 (0x517d bytes) 000081[0] 1744535464.846848 mapwrite 0x733b07 thru 0x7431ee (0xf6e8 bytes) 000082[0] 1744535464.889842 trunc from 0x921218 to 0x3544ad (0x5ccd6a bytes) 000083[1] 1744535464.894764 mapwrite 0x6ad5a2 thru 0x6b8d12 (0xb771 bytes) truncating to largest ever: 0x7fe0a6 000087[0] 1744535464.928734 trunc from 0x6b8d13 to 0x7fe0a5 (0x145393 bytes) 000088[0] 1744535464.939764 mapwrite 0x079b2a thru 0x08686d (0xcd44 bytes) 000090[0] 1744535464.967318 read 0x679def thru 0x67a151 (0x363 bytes) 000093[0] 1744535464.973078 trunc from 0x7fe0a6 to 0x13ae83 (0x6c3222 bytes) 000097[1] 1744535464.983043 write 0x305793 thru 0x30f187 (0x99f5 bytes) 000098[0] 1744535464.989173 read 0x20d3df thru 0x215e62 (0x8a84 bytes) 000102[1] 1744535465.018274 write 0x9b5e2a thru 0x9c561b (0xf7f2 bytes) 000103[1] 1744535465.030572 mapwrite 0x8584f2 thru 0x864d77 (0xc886 bytes) 000108[0] 1744535465.071986 write 0x538d31 thru 0x53cdca (0x409a bytes) 000110[0] 1744535465.078779 mapwrite 0x17e37e thru 0x1841df (0x5e62 bytes) 000111[0] 1744535465.109952 mapread 0x0e92f5 thru 0x0f3755 (0xa461 bytes) 000112[0] 1744535465.124276 mapwrite 0x403a49 thru 0x4075c4 (0x3b7c bytes) 000114[1] 1744535465.149614 mapread 0x98127a thru 0x9817d4 (0x55b bytes) 000116[1] 1744535465.160939 write 0x6fb811 thru 0x700124 (0x4914 bytes) 000117[1] 1744535465.169576 write 0x6589ff thru 0x6616bd (0x8cbf bytes) 000118[0] 1744535465.173043 write 0x345965 thru 0x3489d0 (0x306c bytes) 000120[0] 1744535465.182783 mapread 0x21e1d5 thru 0x22949a (0xb2c6 bytes) 000121[0] 1744535465.199444 mapread 0x511a51 thru 0x516360 (0x4910 bytes) 000126[0] 1744535465.216680 trunc from 0x9c561c to 0x563c59 (0x4619c2 bytes) 000127[1] 1744535465.223342 write 0x99fce5 thru 0x9a7916 (0x7c32 bytes) 000128[1] 1744535465.231792 mapwrite 0x330678 thru 0x3389a9 (0x8332 bytes) 000130[1] 1744535465.267440 mapread 0x8f2b81 thru 0x8f870d (0x5b8d bytes) 000133[0] 1744535465.284567 write 0x869ca1 thru 0x8762bb (0xc61b bytes) 000135[0] 1744535465.293794 write 0x4a8688 thru 0x4b4135 (0xbaae bytes) 000136[1] 1744535465.297674 read 0x448316 thru 0x450071 (0x7d5c bytes) 000138[1] 1744535465.307683 mapread 0x3cb1d5 thru 0x3cbdb6 (0xbe2 bytes) 000139[0] 1744535465.316117 mapread 0x8ef396 thru 0x8f2b27 (0x3792 bytes) 000140[1] 1744535465.320937 mapwrite 0x9d9cfe thru 0x9dfd10 (0x6013 bytes) 000142[1] 1744535465.350298 trunc from 0x9dfd11 to 0x6a8f6b (0x336da5 bytes) 000143[0] 1744535465.355114 trunc from 0x6a8f6c to 0x08861d (0x62094e bytes) 000144[1] 1744535465.363112 mapwrite 0x63a5c5 thru 0x6490ef (0xeb2b bytes) truncating to largest ever: 0x80ce1c 000147[1] 1744535465.399708 trunc from 0x6490f0 to 0x80ce1b (0x1c3d2c bytes) 000149[0] 1744535465.413216 write 0x27ddc7 thru 0x283bc1 (0x5dfb bytes) 000152[1] 1744535465.429698 write 0x690f2d thru 0x69f6c2 (0xe796 bytes) 000157[0] 1744535465.450655 trunc from 0x80ce1c to 0x02f79d (0x7dd67e bytes) 000160[1] 1744535465.463282 write 0x9c4fbe thru 0x9d36d0 (0xe713 bytes) 000161[1] 1744535465.471011 write 0x76a8b1 thru 0x77651e (0xbc6e bytes) 000162[1] 1744535465.478389 write 0x79f43a thru 0x7a4762 (0x5329 bytes) 000163[1] 1744535465.481827 trunc from 0x9d36d1 to 0x04f8da (0x983df6 bytes) 000164[1] 1744535465.489454 mapread 0x022387 thru 0x023a2a (0x16a4 bytes) 000165[1] 1744535465.492787 mapread 0x00a187 thru 0x0194cd (0xf347 bytes) 000166[0] 1744535465.496685 read 0x00b758 thru 0x00cc20 (0x14c9 bytes) 000167[0] 1744535465.507397 trunc from 0x04f8db to 0x328edb (0x2d9601 bytes) 000169[1] 1744535465.517010 mapread 0x1026a1 thru 0x10fd24 (0xd684 bytes) 000174[1] 1744535465.527610 mapwrite 0x7b826b thru 0x7c47d5 (0xc56b bytes) 000175[1] 1744535465.559754 trunc from 0x7c47d6 to 0x7009c5 (0xc3e10 bytes) 000177[0] 1744535465.568815 trunc from 0x7009c6 to 0x3fa20c (0x3067b9 bytes) 000178[0] 1744535465.578421 write 0x3daddb thru 0x3e9b76 (0xed9c bytes) 000184[0] 1744535465.611792 write 0x214592 thru 0x221578 (0xcfe7 bytes) 000185[1] 1744535465.613701 mapread 0x39ad6f thru 0x39f4f2 (0x4784 bytes) 000186[1] 1744535465.629030 read 0x3c631e thru 0x3c915e (0x2e41 bytes) 000187[0] 1744535465.655765 mapwrite 0x40cc8d thru 0x40f0ab (0x241f bytes) 000190[0] 1744535465.685907 mapwrite 0x538dac thru 0x546619 (0xd86e bytes) 000193[1] 1744535465.713597 write 0x4e24f6 thru 0x4e50a6 (0x2bb1 bytes) 000195[0] 1744535465.727603 write 0x8ca6b6 thru 0x8cd149 (0x2a94 bytes) 000196[0] 1744535465.733295 read 0x46d040 thru 0x475d2a (0x8ceb bytes) 000197[0] 1744535465.738952 trunc from 0x8cd14a to 0x447097 (0x4860b2 bytes) 000198[1] 1744535465.744954 read 0x1a3191 thru 0x1a826a (0x50da bytes) 000199[1] 1744535465.753356 mapread 0x0af21f thru 0x0b0fc4 (0x1da6 bytes) 000200[1] 1744535465.757938 write 0x0773e1 thru 0x079e85 (0x2aa5 bytes) 000201[0] 1744535465.764772 trunc from 0x447098 to 0x76306d (0x31bfd6 bytes) 000203[1] 1744535465.776548 mapwrite 0x5817e9 thru 0x590908 (0xf120 bytes) 000204[1] 1744535465.803215 trunc from 0x76306e to 0x0e9e50 (0x67921d bytes) 000205[1] 1744535465.809657 write 0x8559a2 thru 0x86283d (0xce9c bytes) truncating to largest ever: 0x80fc66 000206[0] 1744535465.818448 trunc from 0x86283e to 0x80fc65 (0x52bd8 bytes) 000207[0] 1744535465.831078 read 0x60fc75 thru 0x618117 (0x84a3 bytes) 000208[1] 1744535465.834296 read 0x807ad9 thru 0x80aed2 (0x33fa bytes) 000209[1] 1744535465.840014 mapread 0x67bf88 thru 0x68b572 (0xf5eb bytes) 000210[0] 1744535465.849243 trunc from 0x80fc66 to 0x2d5a5a (0x53a20b bytes) 000212[0] 1744535465.863361 read 0x27a778 thru 0x27eb4f (0x43d8 bytes) 000213[0] 1744535465.883328 read 0x010aed thru 0x0156fe (0x4c12 bytes) 000214[1] 1744535465.883611 trunc from 0x2d5a5b to 0x331f29 (0x5c4cf bytes) 000216[0] 1744535465.893720 write 0x3c3d63 thru 0x3cf333 (0xb5d1 bytes) 000219[0] 1744535465.905712 read 0x33467b thru 0x340bc5 (0xc54b bytes) 000224[0] 1744535465.941412 mapread 0x0c87ed thru 0x0cec50 (0x6464 bytes) 000229[0] 1744535465.958916 trunc from 0x3cf334 to 0x3af710 (0x1fc23 bytes) 000231[0] 1744535465.969175 write 0x4b802b thru 0x4c7deb (0xfdc1 bytes) 000233[0] 1744535465.973817 mapwrite 0x225e2c thru 0x2355ed (0xf7c2 bytes) 000234[1] 1744535465.991177 trunc from 0x4c7dec to 0x60820e (0x140423 bytes) 000235[0] 1744535466.008037 write 0x51b8e4 thru 0x525212 (0x992f bytes) 000237[0] 1744535466.021464 trunc from 0x60820f to 0x2511c8 (0x3b7046 bytes) 000241[0] 1744535466.033113 mapwrite 0x9dc5cf thru 0x9dc880 (0x2b2 bytes) 000243[0] 1744535466.058832 read 0x648fc9 thru 0x64c8e2 (0x391a bytes) 000245[0] 1744535466.065101 trunc from 0x9dc881 to 0x7ed432 (0x1ef44e bytes) 000246[1] 1744535466.069526 read 0x666493 thru 0x66b28f (0x4dfd bytes) 000249[1] 1744535466.078529 mapwrite 0x774d02 thru 0x77cecf (0x81ce bytes) 000250[0] 1744535466.104850 mapwrite 0x418f90 thru 0x41f7c1 (0x6832 bytes) 000251[1] 1744535466.126939 mapwrite 0x9fe44c thru 0x9fffff (0x1bb4 bytes) 000252[0] 1744535466.149404 trunc from 0xa00000 to 0x12c955 (0x8d36aa bytes) 000254[0] 1744535466.161295 mapwrite 0x8f3672 thru 0x8fdaa7 (0xa436 bytes) 000255[0] 1744535466.185851 mapread 0x496adc thru 0x49f17a (0x869f bytes) 000257[1] 1744535466.198391 mapwrite 0x13a969 thru 0x13eae4 (0x417c bytes) 000260[0] 1744535466.229364 mapread 0x7b3791 thru 0x7bad4f (0x75bf bytes) 000261[0] 1744535466.238960 mapread 0x4e35b9 thru 0x4f2be4 (0xf62c bytes) 000262[1] 1744535466.248294 trunc from 0x8fdaa8 to 0x785f45 (0x177b62 bytes) 000265[1] 1744535466.262539 read 0x587d33 thru 0x58aed1 (0x319f bytes) 000266[1] 1744535466.267926 trunc from 0x785f46 to 0x6744b6 (0x111a8f bytes) 000272[0] 1744535466.287690 write 0x4812c9 thru 0x48f972 (0xe6aa bytes) 000273[1] 1744535466.298013 trunc from 0x6744b7 to 0x503efa (0x1705bc bytes) 000274[0] 1744535466.309566 mapread 0x1580cd thru 0x15aa14 (0x2948 bytes) 000275[0] 1744535466.322619 read 0x4f85f5 thru 0x4fe671 (0x607d bytes) 000276[0] 1744535466.329048 write 0x91e703 thru 0x91fe59 (0x1757 bytes) 000277[1] 1744535466.334001 mapwrite 0x6023c2 thru 0x6120fd (0xfd3c bytes) 000282[1] 1744535466.440598 mapwrite 0x262bf0 thru 0x26aa20 (0x7e31 bytes) 000285[0] 1744535466.466851 mapread 0x00ea2c thru 0x010de3 (0x23b8 bytes) 000286[1] 1744535466.475491 mapwrite 0x05a817 thru 0x067b34 (0xd31e bytes) 000287[0] 1744535466.497949 write 0x4f7a8c thru 0x505730 (0xdca5 bytes) 000288[1] 1744535466.505506 read 0x8fbd95 thru 0x8fe252 (0x24be bytes) 000290[1] 1744535466.513151 write 0x6fe3a2 thru 0x6ff3ea (0x1049 bytes) 000292[1] 1744535466.522268 mapread 0x042e09 thru 0x04849c (0x5694 bytes) 000299[0] 1744535466.541185 read 0x050e14 thru 0x052897 (0x1a84 bytes) 000303[0] 1744535466.554962 mapwrite 0x5ffbcb thru 0x60d59e (0xd9d4 bytes) 000304[0] 1744535466.581403 mapread 0x055392 thru 0x058e70 (0x3adf bytes) 000310[1] 1744535466.594121 mapwrite 0x7a65eb thru 0x7b4791 (0xe1a7 bytes) 000311[0] 1744535466.633953 mapread 0x5041fc thru 0x511a0a (0xd80f bytes) 000312[1] 1744535466.643054 trunc from 0x91fe5a to 0x122272 (0x7fdbe7 bytes) 000315[0] 1744535466.661546 write 0x3fd4e8 thru 0x401359 (0x3e72 bytes) 000318[0] 1744535466.669514 read 0x3f3dd5 thru 0x3f7aca (0x3cf6 bytes) 000320[1] 1744535466.674356 read 0x061cc8 thru 0x06e4d4 (0xc80d bytes) 000321[1] 1744535466.680641 mapread 0x05e4e9 thru 0x06105e (0x2b76 bytes) 000322[0] 1744535466.685754 mapread 0x041242 thru 0x04de82 (0xcc41 bytes) 000323[1] 1744535466.695305 write 0x589534 thru 0x591b36 (0x8603 bytes) truncating to largest ever: 0x836e03 000324[1] 1744535466.703436 trunc from 0x591b37 to 0x836e02 (0x2a52cc bytes) 000325[0] 1744535466.711267 read 0x292f65 thru 0x29e225 (0xb2c1 bytes) 000326[0] 1744535466.719148 mapwrite 0x9be1df thru 0x9c572a (0x754c bytes) 000327[0] 1744535466.751574 mapwrite 0x318677 thru 0x31ab8c (0x2516 bytes) 000328[1] 1744535466.771856 mapread 0x6e8b95 thru 0x6ecb30 (0x3f9c bytes) 000329[1] 1744535466.781696 mapread 0x676872 thru 0x67ee9c (0x862b bytes) 000330[0] 1744535466.786319 trunc from 0x9c572b to 0x0eef6d (0x8d67bd bytes) 000332[0] 1744535466.798458 mapwrite 0x56199c thru 0x563acd (0x2132 bytes) truncating to largest ever: 0x879927 000333[1] 1744535466.823366 trunc from 0x563ace to 0x879926 (0x315e59 bytes) 000336[0] 1744535466.837562 write 0x3911ca thru 0x391a21 (0x858 bytes) 000337[0] 1744535466.844248 mapwrite 0x7b8f13 thru 0x7bada8 (0x1e96 bytes) 000342[0] 1744535466.871580 read 0x3a8ffe thru 0x3b1815 (0x8818 bytes) 000344[1] 1744535466.880699 trunc from 0x879927 to 0x21ac81 (0x65eca5 bytes) 000345[1] 1744535466.892736 trunc from 0x21ac82 to 0x4969d1 (0x27bd50 bytes) 000347[0] 1744535466.897442 mapread 0x2926b1 thru 0x29aef8 (0x8848 bytes) 000349[1] 1744535466.912306 mapwrite 0x5a712e thru 0x5aa693 (0x3566 bytes) 000351[1] 1744535466.940510 write 0x4bf449 thru 0x4c77f5 (0x83ad bytes) truncating to largest ever: 0x9b8e02 000355[0] 1744535466.954167 trunc from 0x5aa694 to 0x9b8e01 (0x40e76e bytes) 000360[1] 1744535466.980324 read 0x755cc6 thru 0x75d18f (0x74ca bytes) 000362[0] 1744535466.985240 write 0x68422b thru 0x68d155 (0x8f2b bytes) 000363[0] 1744535466.989203 write 0x4f2b5c thru 0x4f57dd (0x2c82 bytes) 000364[1] 1744535466.994361 write 0x655278 thru 0x65dbcd (0x8956 bytes) 000367[1] 1744535467.009892 write 0x5f3fdb thru 0x5ff1fe (0xb224 bytes) 000370[0] 1744535467.026720 mapread 0x85e0c6 thru 0x86324b (0x5186 bytes) 000371[0] 1744535467.035566 mapwrite 0x2568bb thru 0x2663eb (0xfb31 bytes) 000372[0] 1744535467.063587 mapwrite 0x1b6341 thru 0x1b757a (0x123a bytes) 000374[0] 1744535467.087368 write 0x5a68d1 thru 0x5aaba3 (0x42d3 bytes) 000378[0] 1744535467.097788 mapread 0x95bd52 thru 0x96084f (0x4afe bytes) 000382[1] 1744535467.110589 mapwrite 0x4a09c5 thru 0x4ad45f (0xca9b bytes) 000383[0] 1744535467.141986 read 0x27de4f thru 0x285963 (0x7b15 bytes) 000384[0] 1744535467.147584 read 0x0ceece thru 0x0cf71a (0x84d bytes) 000386[0] 1744535467.155855 write 0x92c268 thru 0x92cf83 (0xd1c bytes) 000387[0] 1744535467.160636 trunc from 0x9b8e02 to 0x0347ba (0x984647 bytes) truncating to largest ever: 0x9f2be4 000392[1] 1744535467.177446 trunc from 0x0347bb to 0x9f2be3 (0x9be429 bytes) 000394[1] 1744535467.191112 mapread 0x8903d3 thru 0x8928f9 (0x2527 bytes) 000395[1] 1744535467.198530 write 0x602494 thru 0x60617b (0x3ce8 bytes) 000397[0] 1744535467.204038 write 0x844580 thru 0x84b31c (0x6d9d bytes) 000398[0] 1744535467.210755 trunc from 0x9f2be4 to 0x80ed36 (0x1e3ead bytes) 000399[1] 1744535467.220846 mapread 0x22255c thru 0x227205 (0x4caa bytes) 000401[0] 1744535467.232201 trunc from 0x80ed37 to 0x270320 (0x59ea16 bytes) 000403[1] 1744535467.241275 read 0x26bd79 thru 0x270320 (0x45a8 bytes) 000406[0] 1744535467.259791 mapwrite 0x1d4b9b thru 0x1e02aa (0xb710 bytes) 000409[1] 1744535467.296744 mapread 0x10ff59 thru 0x115f6a (0x6012 bytes) 000410[0] 1744535467.310334 trunc from 0x270321 to 0x6d3bd1 (0x4638b1 bytes) 000411[0] 1744535467.317789 read 0x213f53 thru 0x215a22 (0x1ad0 bytes) 000412[0] 1744535467.323795 mapread 0x3fc21b thru 0x3fcda7 (0xb8d bytes) 000414[0] 1744535467.330164 write 0x7e560c thru 0x7ed3ea (0x7ddf bytes) 000420[1] 1744535467.343478 read 0x7d7dde thru 0x7df8ea (0x7b0d bytes) 000421[1] 1744535467.353071 mapwrite 0x0927d8 thru 0x09d30e (0xab37 bytes) 000422[1] 1744535467.376889 read 0x24a9f9 thru 0x24c614 (0x1c1c bytes) 000424[0] 1744535467.381565 mapread 0x48851d thru 0x489689 (0x116d bytes) 000428[1] 1744535467.393488 read 0x556b90 thru 0x55d188 (0x65f9 bytes) 000429[0] 1744535467.395369 write 0x426b1a thru 0x431fd3 (0xb4ba bytes) 000430[0] 1744535467.404431 write 0x9a4577 thru 0x9ab2ed (0x6d77 bytes) 000431[0] 1744535467.410192 mapwrite 0x464831 thru 0x46cd7f (0x854f bytes) 000432[0] 1744535467.431690 write 0x698a3e thru 0x6a3cf7 (0xb2ba bytes) 000434[0] 1744535467.440747 read 0x85e115 thru 0x86c849 (0xe735 bytes) 000436[1] 1744535467.443158 trunc from 0x9ab2ee to 0x6bc43d (0x2eeeb0 bytes) 000438[0] 1744535467.458744 mapwrite 0x20357b thru 0x20ba99 (0x851f bytes) 000439[0] 1744535467.483678 read 0x56239c thru 0x563eaf (0x1b14 bytes) 000441[0] 1744535467.488992 trunc from 0x6bc43e to 0x616072 (0xa63cb bytes) 000445[0] 1744535467.499483 mapwrite 0x087ae1 thru 0x090b74 (0x9094 bytes) 000446[0] 1744535467.520776 mapread 0x2d5319 thru 0x2db3b7 (0x609f bytes) 000447[0] 1744535467.525534 mapwrite 0x8eec74 thru 0x8f6b05 (0x7e92 bytes) 000454[0] 1744535467.563917 write 0x76bf95 thru 0x7790b8 (0xd124 bytes) 000455[0] 1744535467.569781 write 0x4cbe6d thru 0x4d6ba4 (0xad38 bytes) 000456[0] 1744535467.572981 mapread 0x095efe thru 0x0a07c3 (0xa8c6 bytes) 000458[1] 1744535467.584216 mapread 0x0a0e4a thru 0x0a5b1d (0x4cd4 bytes) 000460[1] 1744535467.596163 mapwrite 0x17245d thru 0x17eeb6 (0xca5a bytes) 000462[0] 1744535467.624996 read 0x7c849d thru 0x7ce0a5 (0x5c09 bytes) 000463[1] 1744535467.630331 write 0x6a7332 thru 0x6a838f (0x105e bytes) 000465[1] 1744535467.640756 trunc from 0x8f6b06 to 0x9ca8b8 (0xd3db3 bytes) 000466[1] 1744535467.649013 read 0x254413 thru 0x259c91 (0x587f bytes) 000468[1] 1744535467.655718 write 0x81d5bc thru 0x8285e1 (0xb026 bytes) 000470[0] 1744535467.661685 write 0x550209 thru 0x55a227 (0xa01f bytes) 000474[0] 1744535467.679206 mapwrite 0x0cb483 thru 0x0d9d31 (0xe8af bytes) 000475[1] 1744535467.706156 mapread 0x966240 thru 0x9675cc (0x138d bytes) 000476[0] 1744535467.713225 mapwrite 0x5a9f13 thru 0x5b3ec6 (0x9fb4 bytes) 000478[1] 1744535467.731740 trunc from 0x9ca8b9 to 0x212bf9 (0x7b7cbf bytes) 000479[1] 1744535467.743667 mapread 0x1c6577 thru 0x1cd5e4 (0x706e bytes) 000480[0] 1744535467.750609 mapwrite 0x5a64d9 thru 0x5b2a24 (0xc54c bytes) 000483[0] 1744535467.783620 write 0x77fd5f thru 0x78c48a (0xc72c bytes) 000486[0] 1744535467.788365 trunc from 0x78c48b to 0x8897e6 (0xfd35c bytes) 000487[1] 1744535467.795657 trunc from 0x8897e7 to 0x3382a8 (0x55153e bytes) truncating to largest ever: 0x9f4eb2 000490[0] 1744535467.811297 trunc from 0x3382a9 to 0x9f4eb1 (0x6bcc09 bytes) 000491[0] 1744535467.821794 write 0x31d5c2 thru 0x3203ee (0x2e2d bytes) 000492[0] 1744535467.829865 mapread 0x2ff200 thru 0x30eba3 (0xf9a4 bytes) 000497[0] 1744535467.845292 mapwrite 0x2d3615 thru 0x2dae06 (0x77f2 bytes) 000498[0] 1744535467.862381 trunc from 0x9f4eb2 to 0x5db4fd (0x4199b4 bytes) 000500[0] 1744535467.872363 write 0x75e19d thru 0x766eb8 (0x8d1c bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 5467 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[1] 1744535468.102325 write 0x688000 thru 0x69327a (0xb27b bytes) 000003[0] 1744535468.109981 write_OD 0x278000 thru 0x279fff (0x2000 bytes) 000005[0] 1744535468.127843 read_OD 0x101000 thru 0x105fff (0x5000 bytes) 000006[0] 1744535468.130091 read_OD 0x02b000 thru 0x037fff (0xd000 bytes) truncating to largest ever: 0x6f6dec 000009[0] 1744535468.132729 trunc from 0x69327b to 0x6f6deb (0x63b71 bytes) 000011[1] 1744535468.144548 read 0x6f4000 thru 0x6f648c (0x248d bytes) truncating to largest ever: 0x7621f5 000012[1] 1744535468.147963 trunc from 0x6f6dec to 0x7621f4 (0x6b409 bytes) 000013[0] 1744535468.156263 trunc from 0x7621f5 to 0x36ebb5 (0x3f363f bytes) 000014[1] 1744535468.164614 write 0x96f000 thru 0x973d49 (0x4d4a bytes) 000015[0] 1744535468.171914 mapread 0x801000 thru 0x8072cf (0x62d0 bytes) truncating to largest ever: 0x87b40d 000017[1] 1744535468.182038 trunc from 0x973d4a to 0x87b40c (0xf893d bytes) 000020[0] 1744535468.195913 mapread 0x860000 thru 0x86d72a (0xd72b bytes) 000022[0] 1744535468.203691 mapread 0x1ac000 thru 0x1ac8d1 (0x8d2 bytes) 000025[0] 1744535468.207064 read_OD 0x744000 thru 0x752fff (0xf000 bytes) 000027[0] 1744535468.211245 mapread 0x15a000 thru 0x164728 (0xa729 bytes) 000029[1] 1744535468.216852 trunc from 0x87b40d to 0x0903fc (0x7eb010 bytes) 000034[0] 1744535468.233893 trunc from 0x0903fd to 0x33f42b (0x2af02f bytes) 000035[1] 1744535468.240923 write 0x606000 thru 0x60c099 (0x609a bytes) 000037[0] 1744535468.248109 mapwrite 0x1df000 thru 0x1ebf74 (0xcf75 bytes) 000038[0] 1744535468.286568 mapwrite 0x570000 thru 0x571a61 (0x1a62 bytes) truncating to largest ever: 0x8f83ce 000041[1] 1744535468.310444 trunc from 0x60c09a to 0x8f83cd (0x2ec334 bytes) 000046[0] 1744535468.323880 mapwrite 0x15f000 thru 0x168174 (0x9175 bytes) 000048[1] 1744535468.350368 read 0x728000 thru 0x7294ed (0x14ee bytes) 000049[0] 1744535468.353732 write_OD 0x039000 thru 0x03afff (0x2000 bytes) 000054[0] 1744535468.370030 trunc from 0x8f83ce to 0x01f63f (0x8d8d8e bytes) 000058[1] 1744535468.386749 trunc from 0x01f640 to 0x67aa8f (0x65b450 bytes) 000059[1] 1744535468.395916 mapread 0x11a000 thru 0x125321 (0xb322 bytes) 000061[1] 1744535468.406568 write 0x0fa000 thru 0x100575 (0x6576 bytes) 000069[1] 1744535468.420785 mapwrite 0x5c8000 thru 0x5d34e8 (0xb4e9 bytes) 000072[1] 1744535468.446484 mapwrite 0x254000 thru 0x263e58 (0xfe59 bytes) 000073[1] 1744535468.473277 mapwrite 0x8ae000 thru 0x8b6f4b (0x8f4c bytes) 000077[0] 1744535468.504093 mapwrite 0x748000 thru 0x74a6f9 (0x26fa bytes) 000083[1] 1744535468.536630 mapread 0x60a000 thru 0x6190ef (0xf0f0 bytes) 000085[0] 1744535468.548771 mapread 0x248000 thru 0x24d15d (0x515e bytes) 000088[0] 1744535468.554480 mapwrite 0x78a000 thru 0x792e80 (0x8e81 bytes) 000090[0] 1744535468.586835 mapread 0x321000 thru 0x32b82d (0xa82e bytes) 000091[0] 1744535468.596022 write_OD 0x6c5000 thru 0x6cdfff (0x9000 bytes) 000095[0] 1744535468.612784 mapwrite 0x0be000 thru 0x0c5e8d (0x7e8e bytes) 000098[1] 1744535468.632874 write 0x12a000 thru 0x12f9d5 (0x59d6 bytes) 000102[1] 1744535468.644375 read 0x3e6000 thru 0x3e8d63 (0x2d64 bytes) 000103[0] 1744535468.648728 read_OD 0x567000 thru 0x571fff (0xb000 bytes) 000105[0] 1744535468.656455 trunc from 0x8b6f4c to 0x173f1d (0x74302e bytes) 000106[1] 1744535468.666355 mapread 0x04d000 thru 0x057eb6 (0xaeb7 bytes) 000107[0] 1744535468.677588 mapread 0x095000 thru 0x09704a (0x204b bytes) 000109[0] 1744535468.682998 write_OD 0x6e6000 thru 0x6e6fff (0x1000 bytes) 000113[0] 1744535468.701061 mapwrite 0x042000 thru 0x044903 (0x2904 bytes) 000114[1] 1744535468.719856 mapwrite 0x680000 thru 0x687b8d (0x7b8e bytes) 000116[0] 1744535468.743819 trunc from 0x6e7000 to 0x334860 (0x3b279f bytes) 000117[1] 1744535468.750360 read 0x26b000 thru 0x26cc87 (0x1c88 bytes) 000118[1] 1744535468.754465 read 0x0af000 thru 0x0af51e (0x51f bytes) 000119[1] 1744535468.773344 trunc from 0x334861 to 0x4bc3d3 (0x187b73 bytes) 000121[1] 1744535468.786671 read 0x0fc000 thru 0x10384e (0x784f bytes) 000123[1] 1744535468.795442 mapread 0x3e2000 thru 0x3e4960 (0x2961 bytes) 000124[0] 1744535468.801132 write_OD 0x302000 thru 0x30ffff (0xe000 bytes) 000130[1] 1744535468.818637 mapread 0x358000 thru 0x364303 (0xc304 bytes) 000134[1] 1744535468.824653 mapread 0x371000 thru 0x376c69 (0x5c6a bytes) 000135[1] 1744535468.827135 write 0x96c000 thru 0x97a726 (0xe727 bytes) 000137[1] 1744535468.833941 write 0x757000 thru 0x75a36a (0x336b bytes) 000138[0] 1744535468.838336 write_OD 0x976000 thru 0x980fff (0xb000 bytes) 000139[1] 1744535468.855544 write 0x7b6000 thru 0x7c49f0 (0xe9f1 bytes) 000142[0] 1744535468.868351 write_OD 0x2c4000 thru 0x2d2fff (0xf000 bytes) 000143[0] 1744535468.884310 read_OD 0x53e000 thru 0x54cfff (0xf000 bytes) 000146[1] 1744535468.887654 trunc from 0x981000 to 0x20392d (0x77d6d2 bytes) 000148[1] 1744535468.898425 read 0x159000 thru 0x15d20d (0x420e bytes) 000153[1] 1744535468.920001 trunc from 0x20392e to 0x10c5c5 (0xf7368 bytes) 000154[0] 1744535468.925141 write_OD 0x59b000 thru 0x5a5fff (0xb000 bytes) 000155[1] 1744535468.941685 write 0x853000 thru 0x85a1ff (0x7200 bytes) truncating to largest ever: 0x91ecfe 000158[0] 1744535468.955329 trunc from 0x85a200 to 0x91ecfd (0xc4afe bytes) 000160[1] 1744535468.967535 read 0x426000 thru 0x431a62 (0xba63 bytes) 000161[1] 1744535468.972564 read 0x84a000 thru 0x84ae9a (0xe9b bytes) 000162[0] 1744535468.973889 trunc from 0x91ecfe to 0x726bda (0x1f8123 bytes) 000164[1] 1744535468.979280 mapread 0x521000 thru 0x52ad44 (0x9d45 bytes) 000165[1] 1744535468.990763 read 0x11d000 thru 0x11ef25 (0x1f26 bytes) truncating to largest ever: 0x95ed9b 000166[0] 1744535468.993819 trunc from 0x726bdb to 0x95ed9a (0x2381c0 bytes) 000169[0] 1744535469.001157 trunc from 0x95ed9b to 0x27bb72 (0x6e3228 bytes) 000171[1] 1744535469.008835 read 0x269000 thru 0x274c6a (0xbc6b bytes) 000173[1] 1744535469.027135 trunc from 0x27bb73 to 0x3dcbde (0x16106c bytes) 000180[1] 1744535469.052840 trunc from 0x3dcbdf to 0x47a489 (0x9d8ab bytes) 000181[1] 1744535469.057290 trunc from 0x47a48a to 0x258677 (0x221e12 bytes) 000183[0] 1744535469.060478 write_OD 0x12d000 thru 0x12efff (0x2000 bytes) 000184[0] 1744535469.073322 trunc from 0x258678 to 0x338e96 (0xe081f bytes) 000185[0] 1744535469.082241 mapwrite 0x662000 thru 0x66d32f (0xb330 bytes) 000188[0] 1744535469.110056 mapread 0x57f000 thru 0x5813a6 (0x23a7 bytes) 000190[0] 1744535469.118155 trunc from 0x66d330 to 0x859adc (0x1ec7ad bytes) 000192[1] 1744535469.127476 mapread 0x3dc000 thru 0x3e916d (0xd16e bytes) 000194[1] 1744535469.156643 mapread 0x808000 thru 0x8083e2 (0x3e3 bytes) 000195[0] 1744535469.170393 trunc from 0x859add to 0x470293 (0x3e9849 bytes) 000197[0] 1744535469.182195 mapwrite 0x291000 thru 0x29d2d7 (0xc2d8 bytes) 000198[1] 1744535469.208904 read 0x069000 thru 0x070431 (0x7432 bytes) 000199[0] 1744535469.213427 write_OD 0x0ae000 thru 0x0b4fff (0x7000 bytes) 000201[0] 1744535469.226006 mapwrite 0x08f000 thru 0x09493e (0x593f bytes) 000202[1] 1744535469.244653 mapread 0x205000 thru 0x206c6a (0x1c6b bytes) 000203[0] 1744535469.251764 mapread 0x2b4000 thru 0x2c206f (0xe070 bytes) 000204[1] 1744535469.258590 mapwrite 0x874000 thru 0x88374a (0xf74b bytes) 000207[1] 1744535469.295214 mapread 0x649000 thru 0x64bf2c (0x2f2d bytes) 000208[0] 1744535469.302579 mapwrite 0x048000 thru 0x057a53 (0xfa54 bytes) 000209[1] 1744535469.323622 read 0x133000 thru 0x13942d (0x642e bytes) 000212[1] 1744535469.328962 write 0x188000 thru 0x18bd3d (0x3d3e bytes) 000214[1] 1744535469.335106 read 0x6d9000 thru 0x6e5806 (0xc807 bytes) 000216[1] 1744535469.337502 trunc from 0x88374b to 0x6212dc (0x26246e bytes) 000217[0] 1744535469.342027 mapread 0x11c000 thru 0x11e725 (0x2726 bytes) 000218[1] 1744535469.350056 write 0x1c9000 thru 0x1d018a (0x718b bytes) 000223[0] 1744535469.369692 mapwrite 0x085000 thru 0x0902e5 (0xb2e6 bytes) 000224[0] 1744535469.397628 read_OD 0x0a2000 thru 0x0a4fff (0x3000 bytes) 000225[1] 1744535469.401087 write 0x9a7000 thru 0x9b486b (0xd86c bytes) 000228[1] 1744535469.407510 mapread 0x2b3000 thru 0x2b6f0f (0x3f10 bytes) 000231[1] 1744535469.413777 trunc from 0x9b486c to 0x218cfe (0x79bb6d bytes) 000234[1] 1744535469.423219 mapwrite 0x744000 thru 0x749f28 (0x5f29 bytes) 000235[0] 1744535469.447918 mapread 0x5bb000 thru 0x5bbefe (0xeff bytes) 000238[0] 1744535469.454438 mapwrite 0x87a000 thru 0x8858ba (0xb8bb bytes) 000240[0] 1744535469.488220 mapwrite 0x751000 thru 0x75fa8d (0xea8e bytes) 000244[0] 1744535469.516230 mapread 0x166000 thru 0x1720b9 (0xc0ba bytes) 000246[1] 1744535469.525835 mapwrite 0x825000 thru 0x829d4d (0x4d4e bytes) 000248[0] 1744535469.551680 mapread 0x663000 thru 0x666d5b (0x3d5c bytes) 000250[1] 1744535469.562424 trunc from 0x8858bb to 0x8406a2 (0x45218 bytes) 000251[0] 1744535469.569257 trunc from 0x8406a3 to 0x907fb2 (0xc7910 bytes) 000252[1] 1744535469.578882 write 0x421000 thru 0x422226 (0x1227 bytes) 000253[1] 1744535469.582370 mapwrite 0x1ba000 thru 0x1beaf9 (0x4afa bytes) 000259[0] 1744535469.619917 read_OD 0x87f000 thru 0x87ffff (0x1000 bytes) 000260[0] 1744535469.625086 write_OD 0x10a000 thru 0x114fff (0xb000 bytes) 000261[0] 1744535469.640450 mapwrite 0x98a000 thru 0x996aed (0xcaee bytes) 000264[0] 1744535469.681489 read_OD 0x240000 thru 0x249fff (0xa000 bytes) 000265[1] 1744535469.687305 mapread 0x6da000 thru 0x6dbba4 (0x1ba5 bytes) 000268[0] 1744535469.690729 mapread 0x29c000 thru 0x29c0cf (0xd0 bytes) 000269[0] 1744535469.692511 mapwrite 0x3b6000 thru 0x3c5b49 (0xfb4a bytes) 000271[0] 1744535469.715028 mapwrite 0x4b3000 thru 0x4bd1a4 (0xa1a5 bytes) 000277[0] 1744535469.749283 trunc from 0x996aee to 0x1805b9 (0x816534 bytes) 000283[1] 1744535469.762246 mapwrite 0x9a5000 thru 0x9a793b (0x293c bytes) 000284[0] 1744535469.789108 mapwrite 0x146000 thru 0x150ed3 (0xaed4 bytes) 000285[1] 1744535469.812572 read 0x16b000 thru 0x16d976 (0x2977 bytes) 000288[1] 1744535469.817321 trunc from 0x9a793c to 0x75d2ba (0x24a681 bytes) 000291[1] 1744535469.827324 trunc from 0x75d2bb to 0x6621a8 (0xfb112 bytes) 000292[0] 1744535469.831645 mapread 0x0ee000 thru 0x0f8565 (0xa566 bytes) 000296[0] 1744535469.842120 write_OD 0x98e000 thru 0x997fff (0xa000 bytes) 000297[0] 1744535469.857933 mapread 0x39a000 thru 0x39e830 (0x4831 bytes) 000299[1] 1744535469.866390 trunc from 0x998000 to 0x7be58f (0x1d9a70 bytes) 000300[1] 1744535469.876754 mapwrite 0x438000 thru 0x43f9a5 (0x79a6 bytes) 000301[1] 1744535469.899085 mapwrite 0x77e000 thru 0x780d96 (0x2d97 bytes) 000302[1] 1744535469.917671 mapwrite 0x9d9000 thru 0x9dcddd (0x3dde bytes) 000305[1] 1744535469.942517 trunc from 0x9dcdde to 0x25fd48 (0x77d095 bytes) 000306[0] 1744535469.949950 read_OD 0x11b000 thru 0x128fff (0xe000 bytes) 000309[0] 1744535469.957662 mapread 0x1e9000 thru 0x1f8fae (0xffaf bytes) 000313[1] 1744535469.976430 write 0x36b000 thru 0x36eb89 (0x3b8a bytes) 000318[0] 1744535469.994847 write_OD 0x3f9000 thru 0x406fff (0xe000 bytes) 000319[0] 1744535470.012117 read_OD 0x00f000 thru 0x010fff (0x2000 bytes) 000320[0] 1744535470.014626 mapwrite 0x9f8000 thru 0x9f8cee (0xcef bytes) 000321[1] 1744535470.040850 read 0x6f8000 thru 0x7071bc (0xf1bd bytes) 000324[1] 1744535470.045502 mapwrite 0x02c000 thru 0x030923 (0x4924 bytes) 000325[1] 1744535470.068317 write 0x86f000 thru 0x87d00e (0xe00f bytes) 000331[1] 1744535470.081913 write 0x1f0000 thru 0x1f3692 (0x3693 bytes) 000333[0] 1744535470.087574 read_OD 0x6da000 thru 0x6dbfff (0x2000 bytes) 000335[0] 1744535470.094966 mapwrite 0x609000 thru 0x60f144 (0x6145 bytes) 000336[1] 1744535470.119159 mapread 0x346000 thru 0x349bcb (0x3bcc bytes) 000337[1] 1744535470.126225 trunc from 0x9f8cef to 0x069a3d (0x98f2b1 bytes) 000338[0] 1744535470.132711 mapwrite 0x217000 thru 0x21b786 (0x4787 bytes) 000340[0] 1744535470.157513 read_OD 0x080000 thru 0x084fff (0x5000 bytes) 000342[1] 1744535470.163157 mapread 0x00c000 thru 0x00dd8b (0x1d8c bytes) 000343[1] 1744535470.165514 read 0x102000 thru 0x10d1ac (0xb1ad bytes) 000344[0] 1744535470.181536 trunc from 0x21b787 to 0x25bc54 (0x404ce bytes) 000345[0] 1744535470.189782 trunc from 0x25bc55 to 0x8a8757 (0x64cb03 bytes) 000346[1] 1744535470.197860 mapread 0x0f1000 thru 0x0fd7c3 (0xc7c4 bytes) 000348[0] 1744535470.231222 write_OD 0x6a0000 thru 0x6a6fff (0x7000 bytes) 000350[0] 1744535470.247633 mapread 0x1d8000 thru 0x1e52b8 (0xd2b9 bytes) 000353[0] 1744535470.273874 mapwrite 0x353000 thru 0x35903c (0x603d bytes) 000354[1] 1744535470.315406 write 0x88e000 thru 0x89b4ce (0xd4cf bytes) 000357[0] 1744535470.344305 write_OD 0x00b000 thru 0x013fff (0x9000 bytes) 000358[0] 1744535470.363022 write_OD 0x9fa000 thru 0x9fefff (0x5000 bytes) 000362[0] 1744535470.376936 read_OD 0x030000 thru 0x039fff (0xa000 bytes) 000363[0] 1744535470.378973 mapread 0x8db000 thru 0x8e1d76 (0x6d77 bytes) 000365[0] 1744535470.383304 trunc from 0x9ff000 to 0x3f2c44 (0x60c3bb bytes) 000367[0] 1744535470.393243 read_OD 0x1f4000 thru 0x1fdfff (0xa000 bytes) 000370[0] 1744535470.397893 mapwrite 0x017000 thru 0x0214dd (0xa4de bytes) 000372[1] 1744535470.443544 read 0x29e000 thru 0x2a0ced (0x2cee bytes) 000376[1] 1744535470.472491 trunc from 0x3f2c45 to 0x3d79f4 (0x1b250 bytes) 000377[1] 1744535470.484837 trunc from 0x3d79f5 to 0x2154c9 (0x1c252b bytes) 000378[0] 1744535470.492064 write_OD 0x7b0000 thru 0x7b7fff (0x8000 bytes) 000380[1] 1744535470.511485 read 0x473000 thru 0x47d2ad (0xa2ae bytes) 000382[0] 1744535470.515038 trunc from 0x7b8000 to 0x63cd69 (0x17b296 bytes) 000384[1] 1744535470.526107 mapread 0x4d1000 thru 0x4d48b0 (0x38b1 bytes) 000388[1] 1744535470.535201 read 0x490000 thru 0x498176 (0x8177 bytes) 000389[1] 1744535470.537210 trunc from 0x63cd6a to 0x00a86b (0x6324fe bytes) 000391[0] 1744535470.550325 read_OD 0x001000 thru 0x008fff (0x8000 bytes) 000392[1] 1744535470.555394 write 0x5e5000 thru 0x5f3181 (0xe182 bytes) 000393[1] 1744535470.563308 read 0x081000 thru 0x090c13 (0xfc14 bytes) 000394[0] 1744535470.566510 trunc from 0x5f3182 to 0x6a035a (0xad1d9 bytes) 000395[1] 1744535470.577371 write 0x18d000 thru 0x192700 (0x5701 bytes) 000396[1] 1744535470.584763 write 0x6fb000 thru 0x7098bd (0xe8be bytes) 000398[1] 1744535470.585628 read 0x483000 thru 0x4921f9 (0xf1fa bytes) 000399[1] 1744535470.592873 mapread 0x338000 thru 0x3476a6 (0xf6a7 bytes) 000400[0] 1744535470.601315 mapwrite 0x695000 thru 0x6a1de5 (0xcde6 bytes) 000403[1] 1744535470.664503 write 0x13f000 thru 0x14065a (0x165b bytes) 000411[0] 1744535470.691775 mapread 0x6df000 thru 0x6e1a48 (0x2a49 bytes) 000413[0] 1744535470.700774 trunc from 0x7098be to 0x1fe8d7 (0x50afe6 bytes) 000417[0] 1744535470.712552 mapwrite 0x531000 thru 0x53ec66 (0xdc67 bytes) 000419[0] 1744535470.750830 mapread 0x409000 thru 0x4118ab (0x88ac bytes) 000422[0] 1744535470.762092 write_OD 0x208000 thru 0x211fff (0xa000 bytes) 000428[1] 1744535470.776229 mapread 0x40b000 thru 0x412310 (0x7311 bytes) 000429[1] 1744535470.780520 trunc from 0x53ec67 to 0x69c60b (0x15d9a5 bytes) 000430[0] 1744535470.787228 mapwrite 0x41e000 thru 0x426e57 (0x8e58 bytes) 000431[0] 1744535470.821655 read_OD 0x076000 thru 0x081fff (0xc000 bytes) 000432[1] 1744535470.826948 mapwrite 0x08e000 thru 0x095bed (0x7bee bytes) 000433[1] 1744535470.850707 read 0x1f8000 thru 0x206466 (0xe467 bytes) 000438[0] 1744535470.859765 trunc from 0x69c60c to 0x3cf702 (0x2ccf09 bytes) 000439[1] 1744535470.865944 mapread 0x246000 thru 0x248a98 (0x2a99 bytes) 000443[0] 1744535470.875225 read_OD 0x2e5000 thru 0x2f3fff (0xf000 bytes) 000444[0] 1744535470.877081 trunc from 0x3cf703 to 0x851cae (0x4825ac bytes) 000447[1] 1744535470.889182 write 0x0d9000 thru 0x0df768 (0x6769 bytes) 000450[1] 1744535470.902447 trunc from 0x851caf to 0x2dd998 (0x574316 bytes) 000451[1] 1744535470.909346 mapread 0x01e000 thru 0x022609 (0x460a bytes) 000452[0] 1744535470.914956 mapread 0x1f1000 thru 0x1ff604 (0xe605 bytes) 000455[1] 1744535470.937819 read 0x24c000 thru 0x257dd7 (0xbdd8 bytes) 000457[1] 1744535470.939959 trunc from 0x2dd999 to 0x0cf03b (0x20e95d bytes) 000458[0] 1744535470.948077 mapwrite 0x12b000 thru 0x13af8f (0xff90 bytes) 000459[1] 1744535470.987821 trunc from 0x13af90 to 0x71e8d9 (0x5e394a bytes) 000461[1] 1744535470.999534 write 0x812000 thru 0x821779 (0xf77a bytes) 000463[0] 1744535471.000527 mapwrite 0x30e000 thru 0x31363e (0x563f bytes) 000464[0] 1744535471.035561 mapwrite 0x0b9000 thru 0x0c2854 (0x9855 bytes) 000465[1] 1744535471.061710 write 0x044000 thru 0x045d9a (0x1d9b bytes) 000469[0] 1744535471.072097 mapread 0x6e7000 thru 0x6f1486 (0xa487 bytes) 000471[0] 1744535471.086765 mapwrite 0x675000 thru 0x67cc7b (0x7c7c bytes) 000477[0] 1744535471.116930 mapread 0x04b000 thru 0x04efd8 (0x3fd9 bytes) 000478[0] 1744535471.124358 trunc from 0x82177a to 0x273f27 (0x5ad852 bytes) 000479[1] 1744535471.128779 read 0x009000 thru 0x0152ba (0xc2bb bytes) 000480[0] 1744535471.133881 write_OD 0x4c4000 thru 0x4cafff (0x7000 bytes) 000481[0] 1744535471.148366 trunc from 0x4cb000 to 0x271c25 (0x2593da bytes) 000484[0] 1744535471.154686 mapwrite 0x3d8000 thru 0x3e4c4e (0xcc4f bytes) 000487[1] 1744535471.187660 write 0x90b000 thru 0x9130d3 (0x80d4 bytes) 000489[0] 1744535471.194184 trunc from 0x9130d4 to 0x862792 (0xb0941 bytes) 000490[1] 1744535471.201176 mapread 0x75e000 thru 0x761b86 (0x3b87 bytes) 000491[0] 1744535471.209354 mapwrite 0x5ca000 thru 0x5ce2ec (0x42ed bytes) 000494[0] 1744535471.235601 write_OD 0x866000 thru 0x86dfff (0x8000 bytes) 000496[1] 1744535471.248787 mapread 0x426000 thru 0x42f0fb (0x90fc bytes) 000497[1] 1744535471.255607 mapwrite 0x202000 thru 0x211aa2 (0xfaa3 bytes) 000498[1] 1744535471.282397 write 0x24f000 thru 0x253c7a (0x4c7b bytes) 000500[1] 1744535471.287770 trunc from 0x86e000 to 0x235463 (0x638b9c bytes) All operations completed A-OK! PASS 16a (9s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 05:11:12 (1744535472) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.114442 s, 73.3 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.0932606 s, 89.9 MB/s Chance of close/open is 1 in 50 Seed set to 5472 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read 000003[1] 1744535473.131171 write 0x0b889d thru 0x0b9a4b (0x11af bytes) 000006[0] 1744535473.140905 mapread 0x0260ad thru 0x026e17 (0xd6b bytes) 000008[1] 1744535473.152766 read 0x0325f7 thru 0x032faa (0x9b4 bytes) 000014[0] 1744535473.172376 read 0x066126 thru 0x067416 (0x12f1 bytes) 000016[1] 1744535473.177460 write 0x22644f thru 0x2283eb (0x1f9d bytes) 000017[0] 1744535473.184455 mapread 0x0cfc08 thru 0x0d0755 (0xb4e bytes) truncating to largest ever: 0x399b8a 000019[1] 1744535473.198498 trunc from 0x2283ec to 0x399b89 (0x17179e bytes) 000021[0] 1744535473.210930 mapread 0x37b80f thru 0x37b81c (0xe bytes) truncating to largest ever: 0x9e188b 000022[1] 1744535473.219450 trunc from 0x399b8a to 0x9e188a (0x647d01 bytes) 000024[0] 1744535473.236596 mapwrite 0x6be9ae thru 0x6c071f (0x1d72 bytes) 000025[0] 1744535473.269210 read 0x0c1062 thru 0x0c19b3 (0x952 bytes) 000027[0] 1744535473.275443 read 0x08ddff thru 0x08dff5 (0x1f7 bytes) 000029[0] 1744535473.277130 read 0x0eadfb thru 0x0eb8f6 (0xafc bytes) 000032[0] 1744535473.287390 mapwrite 0x2642a8 thru 0x264966 (0x6bf bytes) 000034[0] 1744535473.307664 mapwrite 0x348d69 thru 0x34a6fc (0x1994 bytes) 000039[0] 1744535473.338679 mapwrite 0x639fb2 thru 0x63bb5b (0x1baa bytes) 000043[1] 1744535473.369404 mapwrite 0x99567a thru 0x996e8f (0x1816 bytes) 000044[0] 1744535473.390134 read 0x1c770d thru 0x1c96c4 (0x1fb8 bytes) 000046[1] 1744535473.394662 read 0x43ab3e thru 0x43af74 (0x437 bytes) 000047[1] 1744535473.401371 trunc from 0x9e188b to 0x69880e (0x34907c bytes) 000048[0] 1744535473.409372 mapread 0x30fb04 thru 0x310c44 (0x1141 bytes) 000049[0] 1744535473.416605 trunc from 0x69880f to 0x15d884 (0x53af8a bytes) 000050[1] 1744535473.427619 mapwrite 0x2cf710 thru 0x2d08fa (0x11eb bytes) 000051[1] 1744535473.453752 write 0x59d47c thru 0x59e907 (0x148c bytes) 000053[1] 1744535473.459750 mapread 0x13a909 thru 0x13ad2a (0x422 bytes) 000057[0] 1744535473.473773 mapwrite 0x1594e8 thru 0x15a328 (0xe41 bytes) 000061[1] 1744535473.508963 mapread 0x25433d thru 0x255c9a (0x195e bytes) 000062[1] 1744535473.517680 read 0x26d25e thru 0x26d376 (0x119 bytes) 000063[1] 1744535473.519132 mapread 0x0959fd thru 0x095d0d (0x311 bytes) 000066[1] 1744535473.526572 mapwrite 0x5b3bd2 thru 0x5b4b27 (0xf56 bytes) 000067[0] 1744535473.553365 mapread 0x0bb0f8 thru 0x0bcd47 (0x1c50 bytes) 000068[0] 1744535473.563743 mapread 0x398651 thru 0x399e32 (0x17e2 bytes) 000069[0] 1744535473.570609 trunc from 0x5b4b28 to 0x5374ac (0x7d67b bytes) 000070[1] 1744535473.580000 read 0x179642 thru 0x17b4f9 (0x1eb8 bytes) 000072[0] 1744535473.585206 write 0x35ca35 thru 0x35d3ba (0x986 bytes) 000073[0] 1744535473.594723 write 0x9a81b5 thru 0x9a8e18 (0xc64 bytes) 000074[1] 1744535473.596276 read 0x219607 thru 0x219923 (0x31d bytes) 000075[1] 1744535473.606479 write 0x8ebf73 thru 0x8ed899 (0x1927 bytes) 000076[0] 1744535473.612981 trunc from 0x9a8e19 to 0x423aae (0x58536a bytes) 000077[1] 1744535473.620408 trunc from 0x423aaf to 0x37d020 (0xa6a8e bytes) 000078[1] 1744535473.631191 read 0x358731 thru 0x35a0fb (0x19cb bytes) 000082[0] 1744535473.659383 write 0x48e97a thru 0x48f2c9 (0x950 bytes) 000083[0] 1744535473.665815 write 0x3401f9 thru 0x341c09 (0x1a11 bytes) 000085[1] 1744535473.684446 mapread 0x13270a thru 0x133923 (0x121a bytes) 000086[0] 1744535473.697167 read 0x225e3e thru 0x226e75 (0x1038 bytes) 000088[1] 1744535473.702916 trunc from 0x48f2ca to 0x06af61 (0x424368 bytes) 000093[1] 1744535473.726086 trunc from 0x06af62 to 0x33bcc6 (0x2d0d65 bytes) 000097[1] 1744535473.735847 mapwrite 0x34b4f5 thru 0x34c64d (0x1159 bytes) 000098[1] 1744535473.755981 trunc from 0x34c64e to 0x8ff602 (0x5b2fb5 bytes) 000099[1] 1744535473.764527 trunc from 0x8ff603 to 0x7e15a0 (0x11e062 bytes) 000100[0] 1744535473.772006 mapread 0x71f50f thru 0x71f82a (0x31c bytes) 000103[1] 1744535473.782058 write 0x5bbdf2 thru 0x5bc1ae (0x3bd bytes) 000104[1] 1744535473.811225 read 0x0555d7 thru 0x055dbd (0x7e7 bytes) 000106[1] 1744535473.816512 write 0x423a5c thru 0x42579a (0x1d3f bytes) 000107[0] 1744535473.817238 trunc from 0x7e15a1 to 0x8a50dd (0xc3b3d bytes) 000115[0] 1744535473.848093 read 0x52a9f5 thru 0x52c34a (0x1956 bytes) 000116[0] 1744535473.853276 trunc from 0x8a50de to 0x12b882 (0x77985b bytes) 000117[0] 1744535473.860693 mapwrite 0x231d94 thru 0x233b5a (0x1dc7 bytes) 000118[1] 1744535473.886668 trunc from 0x233b5b to 0x053234 (0x1e0926 bytes) 000119[0] 1744535473.894593 read 0x04269a thru 0x042fa0 (0x907 bytes) 000120[1] 1744535473.906321 write 0x48a4d6 thru 0x48a75f (0x28a bytes) 000126[0] 1744535473.918507 trunc from 0x48a760 to 0x88d893 (0x403134 bytes) 000128[0] 1744535473.935527 mapwrite 0x9b26f6 thru 0x9b383f (0x114a bytes) 000129[0] 1744535473.978525 mapwrite 0x15f188 thru 0x160f16 (0x1d8f bytes) 000133[1] 1744535474.004483 mapread 0x8b0166 thru 0x8b153f (0x13da bytes) 000134[0] 1744535474.017791 write 0x9a2088 thru 0x9a3ce7 (0x1c60 bytes) 000135[1] 1744535474.036603 mapread 0x0284ba thru 0x02863e (0x185 bytes) 000136[0] 1744535474.045134 trunc from 0x9b3840 to 0x67f9be (0x333e81 bytes) 000141[1] 1744535474.059299 read 0x2798f0 thru 0x27a106 (0x817 bytes) 000149[1] 1744535474.074770 mapwrite 0x0957fb thru 0x096fbf (0x17c5 bytes) 000150[0] 1744535474.099332 mapread 0x2f7815 thru 0x2f933d (0x1b29 bytes) 000151[0] 1744535474.110303 mapwrite 0x707982 thru 0x70892d (0xfac bytes) 000152[0] 1744535474.137693 mapread 0x499b6a thru 0x49a8eb (0xd82 bytes) 000154[0] 1744535474.139348 mapread 0x113eb9 thru 0x1154ca (0x1612 bytes) 000156[0] 1744535474.146903 trunc from 0x70892e to 0x32de71 (0x3daabc bytes) 000160[1] 1744535474.163457 write 0x87f8d1 thru 0x880cb1 (0x13e1 bytes) 000163[0] 1744535474.175516 mapwrite 0x7b3608 thru 0x7b3d18 (0x711 bytes) 000164[1] 1744535474.195607 trunc from 0x880cb2 to 0x2f53dd (0x58b8d4 bytes) 000169[1] 1744535474.207751 mapwrite 0x6fca71 thru 0x6fdfa5 (0x1535 bytes) 000175[0] 1744535474.243844 mapread 0x08a045 thru 0x08b355 (0x1311 bytes) 000177[0] 1744535474.256505 read 0x53f530 thru 0x53fe52 (0x923 bytes) 000178[1] 1744535474.262476 write 0x955792 thru 0x955e04 (0x673 bytes) 000179[1] 1744535474.267915 trunc from 0x955e05 to 0x44579e (0x510666 bytes) 000181[1] 1744535474.279062 mapread 0x3df0dc thru 0x3df439 (0x35e bytes) 000183[0] 1744535474.282177 mapread 0x24e56f thru 0x24fcf0 (0x1782 bytes) 000185[1] 1744535474.291940 read 0x334fd0 thru 0x336bcf (0x1c00 bytes) 000186[0] 1744535474.296137 write 0x4085b1 thru 0x40875e (0x1ae bytes) 000188[1] 1744535474.299348 mapwrite 0x0af3bb thru 0x0b015e (0xda4 bytes) 000190[1] 1744535474.323530 read 0x0021e9 thru 0x003275 (0x108d bytes) 000193[0] 1744535474.334119 mapwrite 0x076587 thru 0x07835e (0x1dd8 bytes) 000194[1] 1744535474.353941 write 0x1ddd9f thru 0x1dde4a (0xac bytes) 000195[1] 1744535474.359985 mapread 0x18ef0d thru 0x18fe38 (0xf2c bytes) 000197[1] 1744535474.370544 mapread 0x15004b thru 0x1515db (0x1591 bytes) 000198[1] 1744535474.377832 mapread 0x25c8f6 thru 0x25da83 (0x118e bytes) 000200[1] 1744535474.387468 mapwrite 0x16276d thru 0x1639e9 (0x127d bytes) 000201[1] 1744535474.411820 mapread 0x383bec thru 0x385017 (0x142c bytes) 000204[0] 1744535474.434590 mapwrite 0x16c172 thru 0x16c863 (0x6f2 bytes) 000205[1] 1744535474.457408 write 0x3f71d8 thru 0x3f8e77 (0x1ca0 bytes) 000206[1] 1744535474.469445 mapwrite 0x5ac516 thru 0x5adaae (0x1599 bytes) 000208[0] 1744535474.502150 trunc from 0x5adaaf to 0x4150c9 (0x1989e5 bytes) 000209[1] 1744535474.515089 trunc from 0x4150ca to 0x72100a (0x30bf41 bytes) 000212[1] 1744535474.532934 write 0x58a407 thru 0x58a6ef (0x2e9 bytes) 000213[0] 1744535474.541984 read 0x45becc thru 0x45c905 (0xa3a bytes) 000217[0] 1744535474.557262 write 0x135b55 thru 0x135b67 (0x13 bytes) 000219[1] 1744535474.563251 mapread 0x5a0bf6 thru 0x5a2550 (0x195b bytes) 000220[0] 1744535474.572828 mapwrite 0x05ef3e thru 0x05f8f1 (0x9b4 bytes) 000223[1] 1744535474.605068 mapwrite 0x919d9e thru 0x91a167 (0x3ca bytes) 000227[0] 1744535474.634199 read 0x03a026 thru 0x03a1e2 (0x1bd bytes) 000228[0] 1744535474.637761 mapread 0x41af94 thru 0x41c772 (0x17df bytes) 000230[0] 1744535474.645316 trunc from 0x91a168 to 0x94513f (0x2afd8 bytes) 000231[0] 1744535474.655666 trunc from 0x945140 to 0x32128e (0x623eb1 bytes) 000232[1] 1744535474.662399 mapread 0x293db1 thru 0x29479e (0x9ee bytes) 000234[0] 1744535474.674064 read 0x249c2c thru 0x249fe1 (0x3b6 bytes) 000236[1] 1744535474.681744 mapread 0x2b422e thru 0x2b4c62 (0xa35 bytes) 000239[1] 1744535474.686179 trunc from 0x32128f to 0x7c04e2 (0x49f254 bytes) 000240[0] 1744535474.695523 mapread 0x739612 thru 0x73af18 (0x1907 bytes) 000241[1] 1744535474.705597 trunc from 0x7c04e3 to 0x354214 (0x46c2ce bytes) 000243[1] 1744535474.716388 trunc from 0x354215 to 0x96a755 (0x616541 bytes) 000245[1] 1744535474.727421 read 0x89e055 thru 0x89eb0d (0xab9 bytes) 000246[0] 1744535474.729083 mapwrite 0x7a2e99 thru 0x7a3647 (0x7af bytes) 000247[1] 1744535474.750556 mapwrite 0x9e621b thru 0x9e6def (0xbd5 bytes) 000248[1] 1744535474.772173 read 0x182c6e thru 0x183dbf (0x1152 bytes) 000249[1] 1744535474.777913 write 0x2a5d36 thru 0x2a7189 (0x1454 bytes) 000251[1] 1744535474.780673 mapwrite 0x4105d2 thru 0x412081 (0x1ab0 bytes) 000253[0] 1744535474.800785 read 0x40668c thru 0x407e3f (0x17b4 bytes) 000256[1] 1744535474.808938 mapread 0x413768 thru 0x4147a2 (0x103b bytes) 000257[0] 1744535474.811313 mapread 0x0cdb5d thru 0x0ce905 (0xda9 bytes) 000260[0] 1744535474.820005 read 0x8be1a8 thru 0x8bf9ee (0x1847 bytes) 000261[0] 1744535474.821616 read 0x198834 thru 0x1996fb (0xec8 bytes) 000262[1] 1744535474.823290 mapwrite 0x1bd3a9 thru 0x1be63b (0x1293 bytes) 000263[1] 1744535474.843524 trunc from 0x9e6df0 to 0x0941d1 (0x952c1e bytes) 000264[1] 1744535474.849704 trunc from 0x0941d2 to 0x6b937c (0x6251ab bytes) 000265[0] 1744535474.856947 write 0x8ba09f thru 0x8bad10 (0xc72 bytes) 000266[0] 1744535474.862826 mapread 0x0bcb60 thru 0x0bdb33 (0xfd4 bytes) 000267[1] 1744535474.872330 write 0x695cd6 thru 0x6976c8 (0x19f3 bytes) 000270[0] 1744535474.884521 mapread 0x01faa7 thru 0x01fde0 (0x33a bytes) 000271[0] 1744535474.893287 mapread 0x3df63f thru 0x3df66b (0x2d bytes) 000272[1] 1744535474.898768 write 0x32b2d9 thru 0x32c317 (0x103f bytes) 000273[0] 1744535474.904469 mapread 0x7d25ba thru 0x7d3212 (0xc59 bytes) 000274[1] 1744535474.917706 trunc from 0x8bad11 to 0x5afad7 (0x30b239 bytes) 000275[0] 1744535474.928230 mapwrite 0x18d24f thru 0x18ec36 (0x19e8 bytes) 000276[1] 1744535474.952221 mapread 0x42f6a3 thru 0x430b33 (0x1491 bytes) 000277[0] 1744535474.959076 mapwrite 0x1674b8 thru 0x168190 (0xcd9 bytes) 000279[1] 1744535474.979357 trunc from 0x5afad8 to 0x5e4e55 (0x3537e bytes) 000280[1] 1744535474.989160 read 0x0fbc56 thru 0x0fc976 (0xd21 bytes) 000282[0] 1744535474.991709 mapread 0x0a01c8 thru 0x0a11cc (0x1005 bytes) 000285[1] 1744535475.004486 mapread 0x4ef31c thru 0x4ef89d (0x582 bytes) 000286[0] 1744535475.006124 mapwrite 0x1b3d5a thru 0x1b4b46 (0xded bytes) 000289[1] 1744535475.033626 write 0x3a0513 thru 0x3a2408 (0x1ef6 bytes) 000291[0] 1744535475.042339 write 0x03c1d9 thru 0x03d092 (0xeba bytes) 000292[0] 1744535475.050937 write 0x45aff8 thru 0x45bc31 (0xc3a bytes) 000293[1] 1744535475.058418 mapwrite 0x8a6fda thru 0x8a7bb7 (0xbde bytes) 000297[1] 1744535475.088848 mapread 0x7abe92 thru 0x7ad5a4 (0x1713 bytes) 000298[0] 1744535475.091556 mapwrite 0x07f251 thru 0x07f6e6 (0x496 bytes) 000303[1] 1744535475.120977 trunc from 0x8a7bb8 to 0x37b061 (0x52cb56 bytes) 000305[1] 1744535475.131052 read 0x14014d thru 0x1403f6 (0x2aa bytes) 000307[1] 1744535475.158360 mapwrite 0x7a75d4 thru 0x7a8c78 (0x16a5 bytes) 000308[1] 1744535475.199950 write 0x30657c thru 0x30702f (0xab4 bytes) 000309[0] 1744535475.203520 mapread 0x298267 thru 0x298b62 (0x8fc bytes) 000310[1] 1744535475.215524 mapread 0x5bb23c thru 0x5bcc0d (0x19d2 bytes) 000311[0] 1744535475.219171 mapwrite 0x6af714 thru 0x6b099f (0x128c bytes) 000312[0] 1744535475.240191 write 0x30e530 thru 0x30f722 (0x11f3 bytes) 000317[0] 1744535475.254595 read 0x6b39f5 thru 0x6b3b80 (0x18c bytes) 000318[1] 1744535475.259232 read 0x25ad59 thru 0x25afc6 (0x26e bytes) 000322[0] 1744535475.270441 write 0x3f2d0c thru 0x3f35c7 (0x8bc bytes) 000323[1] 1744535475.274589 trunc from 0x7a8c79 to 0x621f3d (0x186d3b bytes) 000326[0] 1744535475.287134 write 0x4e5291 thru 0x4e5673 (0x3e3 bytes) 000327[0] 1744535475.293746 write 0x9c7886 thru 0x9c8f4d (0x16c8 bytes) 000328[0] 1744535475.296823 mapwrite 0x6e7938 thru 0x6e79ba (0x83 bytes) 000331[1] 1744535475.314535 write 0x67f007 thru 0x67fccf (0xcc9 bytes) 000336[0] 1744535475.331899 write 0x9f5cf7 thru 0x9f7ab5 (0x1dbf bytes) 000341[1] 1744535475.346665 write 0x6da831 thru 0x6dc5e6 (0x1db6 bytes) 000343[0] 1744535475.358897 mapwrite 0x8f8fa7 thru 0x8fa0dc (0x1136 bytes) 000347[1] 1744535475.387196 read 0x8641fb thru 0x86436f (0x175 bytes) 000353[0] 1744535475.399395 trunc from 0x9f7ab6 to 0x3d752e (0x620587 bytes) 000355[0] 1744535475.417413 mapread 0x2e2876 thru 0x2e2e3a (0x5c5 bytes) 000356[1] 1744535475.422870 read 0x25dcfc thru 0x25dfae (0x2b3 bytes) 000357[1] 1744535475.448619 trunc from 0x3d752f to 0x5f1718 (0x21a1ea bytes) 000359[0] 1744535475.459397 mapread 0x5edfdc thru 0x5eeb64 (0xb89 bytes) 000360[0] 1744535475.465864 read 0x4f6292 thru 0x4f73f0 (0x115f bytes) 000364[0] 1744535475.471652 trunc from 0x5f1719 to 0x26d783 (0x383f95 bytes) 000367[1] 1744535475.483328 write 0x8cae49 thru 0x8cc89a (0x1a52 bytes) 000368[1] 1744535475.489734 write 0x91d9c0 thru 0x91e720 (0xd61 bytes) 000370[0] 1744535475.496249 mapread 0x869a00 thru 0x86a650 (0xc51 bytes) 000373[1] 1744535475.513000 write 0x17d2d2 thru 0x17dc6e (0x99d bytes) 000374[0] 1744535475.521644 mapwrite 0x99d468 thru 0x99d766 (0x2ff bytes) 000376[1] 1744535475.548576 mapread 0x1e82f0 thru 0x1e931e (0x102f bytes) 000378[1] 1744535475.558602 write 0x7ef924 thru 0x7f13b2 (0x1a8f bytes) 000379[0] 1744535475.583448 mapread 0x761fb5 thru 0x762207 (0x253 bytes) 000381[0] 1744535475.595716 read 0x21cce8 thru 0x21db7f (0xe98 bytes) 000382[1] 1744535475.597275 mapwrite 0x394e3f thru 0x395cbb (0xe7d bytes) 000383[0] 1744535475.613771 trunc from 0x99d767 to 0x14e875 (0x84eef1 bytes) 000384[0] 1744535475.621918 write 0x01d5ee thru 0x01e1f9 (0xc0c bytes) 000386[0] 1744535475.624514 trunc from 0x14e876 to 0x608955 (0x4ba0e0 bytes) 000391[1] 1744535475.644105 read 0x15d9b7 thru 0x15e4d5 (0xb1f bytes) 000393[1] 1744535475.652378 read 0x3716f6 thru 0x37185d (0x168 bytes) 000394[1] 1744535475.656156 write 0x80ffda thru 0x81022d (0x254 bytes) 000395[0] 1744535475.661875 write 0x2bb178 thru 0x2bc9c6 (0x184f bytes) 000397[1] 1744535475.670624 mapread 0x04519b thru 0x045929 (0x78f bytes) 000401[0] 1744535475.680613 mapwrite 0x591bf0 thru 0x591d17 (0x128 bytes) 000405[0] 1744535475.704238 mapread 0x46fba4 thru 0x470c41 (0x109e bytes) 000406[0] 1744535475.709488 write 0x531b56 thru 0x5336fe (0x1ba9 bytes) 000410[0] 1744535475.718888 write 0x4d2f94 thru 0x4d32cb (0x338 bytes) 000411[1] 1744535475.725002 mapwrite 0x7fd826 thru 0x7fe0fd (0x8d8 bytes) 000412[0] 1744535475.750181 read 0x13ce7c thru 0x13e1f4 (0x1379 bytes) 000415[1] 1744535475.758531 trunc from 0x81022e to 0x95b36a (0x14b13d bytes) 000416[0] 1744535475.770914 mapread 0x604fa9 thru 0x6061da (0x1232 bytes) 000418[0] 1744535475.786088 mapread 0x28ec43 thru 0x28ff5f (0x131d bytes) 000424[0] 1744535475.794159 write 0x28634f thru 0x2874a2 (0x1154 bytes) 000425[1] 1744535475.798687 mapwrite 0x0eee6d thru 0x0ef13b (0x2cf bytes) 000427[1] 1744535475.824384 mapread 0x026e74 thru 0x027680 (0x80d bytes) 000428[1] 1744535475.828353 mapread 0x945b5b thru 0x946be5 (0x108b bytes) 000429[1] 1744535475.835854 read 0x86f72f thru 0x86f8d0 (0x1a2 bytes) 000430[1] 1744535475.840457 mapread 0x896177 thru 0x8977c8 (0x1652 bytes) 000433[1] 1744535475.858310 mapwrite 0x4bb200 thru 0x4bb3b5 (0x1b6 bytes) 000434[0] 1744535475.876371 mapread 0x8d0848 thru 0x8d2743 (0x1efc bytes) 000436[0] 1744535475.890579 write 0x8cfffa thru 0x8d02ff (0x306 bytes) 000438[1] 1744535475.900612 mapwrite 0x969f5e thru 0x96aaea (0xb8d bytes) 000445[0] 1744535475.933890 trunc from 0x96aaeb to 0x284c97 (0x6e5e53 bytes) 000448[0] 1744535475.950231 read 0x148ca4 thru 0x14a7d2 (0x1b2f bytes) 000450[0] 1744535475.968191 read 0x21578a thru 0x216a76 (0x12ed bytes) 000451[0] 1744535475.971960 mapwrite 0x52075f thru 0x5209e1 (0x283 bytes) 000454[1] 1744535475.995634 mapwrite 0x2839f5 thru 0x28449b (0xaa7 bytes) 000457[1] 1744535476.029562 mapread 0x3b7ed8 thru 0x3b80f7 (0x220 bytes) 000459[1] 1744535476.040739 trunc from 0x5209e2 to 0x5b5a95 (0x950b4 bytes) 000461[1] 1744535476.048061 trunc from 0x5b5a96 to 0x5c9eab (0x14416 bytes) 000462[1] 1744535476.051600 write 0x417efe thru 0x418c11 (0xd14 bytes) 000464[1] 1744535476.055183 write 0x4f9293 thru 0x4f9623 (0x391 bytes) 000467[0] 1744535476.067994 mapwrite 0x828d4f thru 0x829af1 (0xda3 bytes) 000468[0] 1744535476.104326 read 0x13da6c thru 0x13de23 (0x3b8 bytes) 000469[1] 1744535476.110162 write 0x3718f7 thru 0x373833 (0x1f3d bytes) 000471[1] 1744535476.120324 mapread 0x573858 thru 0x574b42 (0x12eb bytes) 000474[1] 1744535476.132465 trunc from 0x829af2 to 0x9b4453 (0x18a962 bytes) 000475[0] 1744535476.141449 mapwrite 0x33de6e thru 0x33df1b (0xae bytes) 000477[1] 1744535476.165832 mapwrite 0x74ce9c thru 0x74de52 (0xfb7 bytes) 000481[1] 1744535476.197466 write 0x3ed7d2 thru 0x3ee84d (0x107c bytes) 000483[1] 1744535476.205508 mapwrite 0x7c57e6 thru 0x7c5f2f (0x74a bytes) 000485[1] 1744535476.222688 read 0x10bd64 thru 0x10c8af (0xb4c bytes) 000487[1] 1744535476.224335 trunc from 0x9b4454 to 0x49e477 (0x515fdc bytes) 000489[1] 1744535476.229009 read 0x41ffd0 thru 0x4213ac (0x13dd bytes) 000492[1] 1744535476.238072 mapwrite 0x380519 thru 0x38061a (0x102 bytes) 000495[1] 1744535476.258596 mapwrite 0x0acebd thru 0x0aea9e (0x1be2 bytes) 000498[0] 1744535476.280774 write 0x2e48e5 thru 0x2e6410 (0x1b2c bytes) 000499[1] 1744535476.289334 mapread 0x24cc79 thru 0x24e637 (0x19bf bytes) All operations completed A-OK! PASS 16b (5s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 05:11:17 (1744535477) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 05:11:18 (1744535478) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.120159 s, 69.8 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.0957099 s, 87.6 MB/s ldlm.namespaces.MGC192.168.203.131@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff88012a612800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff88012e275800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff88012a612800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff88012e275800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff88012a612800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff88012e275800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.57724 s, 40.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.785179 s, 134 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.572278 s, 183 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.28875 s, 81.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.6254 s, 64.5 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.82855 s, 127 MB/s PASS 16d (10s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 05:11:28 (1744535488) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0573356 s, 183 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278272 s, 37.7 MB/s PASS 16e (2s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 05:11:30 (1744535490) TIMEOUT PASS 16f (22s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 05:11:52 (1744535512) TIMEOUT PASS 16g (21s) == sanityn test 16h: mmap read after truncate file ======= 05:12:13 (1744535533) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16h (2s) == sanityn test 16i: read after truncate file ============ 05:12:15 (1744535535) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (1s) == sanityn test 16j: race dio with buffered i/o ========== 05:12:16 (1744535536) 1+0 records in 1+0 records out 26214400 bytes (26 MB) copied, 0.326466 s, 80.3 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.270407 s, 96.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.548908 s, 47.8 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.152612 s, 172 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.482326 s, 54.3 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.150825 s, 174 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.519828 s, 50.4 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.421468 s, 62.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.435328 s, 60.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.604202 s, 43.4 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.687005 s, 38.2 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.375117 s, 69.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.426666 s, 61.4 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.604873 s, 43.3 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.642296 s, 40.8 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.235962 s, 111 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.311403 s, 84.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.610344 s, 43.0 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.631269 s, 41.5 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (8s) == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 05:12:24 (1744535544) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x32cce75 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported truncating to largest ever: 0x1a7062c fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x232e1d3 skipping zero size read truncating to largest ever: 0x3c569db truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 All operations completed A-OK! Sun Apr 13 05:12:31 EDT 2025 + fsx 2 done: rc=0 Sun Apr 13 05:12:31 EDT 2025 + fsx 3 done: rc=0 Sun Apr 13 05:12:31 EDT 2025 All operations completed A-OK! + fsx 4 done: rc=0 Sun Apr 13 05:12:32 EDT 2025 + fsx 5 done: rc=0 Sun Apr 13 05:12:32 EDT 2025 PASS 16k (9s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 05:12:34 (1744535554) fail_loc=0x8000030a PASS 17 (4s) == sanityn test 18: mmap sanity check =========================================================================================== 05:12:37 (1744535557) mmap test1: basic mmap operation (PASS, 0.025395s) mmap test2: MAP_PRIVATE not write back (PASS, 0.021689s) mmap test3: concurrent mmap ops on two nodes (PASS, 4.9432s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 4.0848s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.0317s) mmap test6: check mmap write/read content on two nodes (PASS, 0.037822s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: multi-thread mmap access (PASS, 0.30573s) mmap test11: multi-thread shared mmap (PASS, 0.31291s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0123s) PASS 18 (14s) == sanityn test 19: test concurrent uncached read races ========================================================================= 05:12:51 (1744535571) oleg331-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg331-client: oleg331-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 05:12:52 (1744535572) PASS 20 (1s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 05:12:53 (1744535573) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) == sanityn test 23: others should see updated atime while another read============================================================== 05:12:55 (1744535575) now is 1744535575 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 new atime is 1744535636 PASS 23 (62s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 05:13:57 (1744535637) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 33792 3735552 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 33792 3735552 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.3K 402 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 114.5K 399 114.1K 1% /mnt/lustre[OST:1] filesystem_summary: 229.3K 387 229.0K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 402 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 399 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.3K 387 229.0K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 33.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 39.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118066 402 117664 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117199 399 116800 1% /mnt/lustre2[OST:1] filesystem_summary: 234851 387 234464 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 33792 3735552 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 402 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 399 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.3K 387 229.0K 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre[MDT:0] filesystem_summary: 502378 387 501991 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118066 402 117664 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117199 399 116800 1% /mnt/lustre2[OST:1] filesystem_summary: 234851 387 234464 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 33792 3735552 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 33792 3735552 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre2 PASS 24a (2s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 05:13:59 (1744535639) PASS 24b (1s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 05:14:00 (1744535640) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (2s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 05:14:02 (1744535642) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 05:14:03 (1744535643) utime: good mknod times 1744535642 <= 1744535643 <= 1744535643 for /mnt/lustre/f26a utime: good mknod times 1744535642 <= 1744535643 <= 1744535643 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (2s) == sanityn test 26b: sync mtime between ost and mds ====== 05:14:05 (1744535645) PASS 26b (4s) == sanityn test 26c: set-in-past on open file is not lost on close ========================================================== 05:14:09 (1744535649) multiop /mnt/lustre/f26c.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 File: '/mnt/lustre/f26c.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2025-04-13 05:14:09.000000000 -0400 Birth: - File: '/mnt/lustre2/f26c.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2025-04-13 05:14:09.000000000 -0400 Birth: - PASS 26c (2s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 05:14:12 (1744535652) 0 PASS 30 (3s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 05:14:15 (1744535655) fail_loc=0x314 PASS 31a (3s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 05:14:18 (1744535658) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg331-server mds-ost sync done. sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.203.131@tcp:/lustre 7542784 39936 7496704 1% /mnt/lustre2 PASS 31b (17s) == sanityn test 31r: open-rename(replace) race =========== 05:14:35 (1744535675) -rw-r--r-- 1 root root 0 Apr 13 05:14 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 mv: failed to access '/mnt/lustre2/f31r.sanityn.target': Input/output error PASS 31r (4s) == sanityn test 31s: open should not revalidate invalid dentry ========================================================== 05:14:40 (1744535680) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 1213: 13503 Terminated tail -f $tfile PASS 31s (3s) == sanityn test 31t: getattr should not revalidate invalid dentry ========================================================== 05:14:42 (1744535682) PASS 31t (2s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 05:14:44 (1744535684) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 05:14:45 (1744535685) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: dependent transactions should trigger COS ========================================================== 05:14:46 (1744535686) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (0s) == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 05:14:47 (1744535687) SKIP: sanityn test_33e needs >= 2 MDTs SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 05:14:47 (1744535687) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg331-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88012a612800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff88012a612800.ost_server_uuid in FULL state after 0 sec oleg331-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff88012a612800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff88012a612800.ost_server_uuid in IDLE state after 0 sec oleg331-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88012a612800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff88012a612800.ost_server_uuid in FULL state after 0 sec oleg331-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff88012a612800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff88012a612800.ost_server_uuid in IDLE state after 0 sec oleg331-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88012a612800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff88012a612800.ost_server_uuid in FULL state after 0 sec oleg331-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff88012a612800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff88012a612800.ost_server_uuid in IDLE state after 0 sec PASS 34 (86s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 05:16:13 (1744535773) fail_loc=0x80000317 Race attempt 0 Wait for 20139 20278 for 60 sec... total: 4000 open/close in 7.15 seconds: 559.74 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.203.131@tcp:/lustre 7.2G 39M 7.2G 1% /mnt/lustre 192.168.203.131@tcp:/lustre 7.2G 39M 7.2G 1% /mnt/lustre2 PASS 35 (63s) == sanityn test 36: handle ESTALE/open-unlink correctly == 05:17:16 (1744535836) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg331-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.280018 s, 187 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7498752) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.251345 s, 209 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7496704) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.256783 s, 204 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7494656) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.26509 s, 198 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7498752) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.256304 s, 205 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7496704) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.244231 s, 215 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7498752) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.269195 s, 195 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7496704) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.276418 s, 190 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7498752) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.26475 s, 198 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7496704) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.276456 s, 190 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7496704) after_dd(7447552) after(7498752) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.246575 s, 213 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7498752) after_dd(7443456) after(7494656) PASS 36 (229s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 05:21:05 (1744536065) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 - create 8635 (time 1744536076.93 total 10.00 last 863.46) total: 10000 create in 11.82 seconds: 846.18 ops/second PASS 37 (15s) == sanityn test 39a: file mtime does not change after rename ========================================================== 05:21:20 (1744536080) repeat after cancel_lru_locks PASS 39a (2s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 05:21:22 (1744536082) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 05:21:25 (1744536085) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 05:21:28 (1744536088) fail_loc=0x411 fail_loc=0 PASS 39d (1s) == sanityn test 40a: pdirops: create vs others ======================================================================== 05:21:29 (1744536089) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (8s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 05:21:37 (1744536097) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (9s) == sanityn test 40c: pdirops: link and others ======================================================================== 05:21:46 (1744536106) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (7s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 05:21:53 (1744536113) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (8s) == sanityn test 40e: pdirops: rename and others ======================================================================== 05:22:01 (1744536121) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (7s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 05:22:08 (1744536128) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (5s) == sanityn test 41b: pdirops: create vs create ======================================================================== 05:22:13 (1744536133) open: File exists Conflict PASS 41b (5s) == sanityn test 41c: pdirops: create vs link ======================================================================== 05:22:18 (1744536138) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (6s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 05:22:24 (1744536144) Conflict PASS 41d (5s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 05:22:29 (1744536149) Conflict PASS 41e (5s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 05:22:34 (1744536154) Conflict PASS 41f (5s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 05:22:39 (1744536159) Conflict PASS 41g (5s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 05:22:44 (1744536164) Conflict PASS 41h (4s) == sanityn test 41i: reint_open: create vs create ======== 05:22:48 (1744536168) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (673s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 05:34:01 (1744536841) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (4s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 05:34:05 (1744536845) open: File exists Conflict PASS 42b (4s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 05:34:09 (1744536849) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (4s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 05:34:13 (1744536853) Conflict PASS 42d (5s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 05:34:18 (1744536858) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (4s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 05:34:22 (1744536862) Conflict PASS 42f (4s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 05:34:27 (1744536867) Conflict PASS 42g (6s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 05:34:32 (1744536872) Conflict PASS 42h (5s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 05:34:37 (1744536877) PASS 43a (19s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 05:34:56 (1744536896) Conflict PASS 43b (5s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 05:35:01 (1744536901) Conflict PASS 43c (6s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 05:35:07 (1744536907) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (6s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 05:35:13 (1744536913) Conflict PASS 43e (5s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 05:35:18 (1744536918) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (6s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 05:35:24 (1744536924) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (6s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 05:35:30 (1744536930) Conflict PASS 43h (5s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 05:35:35 (1744536935) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 05:35:36 (1744536936) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (44s) == sanityn test 43k: unlink vs create ==================== 05:36:20 (1744536980) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (414s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 05:43:14 (1744537394) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Sun Apr 13 05:43:18 EDT 2025 PASS 44a (6s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 05:43:20 (1744537400) open: File exists Conflict PASS 44b (5s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 05:43:25 (1744537405) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (6s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 05:43:31 (1744537411) Conflict PASS 44d (6s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 05:43:37 (1744537417) Conflict PASS 44e (5s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 05:43:42 (1744537422) Conflict PASS 44f (6s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 05:43:48 (1744537428) Conflict PASS 44g (6s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 05:43:54 (1744537434) Conflict PASS 44h (5s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 05:43:59 (1744537439) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 05:44:00 (1744537440) PASS 45a (39s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 05:44:39 (1744537479) Conflict PASS 45b (5s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 05:44:44 (1744537484) Conflict PASS 45c (6s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 05:44:50 (1744537490) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (5s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 05:44:55 (1744537495) Conflict PASS 45e (6s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 05:45:01 (1744537501) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (5s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 05:45:06 (1744537506) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (5s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 05:45:11 (1744537511) Conflict PASS 45h (5s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 05:45:16 (1744537516) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 05:45:17 (1744537517) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (481s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 05:53:18 (1744537998) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (6s) == sanityn test 46b: pdirops: link vs create ======================================================================== 05:53:24 (1744538004) open: File exists Conflict PASS 46b (6s) == sanityn test 46c: pdirops: link vs link ======================================================================== 05:53:30 (1744538010) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (6s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 05:53:36 (1744538016) Conflict PASS 46d (5s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 05:53:41 (1744538021) Conflict PASS 46e (6s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 05:53:47 (1744538027) Conflict PASS 46f (6s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 05:53:53 (1744538033) Conflict PASS 46g (5s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 05:53:59 (1744538039) Conflict PASS 46h (6s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 05:54:04 (1744538044) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 05:54:06 (1744538046) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (2s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 05:54:07 (1744538047) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 05:54:08 (1744538048) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 05:54:09 (1744538049) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 05:54:10 (1744538050) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 05:54:11 (1744538051) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 05:54:13 (1744538053) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (2s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 05:54:14 (1744538054) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.00721564 s, 1.4 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 05:54:21 (1744538061) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00723185 s, 22.0 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3519: kill: (11344) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 05:54:25 (1744538065) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00497227 s, 206 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023495 s, 44.6 MB/s 3145728 PASS 51b (15s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 05:54:40 (1744538080) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00697155 s, 147 kB/s PASS 51c (7s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 05:54:47 (1744538087) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161915 s, 64.8 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (3s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 05:54:50 (1744538090) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2452 0x994 0x280000400 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2452 0x994 0x280000400 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 05:54:54 (1744538094) llite.lustre-ffff88012a612800.xattr_cache=1 llite.lustre-ffff88012e275800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505012 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-04-13 05:54:55.000000000 -0400 Modify: 2025-04-13 05:54:55.000000000 -0400 Change: 2025-04-13 05:54:55.000000000 -0400 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505015 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-04-13 05:55:00.000000000 -0400 Modify: 2025-04-13 05:55:01.000000000 -0400 Change: 2025-04-13 05:55:01.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505018 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-04-13 05:55:06.000000000 -0400 Modify: 2025-04-13 05:55:06.000000000 -0400 Change: 2025-04-13 05:55:06.000000000 -0400 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505017 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-04-13 05:55:06.000000000 -0400 Modify: 2025-04-13 05:55:06.000000000 -0400 Change: 2025-04-13 05:55:06.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505020 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-04-13 05:55:11.000000000 -0400 Modify: 2025-04-13 05:55:12.000000000 -0400 Change: 2025-04-13 05:55:12.000000000 -0400 Birth: - File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505021 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-04-13 05:55:11.000000000 -0400 Modify: 2025-04-13 05:55:12.000000000 -0400 Change: 2025-04-13 05:55:12.000000000 -0400 Birth: - PASS 54 (25s) == sanityn test 55a: rename vs unlink target dir ========= 05:55:19 (1744538119) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 05:55:26 (1744538126) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (8s) == sanityn test 55c: rename vs unlink orphan target dir == 05:55:34 (1744538134) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (13s) == sanityn test 55d: rename file vs link ================= 05:55:47 (1744538147) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (10s) == sanityn test 56a: test llverdev with single large stripe ========================================================== 05:55:57 (1744538157) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1744538158 chunksize: 4096 size: 16777216000000 write offset: 16384000000kB 1.79527 MB/s read offset: 16384000000kB inf MB/s llverdev: data verified successfully PASS 56a (37s) == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 05:56:34 (1744538194) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1744538195 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 4.35979 MB/s read offset: 32768000000kB -nan MB/s llverdev: data verified successfully PASS 56b (33s) == sanityn test 60: Verify data_version behaviour ======== 05:57:07 (1744538227) PASS 60 (2s) == sanityn test 70a: cd directory && rm directory ======== 05:57:09 (1744538229) PASS 70a (3s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 05:57:12 (1744538232) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 05:57:14 (1744538234) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00371154 s, 17.7 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 05:57:15 (1744538235) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00483431 s, 13.6 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (2s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 05:57:17 (1744538237) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 05:57:18 (1744538238) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00607512 s, 10.8 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 05:57:19 (1744538239) llite.lustre-ffff88012a612800.xattr_cache=1 llite.lustre-ffff88012e275800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (3s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 05:57:22 (1744538242) llite.lustre-ffff88012a612800.xattr_cache=1 llite.lustre-ffff88012e275800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (2s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 05:57:24 (1744538244) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00464199 s, 221 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00397179 s, 258 kB/s 23791: get lock2 23792: get lock1 23791: done 23791: get lock1 23792: done 23792: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 23792: cannot get lock2: Resource deadlock avoided 23792: exit rc=0 23791: done 23791: exit rc=0 PASS 74 (7s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 05:57:31 (1744538251) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0264243 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142562 s, 73.6 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4108: 24433 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 05:57:38 (1744538258) opening files: ........ closing files: ........ PASS 76 (22s) == sanityn test 77a: check FIFO NRS policy =============== 05:58:00 (1744538280) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.158242 s, 106 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0179703 s, 58.4 MB/s , 0.0154796 s, 67.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160394 s, 65.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013656 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0370497 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037728 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361073 s, 29.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333295 s, 31.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0365056 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0320622 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00963119 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0287534 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0514627 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0381831 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0482058 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0514518 s, 20.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a612800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff88012a612800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out , 0.00892871 s, 117 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00973505 s, 108 MB/s , 0.0100002 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00648937 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00877598 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00542873 s, 193 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00679321 s, 154 MB/s , 0.00950431 s, 110 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00128195 s, 818 MB/s , 0.00120148 s, 873 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00125371 s, 836 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00871458 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114414 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00980763 s, 107 MB/s 1048576 bytes (1.0 MB) copied, 0.0120945 s, 86.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115465 s, 90.8 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg331-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (4s) == sanityn test 77b: check CRR-N NRS policy ============== 05:58:04 (1744538284) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.11407 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00663569 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00684182 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011924 s, 87.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143717 s, 73.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00608964 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00826548 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178413 s, 58.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170584 s, 61.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00661838 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170991 s, 61.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0455351 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0473296 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.046827 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0485024 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.065561 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0484627 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128605 s, 81.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0263598 s, 39.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0283423 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112251 s, 93.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140077 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138406 s, 75.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030358 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172563 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183557 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164268 s, 63.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176369 s, 59.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174169 s, 60.2 MB/s , 0.018154 s, 57.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177327 s, 59.1 MB/s , 0.0332203 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210969 s, 49.7 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg331-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.10928 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00987658 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119897 s, 87.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191212 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155944 s, 67.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123242 s, 85.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122917 s, 85.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147797 s, 70.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275831 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141921 s, 73.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261186 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0375734 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0320855 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0432216 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.040917 s, 25.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0241576 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274472 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00942409 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108596 s, 96.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144261 s, 72.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0098008 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115138 s, 91.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0146008 s, 71.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0188972 s, 55.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137341 s, 76.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147178 s, 71.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104165 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151563 s, 69.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160301 s, 65.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164288 s, 63.8 MB/s 1048576 bytes (1.0 MB) copied, 0.00561826 s, 187 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172149 s, 60.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188071 s, 55.8 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg331-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (4s) == sanityn test 77c: check ORR NRS policy ================ 05:58:08 (1744538288) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.134779 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160614 s, 65.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165111 s, 63.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423764 s, 24.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0450618 s, 23.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0463855 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0507826 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0416099 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.049839 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0391552 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0501805 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.041893 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0542522 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0486756 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0421506 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0432711 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0502303 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0129198 s, 81.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0143157 s, 73.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010767 s, 97.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116401 s, 90.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179006 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175365 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168182 s, 62.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145399 s, 72.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147207 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158828 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00762583 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172138 s, 60.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186095 s, 56.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0148754 s, 70.5 MB/s , 0.0166826 s, 62.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183536 s, 57.1 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg331-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.122355 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142614 s, 73.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161073 s, 65.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170261 s, 61.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144616 s, 72.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286377 s, 36.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0420961 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0535815 s, 19.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0541103 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0592163 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0580596 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0623621 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0576595 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0635146 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0506859 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0525243 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0510079 s, 20.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0167695 s, 62.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0173411 s, 60.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184305 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186376 s, 56.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0194994 s, 53.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199244 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265861 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.035072 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256162 s, 40.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0242881 s, 43.2 MB/s , 0.0241978 s, 43.3 MB/s , 0.0244561 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248482 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0446046 s, 23.5 MB/s 1048576 bytes (1.0 MB) copied, 0.02703 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284253 s, 36.9 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg331-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.134183 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155503 s, 67.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184553 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195911 s, 53.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0458653 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0529687 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0543932 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0394524 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0444184 s, 23.6 MB/s 1048576 bytes (1.0 MB) copied, 0.044298 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0497107 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0568973 s, 18.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0520893 s, 20.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0692522 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0533271 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0654984 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0662953 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176218 s, 59.5 MB/s , 0.0178802 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245022 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263565 s, 39.8 MB/s , 0.0168062 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195332 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199901 s, 52.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0223314 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216081 s, 48.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037412 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225867 s, 46.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0367868 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237519 s, 44.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235003 s, 44.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246162 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261353 s, 40.1 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg331-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (6s) == sanityn test 77d: check TRR nrs policy ================ 05:58:14 (1744538294) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.163313 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022362 s, 46.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0252516 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192339 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.039241 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280482 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0367437 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0402553 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0367057 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0489923 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0373434 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0617498 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.074892 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0640568 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0652936 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0650949 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.074204 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122275 s, 85.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122803 s, 85.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291303 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154732 s, 67.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291896 s, 35.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275081 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169354 s, 61.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148898 s, 70.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179968 s, 58.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288783 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169514 s, 61.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174115 s, 60.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0176981 s, 59.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0192782 s, 54.4 MB/s , 0.0217549 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203892 s, 51.4 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg331-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.119123 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119274 s, 87.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212681 s, 49.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106663 s, 98.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0244282 s, 42.9 MB/s 1048576 bytes (1.0 MB) copied, 0.019947 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00912247 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250504 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00570161 s, 184 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.041076 s, 25.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0474723 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.029933 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251807 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0456787 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356182 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0507087 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.063423 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118416 s, 88.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00934543 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016555 s, 63.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138158 s, 75.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153915 s, 68.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173782 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0151381 s, 69.3 MB/s , 0.0158154 s, 66.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149593 s, 70.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.017118 s, 61.3 MB/s , 0.0147257 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155955 s, 67.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0161146 s, 65.1 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0185149 s, 56.6 MB/s 1048576 bytes (1.0 MB) copied, 0.016084 s, 65.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173194 s, 60.5 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg331-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.132765 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137664 s, 76.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129703 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176121 s, 59.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174779 s, 60.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.046207 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0496728 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0684785 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0704837 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0655335 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0554967 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0647657 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0731627 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0602382 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0569273 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0694566 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0769294 s, 13.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a612800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff88012a612800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00845842 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128253 s, 81.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.011441 s, 91.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0128897 s, 81.3 MB/s 1048576 bytes (1.0 MB) copied, 0.00864026 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.009635 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100389 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00964988 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0096802 s, 108 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983141 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00991557 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106461 s, 98.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0123426 s, 85.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00986777 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119918 s, 87.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141372 s, 74.2 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg331-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (5s) == sanityn test 77e: check TBF NID nrs policy ============ 05:58:19 (1744538299) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.203.31@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.13173 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00924946 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105508 s, 99.4 MB/s , 0.0139406 s, 75.2 MB/s , 0.0133893 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273428 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0310679 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03201 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0382715 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0425325 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.036203 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0504425 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.16596 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15617 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156681 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.162149 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.173223 s, 6.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012a612800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132196 s, 79.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013543 s, 77.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139576 s, 75.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181035 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141219 s, 74.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172557 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159271 s, 65.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158527 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163244 s, 64.2 MB/s , 0.0169252 s, 62.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175809 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015491 s, 67.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155967 s, 67.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0162363 s, 64.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0162492 s, 64.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176451 s, 59.4 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg331-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.129091 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145574 s, 72.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153171 s, 68.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0415749 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0455396 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0472365 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446256 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390962 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0480396 s, 21.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0599743 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.046874 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0511037 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0568706 s, 18.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0615985 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0513118 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0575033 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0605468 s, 17.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00273523 s, 383 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014241 s, 73.6 MB/s 1048576 bytes (1.0 MB) copied, 0.00278724 s, 376 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185754 s, 56.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158851 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190391 s, 55.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.016737 s, 62.7 MB/s , 0.0105455 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101055 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171603 s, 61.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163903 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168071 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171361 s, 61.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186898 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181762 s, 57.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185342 s, 56.6 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg331-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.117504 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161068 s, 65.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180122 s, 58.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274911 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0391175 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413591 s, 25.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0503242 s, 20.8 MB/s , 0.0484624 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447702 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0484992 s, 21.6 MB/s , 0.0380761 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400416 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0483648 s, 21.7 MB/s , 0.0432161 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0484485 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0500275 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0608571 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115027 s, 91.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124742 s, 84.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113041 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107061 s, 97.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294676 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285491 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280258 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147791 s, 71.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148183 s, 70.8 MB/s 1+0 records in 1+0 records out , 0.0161017 s, 65.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0150409 s, 69.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030735 s, 34.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0164292 s, 63.8 MB/s , 0.0153273 s, 68.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155118 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211409 s, 49.6 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg331-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.136853 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00946609 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133827 s, 78.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018309 s, 57.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149014 s, 70.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279083 s, 37.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216756 s, 48.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206313 s, 50.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193874 s, 54.1 MB/s 1048576 bytes (1.0 MB) copied, 0.024109 s, 43.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0115218 s, 91.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01372 s, 76.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.029222 s, 35.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186364 s, 56.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0429822 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00952241 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025219 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0126469 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144746 s, 72.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119575 s, 87.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144789 s, 72.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0147168 s, 71.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012576 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124498 s, 84.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132139 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014077 s, 74.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01336 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130169 s, 80.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155114 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143926 s, 72.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163751 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165908 s, 63.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152099 s, 68.9 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg331-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (8s) == sanityn test 77f: check TBF JobID nrs policy ========== 05:58:27 (1744538307) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg331-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.164245 s, 102 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00928459 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119116 s, 88.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112534 s, 93.2 MB/s , 0.0168317 s, 62.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01649 s, 63.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.086469 s, 12.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0885596 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0989114 s, 10.6 MB/s 1048576 bytes (1.0 MB) copied, 0.100861 s, 10.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0980641 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.10382 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.102611 s, 10.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.107207 s, 9.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0998533 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.111627 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121455 s, 8.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247297 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139094 s, 75.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0284221 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156118 s, 67.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184606 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173998 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177795 s, 59.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0315681 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199969 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282505 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212323 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215779 s, 48.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0224607 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219321 s, 47.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214528 s, 48.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0252053 s, 41.6 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg331-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg331-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.137562 s, 122 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148525 s, 70.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184247 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202249 s, 51.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0380288 s, 27.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0373565 s, 28.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0578357 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.05857 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0570053 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0561222 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0578223 s, 18.1 MB/s , 0.0567845 s, 18.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0512826 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0620643 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0578615 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0585435 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0536639 s, 19.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out , 0.0151378 s, 69.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0152337 s, 68.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123433 s, 85.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0298289 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178014 s, 58.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335184 s, 31.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0272775 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0325178 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169223 s, 62.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167498 s, 62.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173724 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184501 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182139 s, 57.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180518 s, 58.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205217 s, 51.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208462 s, 50.3 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg331-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg331-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.134726 s, 125 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136342 s, 76.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170957 s, 61.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164861 s, 63.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034024 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0516929 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0564557 s, 18.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0553568 s, 18.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0563818 s, 18.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0603525 s, 17.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0666559 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0626119 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0707195 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0696835 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0749669 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0793225 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0925103 s, 11.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125748 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128169 s, 81.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136289 s, 76.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128257 s, 81.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161422 s, 65.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00174926 s, 599 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148542 s, 70.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011386 s, 92.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163549 s, 64.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124452 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154854 s, 67.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133871 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00552306 s, 190 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148382 s, 70.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015027 s, 69.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175397 s, 59.8 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg331-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg331-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.143646 s, 117 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189596 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170569 s, 61.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212187 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176165 s, 59.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189899 s, 55.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238996 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142925 s, 73.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0422316 s, 24.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266583 s, 39.3 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242018 s, 43.3 MB/s , 0.047786 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0404538 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0348508 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0470417 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0448033 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0506877 s, 20.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0118052 s, 88.8 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0103388 s, 101 MB/s , 0.0100729 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135542 s, 77.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117487 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137125 s, 76.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00631479 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010879 s, 96.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00996291 s, 105 MB/s , 0.0118624 s, 88.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00219656 s, 477 MB/s , 0.00242319 s, 433 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114197 s, 91.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0018074 s, 580 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00278476 s, 377 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00137441 s, 763 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg331-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (9s) == sanityn test 77g: Change TBF type directly ============ 05:58:36 (1744538316) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (4s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 05:58:40 (1744538320) oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg331-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg331-client: oleg331-server: ssh exited with exit code 2 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 oleg331-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Unknown error 524 pdsh@oleg331-client: oleg331-server: ssh exited with exit code 12 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 oleg331-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Unknown error 524 pdsh@oleg331-client: oleg331-server: ssh exited with exit code 12 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 oleg331-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Unknown error 524 pdsh@oleg331-client: oleg331-server: ssh exited with exit code 12 PASS 77h (4s) == sanityn test 77i: Change rank of TBF rule ============= 05:58:44 (1744538324) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (8s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 05:58:52 (1744538332) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.124346 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155899 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135876 s, 77.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0349071 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0454671 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0496869 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0468913 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0385623 s, 27.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0441368 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0532362 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0516059 s, 20.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0506862 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0487086 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.050153 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0486453 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.047324 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.047154 s, 22.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181411 s, 57.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175309 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137605 s, 76.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015962 s, 65.7 MB/s , 0.0168153 s, 62.4 MB/s , 0.0162983 s, 64.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0193149 s, 54.3 MB/s , 0.0174402 s, 60.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019491 s, 53.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017807 s, 58.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197828 s, 53.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0194396 s, 53.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0189074 s, 55.5 MB/s , 0.0188256 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212984 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023872 s, 43.9 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg331-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.99142 s, 21.0 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4466 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (52s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 05:59:44 (1744538384) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg331-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg331-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg331-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg331-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg331-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.23304 s, 13.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229279 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03378 s, 31.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0199759 s, 52.5 MB/s , 0.0211627 s, 49.5 MB/s , 0.0371914 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167211 s, 62.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392354 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0507524 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158532 s, 6.6 MB/s , 0.159902 s, 6.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.171644 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149471 s, 7.0 MB/s , 0.166067 s, 6.3 MB/s 1048576 bytes (1.0 MB) copied, 0.157765 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.192226 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.197813 s, 5.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173507 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163921 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169596 s, 61.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016841 s, 62.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0179419 s, 58.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184056 s, 57.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184215 s, 56.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171172 s, 61.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0198081 s, 52.9 MB/s , 0.0166488 s, 63.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184197 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199378 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198599 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205626 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200479 s, 52.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216052 s, 48.5 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg331-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8506 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9295 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg331-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg331-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg331-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg331-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg331-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.466602 s, 36.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275952 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235989 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274101 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0296269 s, 35.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02669 s, 39.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256791 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202556 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149309 s, 70.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.11361 s, 9.2 MB/s , 0.109131 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.104935 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.104834 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106916 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.135186 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14034 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.114249 s, 9.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0141542 s, 74.1 MB/s 1048576 bytes (1.0 MB) copied, 0.00617361 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161463 s, 64.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176741 s, 59.3 MB/s 1048576 bytes (1.0 MB) copied, 0.018387 s, 57.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021834 s, 48.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161774 s, 64.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0226351 s, 46.3 MB/s , 0.0200354 s, 52.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239142 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168882 s, 62.1 MB/s , 0.00975389 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174379 s, 60.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209916 s, 50.0 MB/s , 0.0156878 s, 66.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198226 s, 52.9 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg331-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.6785 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.927 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (132s) == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 06:01:56 (1744538516) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg331-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg331-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg331-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg331-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg331-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.23745 s, 13.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0349248 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0385868 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289514 s, 36.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.046581 s, 22.5 MB/s , 0.0415651 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0719745 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0608359 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0485565 s, 21.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.151957 s, 6.9 MB/s , 0.152085 s, 6.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.143747 s, 7.3 MB/s , 0.142935 s, 7.3 MB/s , 0.160712 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.173811 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.170172 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.174766 s, 6.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019824 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160424 s, 65.4 MB/s , 0.020894 s, 50.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206416 s, 50.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208797 s, 50.2 MB/s , 0.0113112 s, 92.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0219603 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208317 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218084 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242815 s, 43.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251118 s, 41.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0233748 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226793 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237697 s, 44.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238397 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281207 s, 37.3 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg331-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.7149 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9221 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg331-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg331-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg331-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg331-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg331-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.23426 s, 13.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243593 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102415 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0302096 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174763 s, 60.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161699 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0299312 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0301145 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153164 s, 68.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.150505 s, 7.0 MB/s , 0.152779 s, 6.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.141721 s, 7.4 MB/s , 0.149608 s, 7.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.165478 s, 6.3 MB/s , 0.170057 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.184578 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188196 s, 5.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175959 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0202818 s, 51.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0188778 s, 55.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177855 s, 59.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208136 s, 50.4 MB/s , 0.0210126 s, 49.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211253 s, 49.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0243594 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205054 s, 51.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215342 s, 48.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0267741 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212512 s, 49.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214162 s, 49.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020892 s, 50.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250714 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025502 s, 41.1 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg331-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.7466 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9269 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (132s) == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 06:04:08 (1744538648) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg331-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.291487 s, 57.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190087 s, 55.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0223227 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182646 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174839 s, 60.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0474234 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0359502 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0524902 s, 20.0 MB/s , 0.0460474 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0585022 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0431561 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0586284 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0511545 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0502308 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0498919 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0444567 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0541646 s, 19.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140815 s, 74.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out , 0.0104482 s, 100 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0152222 s, 68.9 MB/s , 0.0141415 s, 74.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.016768 s, 62.5 MB/s , 0.0186859 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131152 s, 80.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157597 s, 66.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00261818 s, 400 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0133182 s, 78.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0147887 s, 70.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208639 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180578 s, 58.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184395 s, 56.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0169727 s, 61.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174115 s, 60.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg331-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.9399 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74227 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.31@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.31@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.16928 s, 99.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140598 s, 74.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194841 s, 53.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019369 s, 54.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229485 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388535 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0410964 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0340782 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0437755 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0382379 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0339625 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400394 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.044044 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.160093 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152326 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.202083 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.209213 s, 5.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.016451 s, 63.7 MB/s , 0.0158964 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143389 s, 73.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193599 s, 54.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167934 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172339 s, 60.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0198989 s, 52.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154208 s, 68.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167184 s, 62.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198127 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180015 s, 58.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185426 s, 56.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179738 s, 58.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191412 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190234 s, 55.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214408 s, 48.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg331-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95453 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73891 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.31@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg331-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.158603 s, 106 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00724847 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00894618 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118725 s, 88.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249672 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123488 s, 84.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0264624 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202314 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0305404 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286444 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203338 s, 51.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289078 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236592 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136034 s, 77.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281512 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0339744 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.035933 s, 29.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101215 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121899 s, 86.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010385 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984528 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112413 s, 93.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110297 s, 95.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0110051 s, 95.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114106 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111567 s, 94.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119853 s, 87.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123597 s, 84.8 MB/s , 0.0120747 s, 86.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133648 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125368 s, 83.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129777 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153298 s, 68.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg331-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.98085 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.89227 s, 21.4 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg331-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.151166 s, 111 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126513 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188186 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0363105 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248881 s, 42.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343565 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0467381 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0437693 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0434138 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0514734 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411078 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.036849 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0430253 s, 24.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0916901 s, 11.4 MB/s , 0.0831 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0898982 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0843953 s, 12.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159664 s, 65.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016192 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0106233 s, 98.7 MB/s , 0.0126836 s, 82.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00952338 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0106392 s, 98.6 MB/s , 0.00842913 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116126 s, 90.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211759 s, 49.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0142835 s, 73.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0169004 s, 62.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142515 s, 73.6 MB/s , 0.0222096 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0230852 s, 45.4 MB/s , 0.0208684 s, 50.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232151 s, 45.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg331-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.80527 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74125 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95282 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74722 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg331-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.157025 s, 107 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118011 s, 88.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265424 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233397 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0330715 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254827 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198028 s, 53.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019218 s, 54.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0314814 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0754262 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0858072 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0988075 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.102743 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.101362 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0967989 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.109802 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.113458 s, 9.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0130109 s, 80.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0146864 s, 71.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129873 s, 80.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118399 s, 88.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132694 s, 79.0 MB/s , 0.0198895 s, 52.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161441 s, 65.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166623 s, 62.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00456249 s, 230 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161481 s, 64.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167467 s, 62.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168447 s, 62.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184278 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195329 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185621 s, 56.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207156 s, 50.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg331-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.7014 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4459 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg331-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.170414 s, 98.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0164176 s, 63.9 MB/s , 0.0187659 s, 55.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0340828 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353155 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0518854 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.044968 s, 23.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0519924 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0496 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0448623 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0565954 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.048984 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.04615 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0489047 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0421544 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.056496 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0598878 s, 17.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111757 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00980407 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019083 s, 54.9 MB/s , 0.0152534 s, 68.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153545 s, 68.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0172014 s, 61.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0171372 s, 61.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120774 s, 86.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0149725 s, 70.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188058 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014079 s, 74.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135031 s, 77.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150347 s, 69.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151771 s, 69.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181284 s, 57.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210874 s, 49.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg331-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.01226 s, 20.9 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74633 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg331-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.141451 s, 119 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014163 s, 74.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0156748 s, 66.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360422 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.041654 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0410322 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0424931 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400288 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0403229 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0311374 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395714 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0303866 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0350252 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0871646 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0932309 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.101099 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0936796 s, 11.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0135094 s, 77.6 MB/s , 0.0136474 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138888 s, 75.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180779 s, 58.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158193 s, 66.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181176 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00256938 s, 408 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159618 s, 65.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177137 s, 59.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179975 s, 58.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173383 s, 60.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164331 s, 63.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179524 s, 58.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184818 s, 56.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181548 s, 57.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205516 s, 51.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg331-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.80877 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74388 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.80393 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74176 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (347s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 06:09:56 (1744538996) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (4s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 06:09:59 (1744538999) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg331-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.67771 s, 285 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg331-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0207 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (51s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 06:10:50 (1744539050) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.132894 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114108 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105757 s, 99.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037851 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0459904 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.045276 s, 23.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0463796 s, 22.6 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0511127 s, 20.5 MB/s , 0.0510493 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447103 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.052075 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0430998 s, 24.3 MB/s , 0.0425064 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0424753 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0520413 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0621818 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0482954 s, 21.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a612800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88012a612800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117337 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105468 s, 99.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0131253 s, 79.9 MB/s , 0.0152619 s, 68.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118251 s, 88.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121828 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125807 s, 83.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146376 s, 71.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139419 s, 75.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156737 s, 66.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159081 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01603 s, 65.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150527 s, 69.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154307 s, 68.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156811 s, 66.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172625 s, 60.7 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg331-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96771 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97723 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.258506 s, 64.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126432 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124284 s, 84.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186827 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0414229 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0528016 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0504302 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0546133 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0535956 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464267 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0456495 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.049694 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0422738 s, 24.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.250321 s, 4.2 MB/s , 0.251982 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.291484 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.289337 s, 3.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016042 s, 65.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0188156 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168784 s, 62.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129481 s, 81.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0151291 s, 69.3 MB/s , 0.0153181 s, 68.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0136865 s, 76.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183686 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015645 s, 67.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155381 s, 67.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017224 s, 60.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174319 s, 60.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178285 s, 58.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180445 s, 58.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197625 s, 53.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175003 s, 59.9 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg331-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg331-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97568 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97611 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (64s) == sanityn test 77o: Changing rank should not panic ====== 06:11:55 (1744539115) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (5s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 06:11:59 (1744539119) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (36s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 06:12:35 (1744539155) mds.MDS.mdt.nrs_policies=tbf oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg331-client: oleg331-server: ssh exited with exit code 36 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 oleg331-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg331-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (13s) == sanityn test 77r: Change type of tbf policy at run time ========================================================== 06:12:48 (1744539168) mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 21264 kill md_thread xargs PASS 77r (37s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 06:13:25 (1744539205) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_policies=fifo PASS 78 (3s) == sanityn test 79: xattr: intent error ================== 06:13:28 (1744539208) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272506730 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-04-13 06:13:29.000000000 -0400 Modify: 2025-04-13 06:13:29.000000000 -0400 Change: 2025-04-13 06:13:29.000000000 -0400 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (13s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 06:13:41 (1744539221) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 06:13:42 (1744539222) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 06:13:44 (1744539224) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 06:13:45 (1744539225) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 06:13:46 (1744539226) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 06:13:47 (1744539227) - open/close 2395 (time 1744539239.26 total 10.00 last 239.43) total: 2500 open/close in 10.47 seconds: 238.81 ops/second snapshot_time 1744539259.642435815 secs.nsecs start_time 1744539227.977529437 secs.nsecs elapsed_time 31.664906378 secs.nsecs open 2505 samples [usecs] 89 4570 1070675 497261247 close 2504 samples [usecs] 64 1225 354423 55672735 mknod 2501 samples [usecs] 172 4335 943510 392973884 unlink 2500 samples [usecs] 262 17624 1747350 4185466892 mkdir 2 samples [usecs] 746 1041 1787 1640197 rmdir 2 samples [usecs] 3235 5347 8582 39055634 rename 2500 samples [usecs] 214 6515 3403033 7255995419 getattr 3761 samples [usecs] 5 237 74599 1921915 parallel_rename_file 2500 samples [usecs] 214 6515 3403033 7255995419 crossdir_rename 2500 samples [usecs] 214 6515 3403033 7255995419 PASS 81d (34s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 06:14:22 (1744539262) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7032 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7032 PASS 82 (3s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 06:14:24 (1744539264) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 06:14:25 (1744539265) fail_loc=0x8000060b PASS 84 (8s) == sanityn test 85: Lustre API root cache race =========== 06:14:33 (1744539273) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1744539273 Finishing test test1 at 1744539274 PASS 85 (3s) == sanityn test 90: open/create and unlink striped directory ========================================================== 06:14:36 (1744539276) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 06:14:37 (1744539277) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (2s) == sanityn test 92: create remote directory under orphan directory ========================================================== 06:14:39 (1744539279) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 06:14:40 (1744539280) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6082 0x17c2 0x240000400 1 6067 0x17b3 0x280000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6083 0x17c3 0x240000400 1 6068 0x17b4 0x280000400 rc1=0 and rc2=0 PASS 93 (9s) == sanityn test 94: signal vs CP callback race =========== 06:14:49 (1744539289) osc.lustre-OST0000-osc-ffff88012a612800.idle_timeout=0 osc.lustre-OST0000-osc-ffff88012e275800.idle_timeout=0 osc.lustre-OST0001-osc-ffff88012a612800.idle_timeout=0 osc.lustre-OST0001-osc-ffff88012e275800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0516488 s, 159 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 3.00587 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5764: 10232 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0217588 s, 188 kB/s osc.lustre-OST0000-osc-ffff88012a612800.idle_timeout=debug osc.lustre-OST0000-osc-ffff88012e275800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012a612800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012e275800.idle_timeout=debug PASS 94 (10s) == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 06:15:00 (1744539300) llite.lustre-ffff88012a612800.fast_read=0 llite.lustre-ffff88012e275800.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0220424 s, 95.1 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff88012a612800.fast_read=1 llite.lustre-ffff88012e275800.fast_read=1 PASS 95a (19s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 06:15:19 (1744539319) llite.lustre-ffff88012a612800.fast_read=0 llite.lustre-ffff88012e275800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00293649 s, 4.2 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff88012a612800.fast_read=1 llite.lustre-ffff88012e275800.fast_read=1 PASS 95b (8s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 06:15:27 (1744539327) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (2s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 06:15:29 (1744539329) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00298922 s, 1.4 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 06:15:31 (1744539331) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0329723 s, 63.6 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 06:15:33 (1744539333) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0329091 s, 63.7 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (3s) == sanityn test 100e: DoM: read on open and file size ==== 06:15:36 (1744539336) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00410235 s, 2.9 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 06:15:38 (1744539338) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000745361 s, 5.5 MB/s PASS 101a (3s) == sanityn test 101b: Discard DoM data on rename ========= 06:15:41 (1744539341) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00110602 s, 3.7 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 06:15:43 (1744539343) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000838537 s, 4.9 MB/s PASS 101c (4s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 06:15:47 (1744539347) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1744539348 lockahead_test: lockahead_test.c:1191: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1744539351 Finishing test test23 at 1744539351 PASS 103 (8s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 06:15:55 (1744539355) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 06:15:56 (1744539356) fail_loc=0x416 PASS 105 (23s) == sanityn test 106a: Verify the btime via statx() ======= 06:16:19 (1744539379) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 06:16:20 (1744539380) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) == sanityn test 106c: Verify statx attributes mask ======= 06:16:21 (1744539381) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) == sanityn test 107a: Basic grouplock conflict =========== 06:16:22 (1744539382) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0880894 s, 119 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 6073 0x17b9 0x280000400 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 06:16:27 (1744539387) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0692463 s, 151 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6091 0x17cb 0x240000400 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7032 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (9s) == sanityn test 108a: lseek: parallel updates ============ 06:16:35 (1744539395) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00264648 s, 3.8 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.23343 s, 2.0 MB/s PASS 108a (6s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 06:16:42 (1744539402) 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions oleg331-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 2 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions oleg331-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 4 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions oleg331-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 10 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_refcnt: could not read integer from '/sys/module/pcc_cpufreq/refcnt': 'No such device' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions oleg331-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 15 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions oleg331-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 17 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions oleg331-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 24 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions oleg331-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 25 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions oleg331-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 26 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions oleg331-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 32 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions oleg331-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 37 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions oleg331-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 38 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions oleg331-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 40 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions oleg331-server: libkmod: kmod_module_get_refcnt: could not read integer from '/sys/module/pcc_cpufreq/refcnt': 'No such device' Iteration 49 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 192.168.203.131@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.203.131@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg331-server' oleg331-server: oleg331-server.virtnet: executing load_modules_local oleg331-server: Loading modules from /home/green/git/lustre-release/lustre oleg331-server: detected 4 online CPUs by sysfs oleg331-server: MODOPTS_LIBCFS= oleg331-server: Force libcfs to create 2 CPU partitions oleg331-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre Starting client: oleg331-client.virtnet: -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre2 PASS 109 (465s) == sanityn test 110: do not grant another lock on resend ========================================================== 06:24:27 (1744539867) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-04-13 06:24:28.000000000 -0400 Modify: 2025-04-13 06:24:28.000000000 -0400 Change: 2025-04-13 06:24:28.000000000 -0400 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg331-client.virtnet oleg331-client.virtnet PASS 110 (66s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 06:25:33 (1744539933) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 06:25:34 (1744539934) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 06:25:35 (1744539935) lustre-OST0000-osc-ffff8801372fe000 active. lustre-OST0001-osc-ffff8801372fe000 active. lustre-OST0000-osc-ffff8801372fe000 active. lustre-OST0001-osc-ffff8801372fe000 active. MGC192.168.203.131@tcp active. PASS 113 (2s) == sanityn test 114: implicit default LMV inherit ======== 06:25:37 (1744539937) SKIP: sanityn test_114 We need at least 2 MDTs for this test SKIP 114 (1s) == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 06:25:38 (1744539938) SKIP: sanityn test_115 ldiskfs only test SKIP 115 (2s) == sanityn test 116: DNE: Set default LMV layout from a remote client ========================================================== 06:25:40 (1744539940) SKIP: sanityn test_116 needs >= 2 MDTs SKIP 116 (1s) == sanityn test 200: service remains healthy while able to process request ========================================================== 06:25:41 (1744539941) at_max=10 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=21 fail_loc=0x1e1 fail_val=20 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.0336 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1241 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1342 s, 0.2 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied1+0 records in 1+0 records out , 20.1304 s, 0.2 kB/s 4096 bytes (4.1 kB) copied, 20.1396 s, 0.2 kB/s 1+0 records in 1+0 records out , 20.1247 s, 0.2 kB/s 4096 bytes (4.1 kB) copied, 20.1369 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out , 20.1277 s, 0.2 kB/s 4096 bytes (4.1 kB) copied, 20.1343 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.0947 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.1145 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 40.1523 s, 0.1 kB/s , 40.0917 s, 0.1 kB/s , 40.095 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.1923 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.2228 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out , 40.2138 s, 0.1 kB/s 4096 bytes (4.1 kB) copied, 40.2202 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.22 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.2207 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied1+0 records in 1+0 records out , 60.2565 s, 0.1 kB/s , 60.2691 s, 0.1 kB/s 4096 bytes (4.1 kB) copied, 60.1891 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.2371 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.2763 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.3555 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.3217 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.201 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.2201 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3279 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 80.3246 s, 0.1 kB/s , 80.4006 s, 0.1 kB/s , 80.3215 s, 0.1 kB/s , 80.3276 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3244 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3399 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.346 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.372 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.323 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.33 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.382 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.437 s, 0.0 kB/s at_max=600 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=116 oleg331-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88012fe97000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff88012fe97000.ost_server_uuid in IDLE state after 0 sec PASS 200 (107s) cleanup: ====================================================== == sanityn test complete, duration 4803 sec ============== 06:27:29 (1744540049) === sanityn: start cleanup 06:27:29 (1744540049) === Stopping clients: oleg331-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg331-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 06:28:47 (1744540127) ===