-----============= acceptance-small: sanityn ============----- Sun Aug 4 14:09:42 EDT 2024 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 excepting tests: 27 28 102 skipping tests SLOW=no: 33a === sanityn: start setup 14:09:47 (1722794987) === Starting client oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 Started clients oleg310-client.virtnet: 192.168.203.110@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project) oleg310-client.virtnet: executing check_config_client /mnt/lustre oleg310-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg310-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800b5870000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b5904800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b5870000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b5904800.idle_timeout=debug disable quota as required oleg310-server: oleg310-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 14:09:53 (1722794993) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111828 s, 93.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6938] == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 14:09:53 (1722794993) PASS 0 (2s) == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 14:09:55 (1722794995) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (1s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 14:09:56 (1722794996) -rw-r--r-- 1 root root 0 Aug 4 14:09 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (2s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 14:09:58 (1722794998) -rw-r--r-- 1 root root 0 Aug 4 14:09 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (1s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 14:10:00 (1722795000) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (2s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 14:10:01 (1722795001) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (1s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 14:10:03 (1722795003) total 1 -rwxrwxrwx 1 root root 0 Aug 4 14:09 f2a -rwxrwxrwx 1 root root 0 Aug 4 14:09 f2b total 1 -rwxrwxrwx 1 root root 0 Aug 4 14:09 f2a -rwxrwxrwx 1 root root 0 Aug 4 14:09 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00246508 s, 208 kB/s PASS 2e (2s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 14:10:04 (1722795004) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 14:10:05 (1722795005) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0841269 s, 24.9 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (1s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 14:10:06 (1722795006) PASS 3 (1s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 14:10:07 (1722795007) PASS 4 (3s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 14:10:10 (1722795010) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (1s) == sanityn test 6: remove of open file on other node ============================================================================ 14:10:11 (1722795011) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (2s) == sanityn test 7: remove of open directory on other node ======================================================================= 14:10:13 (1722795013) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (1s) == sanityn test 8: remove of open special file on other node ==================================================================== 14:10:14 (1722795014) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (2s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 14:10:16 (1722795016) PASS 9a (1s) == sanityn test 9b: append to striped sparse file ======== 14:10:17 (1722795017) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118466 s, 88.5 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0122792 s, 0.2 kB/s Data read (expecting 'foo'): foo PASS 9b (2s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 14:10:19 (1722795019) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00148989 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00334684 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00235886 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00242591 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0021171 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00274417 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00285637 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00382826 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00267869 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00389747 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00249884 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00243145 s, 0.4 kB/s PASS 10a (2s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 14:10:21 (1722795021) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00242562 s, 1.3 MB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00814267 s, 503 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000236389 s, 13.0 MB/s PASS 10b (1s) == sanityn test 11: execution of file opened for write should return error ============================================================== 14:10:22 (1722795022) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 301: /mnt/lustre2/d11/f: Text file busy PASS 11 (2s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 14:10:24 (1722795024) warning: '-runtime' deprecated, use '-t runtime' instead - link 2461 (time 1722795034.59 total 10.00 last 246.05) - link 4637 (time 1722795044.59 total 20.01 last 217.53) - link 6787 (time 1722795054.59 total 30.01 last 214.93) - link 9132 (time 1722795064.59 total 40.01 last 234.50) - link 10000 (time 1722795067.98 total 43.39 last 256.67) - link 13047 (time 1722795077.98 total 53.39 last 304.64) - link 15587 (time 1722795087.98 total 63.40 last 253.93) - link 18248 (time 1722795097.98 total 73.40 last 266.03) - link 20000 (time 1722795104.79 total 80.20 last 257.45) - link 22592 (time 1722795114.79 total 90.21 last 259.14) total: 25112 link in 100.00 seconds: 251.11 ops/second using seed 1009279365 running for 100 seconds - stat 10000 (time 1722795030 ; total 5 ; last 5) - stat 20000 (time 1722795036 ; total 11 ; last 6) - stat 30000 (time 1722795042 ; total 17 ; last 6) - stat 40000 (time 1722795047 ; total 22 ; last 5) - stat 50000 (time 1722795053 ; total 28 ; last 6) - stat 60000 (time 1722795058 ; total 33 ; last 5) - stat 70000 (time 1722795064 ; total 39 ; last 6) - stat 80000 (time 1722795069 ; total 44 ; last 5) - stat 90000 (time 1722795074 ; total 49 ; last 5) - stat 99086 (time 1722795085 ; total 60 ; last 11) - stat 100000 (time 1722795086 ; total 61 ; last 1) - stat 108370 (time 1722795097 ; total 72 ; last 11) - stat 110000 (time 1722795099 ; total 74 ; last 2) - stat 117881 (time 1722795110 ; total 85 ; last 11) - stat 120000 (time 1722795112 ; total 87 ; last 2) - stat 127612 (time 1722795123 ; total 98 ; last 11) total: 129624 stats in 100 seconds: 1296.239990 stats/second - unlinked 0 (time 1722795075 ; total 0 ; last 0) - unlinked 10000 (time 1722795131 ; total 56 ; last 56) - unlinked 20000 (time 1722795159 ; total 84 ; last 28) unlink(/mnt/lustre2/lockdir/lockfile25112) error: No such file or directory total: 25111 unlinks in 96 seconds: 261.572906 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (21235) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (21238) - No such process PASS 12 (148s) == sanityn test 13: test directory page revocation ======= 14:12:52 (1722795172) f13 f13 PASS 13 (1s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 14:12:53 (1722795173) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 336: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4759: 22621 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 14:12:55 (1722795175) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4759: 23251 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (1s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 14:12:56 (1722795176) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 351: 23879 Terminated $DIR1/$tdir/sleep 60 PASS 14b (2s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 14:12:58 (1722795178) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 363: 24509 Terminated $DIR1/$tdir/sleep 60 PASS 14c (1s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 14:12:59 (1722795179) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 374: 25140 Terminated $DIR1/$tdir/sleep 60 PASS 14d (2s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 14:13:01 (1722795181) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_CODE=117440512 CLIENT_OS_VERSION_ID=7 STRIPECOUNT=2 ORIGFREE=7523328 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7523328kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (18s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 14:13:19 (1722795199) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.107374 s, 78.1 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.104518 s, 80.3 MB/s Chance of close/open is 1 in 50 Seed set to 5199 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000003[1] 1722795200.169475 write 0x478271 thru 0x47de0c (0x5b9c bytes) 000004[0] 1722795200.175487 mapwrite 0x69c451 thru 0x6abdcb (0xf97b bytes) truncating to largest ever: 0x2ec628 000008[1] 1722795200.225096 trunc from 0x6abdcc to 0x2ec627 (0x3bf7a4 bytes) 000009[1] 1722795200.232592 trunc from 0x2ec628 to 0x0cdff1 (0x21e636 bytes) 000012[1] 1722795200.239823 read 0x0034c7 thru 0x0102b5 (0xcdef bytes) 000014[1] 1722795200.247790 write 0x9e7b1b thru 0x9ebbf1 (0x40d7 bytes) 000015[1] 1722795200.249724 mapread 0x6a7e7b thru 0x6abe48 (0x3fce bytes) 000017[1] 1722795200.256608 read 0x786306 thru 0x787324 (0x101f bytes) 000026[1] 1722795200.307376 mapwrite 0x975b29 thru 0x97c889 (0x6d61 bytes) 000027[1] 1722795200.322982 read 0x5eb6e3 thru 0x5ecaac (0x13ca bytes) 000029[0] 1722795200.326450 write 0x4e1f9a thru 0x4e67bd (0x4824 bytes) 000031[1] 1722795200.338437 write 0x958d24 thru 0x9625ac (0x9889 bytes) 000032[0] 1722795200.343663 trunc from 0x9ebbf2 to 0x05e050 (0x98dba1 bytes) 000037[0] 1722795200.376857 read 0x053b43 thru 0x05e050 (0xa50e bytes) 000038[1] 1722795200.384113 mapwrite 0x2fc2e4 thru 0x305eeb (0x9c08 bytes) truncating to largest ever: 0x338de7 000042[0] 1722795200.441009 trunc from 0x305eec to 0x338de6 (0x32efb bytes) 000043[0] 1722795200.455244 mapread 0x1eaa96 thru 0x1f0307 (0x5872 bytes) 000045[1] 1722795200.468492 write 0x65eb8a thru 0x66ce0b (0xe282 bytes) 000048[0] 1722795200.483208 write 0x864f33 thru 0x86cd31 (0x7dff bytes) truncating to largest ever: 0x7c77e7 000050[0] 1722795200.494387 trunc from 0x86cd32 to 0x7c77e6 (0xa554b bytes) 000053[1] 1722795200.511704 mapwrite 0x78cdcd thru 0x78fde5 (0x3019 bytes) 000054[0] 1722795200.536553 mapread 0x24e299 thru 0x2584d7 (0xa23f bytes) 000055[0] 1722795200.546868 mapread 0x5d91db thru 0x5e0ff5 (0x7e1b bytes) 000057[1] 1722795200.567123 mapwrite 0x022223 thru 0x02cd87 (0xab65 bytes) truncating to largest ever: 0x8cd763 000058[1] 1722795200.618421 trunc from 0x7c77e7 to 0x8cd762 (0x105f7c bytes) 000060[0] 1722795200.626946 trunc from 0x8cd763 to 0x563f88 (0x3697da bytes) 000061[1] 1722795200.634416 write 0x99afd4 thru 0x99f9ac (0x49d9 bytes) 000064[1] 1722795200.649378 mapread 0x61ee43 thru 0x629fce (0xb18c bytes) 000067[1] 1722795200.668745 write 0x61fb38 thru 0x628b2c (0x8ff5 bytes) 000069[0] 1722795200.673718 read 0x5e5bb8 thru 0x5e8f48 (0x3391 bytes) 000070[0] 1722795200.679417 mapread 0x87369f thru 0x879e48 (0x67aa bytes) 000071[0] 1722795200.690050 write 0x5bc3b2 thru 0x5bd8a1 (0x14f0 bytes) 000075[1] 1722795200.697477 mapwrite 0x485d44 thru 0x48e62a (0x88e7 bytes) 000077[0] 1722795200.729691 read 0x4711d5 thru 0x475adc (0x4908 bytes) 000079[0] 1722795200.757459 mapread 0x8644a8 thru 0x873744 (0xf29d bytes) 000081[1] 1722795200.769008 trunc from 0x99f9ad to 0x47ea13 (0x520f99 bytes) 000082[1] 1722795200.786366 mapread 0x25605b thru 0x261344 (0xb2ea bytes) 000083[0] 1722795200.803927 write 0x9eb169 thru 0x9ef087 (0x3f1f bytes) 000088[1] 1722795200.830533 mapread 0x590e80 thru 0x5942c5 (0x3446 bytes) 000091[1] 1722795200.856288 mapwrite 0x2c6c82 thru 0x2d187f (0xabfe bytes) 000093[1] 1722795200.888395 read 0x80789c thru 0x81521f (0xd984 bytes) 000095[0] 1722795200.929930 read 0x4aac3b thru 0x4b9c4c (0xf012 bytes) 000097[0] 1722795200.949034 write 0x155273 thru 0x15f9f8 (0xa786 bytes) 000098[1] 1722795200.982931 write 0x6ae8c3 thru 0x6b4e4c (0x658a bytes) 000100[1] 1722795200.991245 mapread 0x0156bf thru 0x01c9c4 (0x7306 bytes) 000102[0] 1722795201.013907 write 0x260312 thru 0x264563 (0x4252 bytes) 000104[1] 1722795201.023144 read 0x7cd860 thru 0x7d656d (0x8d0e bytes) 000108[1] 1722795201.043074 mapread 0x91e770 thru 0x9257dc (0x706d bytes) 000109[1] 1722795201.082595 read 0x149a61 thru 0x15623c (0xc7dc bytes) 000110[1] 1722795201.085791 mapread 0x117cfc thru 0x11ff68 (0x826d bytes) 000111[0] 1722795201.088700 read 0x100a79 thru 0x10921e (0x87a6 bytes) 000112[1] 1722795201.113931 write 0x1221da thru 0x1246ac (0x24d3 bytes) 000116[1] 1722795201.146924 read 0x8771e8 thru 0x8856e9 (0xe502 bytes) 000119[0] 1722795201.166348 mapwrite 0x7b561a thru 0x7b7b16 (0x24fd bytes) 000122[0] 1722795201.229667 mapread 0x00a658 thru 0x018cd7 (0xe680 bytes) 000123[1] 1722795201.251652 trunc from 0x9ef088 to 0x142a60 (0x8ac627 bytes) 000124[1] 1722795201.261939 write 0x07ef5b thru 0x08ced9 (0xdf7f bytes) 000125[1] 1722795201.264495 trunc from 0x142a61 to 0x56c54f (0x429aef bytes) 000126[0] 1722795201.268504 write 0x647c82 thru 0x648a3a (0xdb9 bytes) 000127[0] 1722795201.274666 write 0x97f7a7 thru 0x98a8f1 (0xb14b bytes) 000130[0] 1722795201.285431 read 0x80d7b0 thru 0x81730e (0x9b5f bytes) 000133[1] 1722795201.326804 mapread 0x09454e thru 0x097213 (0x2cc6 bytes) 000135[1] 1722795201.343048 read 0x088134 thru 0x090c27 (0x8af4 bytes) 000136[1] 1722795201.366285 mapwrite 0x5e17a6 thru 0x5e42a7 (0x2b02 bytes) 000137[0] 1722795201.401598 trunc from 0x98a8f2 to 0x5ca18f (0x3c0762 bytes) 000138[0] 1722795201.416600 read 0x4329b6 thru 0x43b4f4 (0x8b3f bytes) 000139[0] 1722795201.438340 mapwrite 0x9f18c1 thru 0x9fd7d5 (0xbf15 bytes) 000140[1] 1722795201.476168 mapread 0x52d223 thru 0x52dc3c (0xa1a bytes) 000143[1] 1722795201.503632 trunc from 0x9fd7d6 to 0x6decf7 (0x31eade bytes) 000144[1] 1722795201.517625 read 0x3346d0 thru 0x33e6a5 (0x9fd6 bytes) 000147[0] 1722795201.558062 mapwrite 0x172a3e thru 0x17fdc7 (0xd38a bytes) 000148[0] 1722795201.593535 write 0x6221d9 thru 0x62cf63 (0xad8b bytes) 000150[1] 1722795201.602203 mapwrite 0x5f05e3 thru 0x5f77a0 (0x71be bytes) 000151[0] 1722795201.631657 trunc from 0x6decf8 to 0x46aeb1 (0x273e46 bytes) 000152[1] 1722795201.637236 trunc from 0x46aeb2 to 0x011720 (0x459791 bytes) 000154[1] 1722795201.649546 mapread 0x00bd77 thru 0x00ed1f (0x2fa9 bytes) 000157[1] 1722795201.662577 mapwrite 0x607fa8 thru 0x60f622 (0x767b bytes) 000161[0] 1722795201.699293 read 0x48736e thru 0x490be1 (0x9874 bytes) 000162[1] 1722795201.710168 trunc from 0x60f623 to 0x7c7577 (0x1b7f55 bytes) 000166[0] 1722795201.723420 mapread 0x146036 thru 0x14f415 (0x93e0 bytes) 000169[1] 1722795201.754430 mapread 0x6d6246 thru 0x6de018 (0x7dd3 bytes) 000173[1] 1722795201.777154 mapread 0x100765 thru 0x104a58 (0x42f4 bytes) 000177[0] 1722795201.790636 write 0x916d38 thru 0x91e058 (0x7321 bytes) 000179[1] 1722795201.803862 write 0x0876ed thru 0x088834 (0x1148 bytes) 000180[0] 1722795201.812792 write 0x7ea11a thru 0x7eaa1e (0x905 bytes) 000182[1] 1722795201.820962 trunc from 0x91e059 to 0x6fbf1d (0x22213b bytes) 000183[1] 1722795201.830834 mapwrite 0x72dee5 thru 0x72f029 (0x1145 bytes) 000186[1] 1722795201.849168 mapread 0x6115df thru 0x61433b (0x2d5d bytes) 000187[1] 1722795201.853379 mapwrite 0x1c627e thru 0x1d14c2 (0xb245 bytes) 000190[1] 1722795201.883021 mapread 0x263446 thru 0x26417c (0xd37 bytes) 000192[0] 1722795201.885575 write 0x5c611e thru 0x5d5e0f (0xfcf2 bytes) 000194[0] 1722795201.910983 read 0x616d32 thru 0x625138 (0xe407 bytes) 000195[1] 1722795201.913735 mapread 0x18efad thru 0x198b72 (0x9bc6 bytes) 000198[1] 1722795201.934714 trunc from 0x72f02a to 0x21b16f (0x513eba bytes) 000200[0] 1722795201.949927 mapwrite 0x668715 thru 0x66f324 (0x6c10 bytes) 000203[1] 1722795201.999069 write 0x3c12e8 thru 0x3ca7cb (0x94e4 bytes) 000204[0] 1722795202.008994 write 0x105379 thru 0x1083c3 (0x304b bytes) 000205[0] 1722795202.017554 read 0x63b247 thru 0x642f90 (0x7d4a bytes) 000206[1] 1722795202.020497 trunc from 0x66f325 to 0x0502f9 (0x61f02b bytes) 000209[1] 1722795202.036515 write 0x49e2b2 thru 0x4a9f19 (0xbc68 bytes) 000211[1] 1722795202.041028 read 0x07d78f thru 0x088d4c (0xb5be bytes) 000212[0] 1722795202.058645 trunc from 0x4a9f1a to 0x491794 (0x18785 bytes) 000213[1] 1722795202.068834 write 0x10c508 thru 0x10e88e (0x2387 bytes) 000218[0] 1722795202.108494 write 0x52eae4 thru 0x539fd0 (0xb4ed bytes) 000228[0] 1722795202.137775 read 0x1b4878 thru 0x1b59c5 (0x114e bytes) 000229[1] 1722795202.171143 mapread 0x3e5147 thru 0x3e6936 (0x17f0 bytes) 000233[0] 1722795202.182023 read 0x003d88 thru 0x013747 (0xf9c0 bytes) 000239[1] 1722795202.190628 mapwrite 0x87fb78 thru 0x881dd4 (0x225d bytes) 000243[1] 1722795202.248091 mapread 0x11041c thru 0x118cfc (0x88e1 bytes) 000245[0] 1722795202.259718 mapwrite 0x302ea0 thru 0x30a74c (0x78ad bytes) 000247[0] 1722795202.306970 mapwrite 0x2e2c91 thru 0x2e86a6 (0x5a16 bytes) truncating to largest ever: 0x9e2fb8 000248[0] 1722795202.326749 trunc from 0x881dd5 to 0x9e2fb7 (0x1611e3 bytes) 000252[0] 1722795202.345504 read 0x120307 thru 0x122d80 (0x2a7a bytes) 000253[0] 1722795202.345935 trunc from 0x9e2fb8 to 0x505985 (0x4dd632 bytes) 000254[0] 1722795202.353794 mapwrite 0x392152 thru 0x39c19c (0xa04b bytes) 000256[1] 1722795202.372102 write 0x67e9e1 thru 0x68110b (0x272b bytes) 000257[0] 1722795202.379210 mapwrite 0x3540ad thru 0x363e2d (0xfd81 bytes) 000258[0] 1722795202.400769 trunc from 0x68110c to 0x7d05e1 (0x14f4d6 bytes) 000259[0] 1722795202.408925 mapread 0x1a0c6c thru 0x1ad920 (0xccb5 bytes) 000262[0] 1722795202.413309 mapwrite 0x98d96d thru 0x99c673 (0xed07 bytes) 000264[1] 1722795202.464867 mapread 0x506b2c thru 0x51408b (0xd560 bytes) 000273[0] 1722795202.503978 write 0x0c3c5f thru 0x0d37fe (0xfba0 bytes) 000274[0] 1722795202.508165 write 0x3bfaa1 thru 0x3c8699 (0x8bf9 bytes) 000277[0] 1722795202.519323 mapread 0x06606b thru 0x068ea1 (0x2e37 bytes) 000278[0] 1722795202.522449 mapread 0x1952f4 thru 0x196e4c (0x1b59 bytes) 000290[1] 1722795202.539144 mapwrite 0x147e73 thru 0x153b09 (0xbc97 bytes) 000292[0] 1722795202.568380 mapwrite 0x09b280 thru 0x0a55ee (0xa36f bytes) 000293[0] 1722795202.603373 mapread 0x65edea thru 0x666c94 (0x7eab bytes) 000295[1] 1722795202.615821 mapread 0x06abe6 thru 0x070dd1 (0x61ec bytes) 000296[0] 1722795202.621462 read 0x7c034b thru 0x7cf5a7 (0xf25d bytes) 000299[0] 1722795202.639126 trunc from 0x99c674 to 0x42b83e (0x570e35 bytes) 000300[0] 1722795202.646906 trunc from 0x42b83f to 0x5b41de (0x1889a0 bytes) 000302[1] 1722795202.654571 read 0x594e32 thru 0x599fa0 (0x516f bytes) 000304[0] 1722795202.665087 mapread 0x096ea9 thru 0x097140 (0x298 bytes) 000307[1] 1722795202.691701 mapwrite 0x208830 thru 0x20b514 (0x2ce5 bytes) 000308[1] 1722795202.731121 mapread 0x05f7b8 thru 0x0694db (0x9d24 bytes) 000309[1] 1722795202.736167 read 0x4db6ff thru 0x4e447e (0x8d80 bytes) 000312[0] 1722795202.762727 mapread 0x54d695 thru 0x557314 (0x9c80 bytes) 000313[0] 1722795202.784237 mapwrite 0x608958 thru 0x60cca6 (0x434f bytes) 000316[1] 1722795202.826052 read 0x145683 thru 0x154ffb (0xf979 bytes) 000318[0] 1722795202.855371 trunc from 0x60cca7 to 0x3bf779 (0x24d52d bytes) 000320[0] 1722795202.875100 write 0x76bf64 thru 0x77bf23 (0xffc0 bytes) 000322[1] 1722795202.878026 trunc from 0x77bf24 to 0x058209 (0x723d1a bytes) 000325[0] 1722795202.890215 mapread 0x0200cc thru 0x02bf15 (0xbe4a bytes) 000327[0] 1722795202.913739 write 0x607aa7 thru 0x60c909 (0x4e63 bytes) 000329[0] 1722795202.918286 mapread 0x2dfaaf thru 0x2e025d (0x7af bytes) 000330[0] 1722795202.920968 read 0x34e4d4 thru 0x35cebf (0xe9ec bytes) 000331[1] 1722795202.943739 trunc from 0x60c90a to 0x806f1a (0x1fa611 bytes) 000333[1] 1722795202.962056 read 0x79cbc8 thru 0x7aab29 (0xdf62 bytes) 000334[1] 1722795203.002121 read 0x67472f thru 0x67606e (0x1940 bytes) 000336[0] 1722795203.006559 read 0x08aa3d thru 0x08ef2f (0x44f3 bytes) 000338[0] 1722795203.039184 write 0x4a2fc0 thru 0x4ad007 (0xa048 bytes) 000340[1] 1722795203.072562 read 0x161b62 thru 0x16c2f8 (0xa797 bytes) 000341[0] 1722795203.105442 read 0x6ae3c9 thru 0x6b1b73 (0x37ab bytes) 000343[0] 1722795203.109672 mapwrite 0x672711 thru 0x6823fe (0xfcee bytes) 000344[1] 1722795203.141593 write 0x86e6c6 thru 0x8756f5 (0x7030 bytes) 000346[0] 1722795203.162161 trunc from 0x8756f6 to 0x5fe65f (0x277096 bytes) 000347[0] 1722795203.183478 trunc from 0x5fe660 to 0x1fc006 (0x402659 bytes) 000348[0] 1722795203.195700 read 0x12589b thru 0x125a6a (0x1d0 bytes) 000350[0] 1722795203.214720 write 0x6a8806 thru 0x6ab7a5 (0x2fa0 bytes) 000351[0] 1722795203.221939 read 0x61158e thru 0x611904 (0x377 bytes) 000352[0] 1722795203.224047 mapread 0x4acd82 thru 0x4bc9aa (0xfc29 bytes) 000353[0] 1722795203.244575 write 0x7e1ad1 thru 0x7e1fe1 (0x511 bytes) 000356[0] 1722795203.251530 trunc from 0x7e1fe2 to 0x324b4d (0x4bd494 bytes) 000357[0] 1722795203.259795 trunc from 0x324b4e to 0x2d2940 (0x5220d bytes) 000358[1] 1722795203.267321 read 0x035e55 thru 0x045b7a (0xfd26 bytes) 000361[1] 1722795203.295209 trunc from 0x2d2941 to 0x0a5c7c (0x22ccc4 bytes) 000364[0] 1722795203.316198 read 0x010871 thru 0x015624 (0x4db4 bytes) 000365[0] 1722795203.331515 mapread 0x062994 thru 0x07169a (0xed07 bytes) 000366[1] 1722795203.334865 write 0x6ae2b7 thru 0x6bdcb6 (0xfa00 bytes) 000368[0] 1722795203.341096 read 0x5f6235 thru 0x601148 (0xaf14 bytes) 000370[0] 1722795203.363129 read 0x47c279 thru 0x480995 (0x471d bytes) 000372[0] 1722795203.370407 write 0x2b9b9e thru 0x2baad4 (0xf37 bytes) 000374[1] 1722795203.380003 mapread 0x64b460 thru 0x64b49c (0x3d bytes) 000377[0] 1722795203.386525 mapread 0x53aa28 thru 0x549b26 (0xf0ff bytes) 000379[0] 1722795203.408077 trunc from 0x6bdcb7 to 0x449a18 (0x27429e bytes) 000382[1] 1722795203.427735 read 0x0f5056 thru 0x0f774b (0x26f6 bytes) 000387[0] 1722795203.467982 mapread 0x0e567d thru 0x0e916c (0x3af0 bytes) 000389[1] 1722795203.494687 read 0x35d0ec thru 0x361210 (0x4125 bytes) 000391[0] 1722795203.500166 read 0x00b019 thru 0x016d6e (0xbd56 bytes) 000392[1] 1722795203.523677 mapwrite 0x82d95a thru 0x835ac0 (0x8167 bytes) 000393[1] 1722795203.594117 read 0x30b670 thru 0x31347f (0x7e10 bytes) 000395[0] 1722795203.594743 read 0x3ce225 thru 0x3d9314 (0xb0f0 bytes) 000398[1] 1722795203.632345 mapwrite 0x03a4e6 thru 0x046b85 (0xc6a0 bytes) 000405[1] 1722795203.684354 read 0x6e7d3f thru 0x6f6011 (0xe2d3 bytes) 000406[1] 1722795203.687233 write 0x2da744 thru 0x2dbf14 (0x17d1 bytes) 000407[1] 1722795203.691089 mapwrite 0x5ad14a thru 0x5b24a1 (0x5358 bytes) 000409[1] 1722795203.711080 mapwrite 0x674748 thru 0x678a04 (0x42bd bytes) 000413[0] 1722795203.728232 mapwrite 0x541d8a thru 0x54d88d (0xbb04 bytes) 000414[1] 1722795203.779693 mapwrite 0x84ea27 thru 0x8575e2 (0x8bbc bytes) 000416[1] 1722795203.823371 write 0x176842 thru 0x17d60f (0x6dce bytes) 000418[0] 1722795203.824023 read 0x84d0bf thru 0x8533fb (0x633d bytes) 000421[0] 1722795203.866284 mapread 0x5d7c62 thru 0x5e692c (0xeccb bytes) 000423[0] 1722795203.900689 write 0x69f25c thru 0x6aba0f (0xc7b4 bytes) 000425[1] 1722795203.912873 read 0x27add4 thru 0x281dab (0x6fd8 bytes) 000427[0] 1722795203.936381 mapwrite 0x2589bb thru 0x25dbf3 (0x5239 bytes) 000430[0] 1722795203.983845 write 0x530f56 thru 0x5364c4 (0x556f bytes) 000431[1] 1722795203.989830 trunc from 0x8575e3 to 0x13a425 (0x71d1bd bytes) 000432[0] 1722795204.001538 write 0x361991 thru 0x36f8cf (0xdf3f bytes) 000433[1] 1722795204.010041 write 0x1247f6 thru 0x12f331 (0xab3c bytes) 000434[0] 1722795204.026756 write 0x317069 thru 0x3202cb (0x9263 bytes) 000435[1] 1722795204.040658 write 0x49acf6 thru 0x49fd16 (0x5021 bytes) 000438[1] 1722795204.054315 write 0x939d94 thru 0x948b71 (0xedde bytes) 000439[0] 1722795204.065688 mapread 0x243a38 thru 0x253052 (0xf61b bytes) 000442[0] 1722795204.106258 read 0x1c0f75 thru 0x1c449e (0x352a bytes) 000443[0] 1722795204.134433 trunc from 0x948b72 to 0x957881 (0xed10 bytes) 000444[1] 1722795204.155428 write 0x7fc4af thru 0x80ac64 (0xe7b6 bytes) 000445[0] 1722795204.192529 trunc from 0x957882 to 0x88ae8d (0xcc9f4 bytes) 000446[1] 1722795204.213003 trunc from 0x88ae8e to 0x3dca5d (0x4ae430 bytes) 000450[0] 1722795204.229709 mapwrite 0x1150fc thru 0x11b587 (0x648c bytes) 000453[0] 1722795204.272354 write 0x9d89e2 thru 0x9e6130 (0xd74f bytes) 000454[1] 1722795204.277891 mapread 0x03502e thru 0x03a492 (0x5465 bytes) 000456[0] 1722795204.309662 mapwrite 0x02e44f thru 0x0380d6 (0x9c88 bytes) 000458[1] 1722795204.341471 read 0x77ed58 thru 0x788d28 (0x9fd1 bytes) 000460[1] 1722795204.358014 trunc from 0x9e6131 to 0x9504d5 (0x95c5b bytes) 000461[0] 1722795204.363954 read 0x749c4a thru 0x74a0b4 (0x46b bytes) 000462[0] 1722795204.369736 read 0x188134 thru 0x194b2c (0xc9f9 bytes) 000466[1] 1722795204.407619 mapwrite 0x17a2bf thru 0x181dc2 (0x7b04 bytes) 000468[0] 1722795204.459375 mapread 0x1c9347 thru 0x1cc01f (0x2cd9 bytes) 000469[1] 1722795204.472947 write 0x157d00 thru 0x15b843 (0x3b44 bytes) 000470[0] 1722795204.506078 write 0x3114a7 thru 0x320a50 (0xf5aa bytes) 000472[1] 1722795204.520725 mapwrite 0x7cd7a6 thru 0x7d657a (0x8dd5 bytes) 000475[0] 1722795204.576324 trunc from 0x9504d6 to 0x33924e (0x617287 bytes) 000476[1] 1722795204.582896 trunc from 0x33924f to 0x4591cd (0x11ff7f bytes) 000481[0] 1722795204.594377 mapwrite 0x05f000 thru 0x06bb67 (0xcb68 bytes) 000483[1] 1722795204.636175 mapread 0x08342a thru 0x08ce20 (0x99f7 bytes) 000485[0] 1722795204.654296 write 0x173394 thru 0x17c098 (0x8d05 bytes) 000486[0] 1722795204.661487 write 0x5f1c27 thru 0x5f9616 (0x79f0 bytes) 000488[0] 1722795204.664615 trunc from 0x5f9617 to 0x46d559 (0x18c0bd bytes) 000489[0] 1722795204.671413 trunc from 0x46d55a to 0x3b2062 (0xbb4f7 bytes) 000490[1] 1722795204.678110 write 0x8744c6 thru 0x881052 (0xcb8d bytes) 000491[0] 1722795204.686868 write 0x9caea8 thru 0x9cef85 (0x40de bytes) 000498[1] 1722795204.702516 read 0x27ca5b thru 0x281d9e (0x5344 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 5204 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x5d384f 000003[0] 1722795204.932959 trunc from 00000000 to 0x5d384e (0x5d384f bytes) 000004[0] 1722795204.939558 write_OD 0x528000 thru 0x52afff (0x3000 bytes) 000006[0] 1722795204.954524 trunc from 0x5d384f to 0x1547c4 (0x47f08a bytes) 000007[0] 1722795204.962706 write_OD 0x7c3000 thru 0x7cffff (0xd000 bytes) 000008[0] 1722795204.980535 mapread 0x275000 thru 0x276220 (0x1221 bytes) 000009[0] 1722795204.986413 trunc from 0x7d0000 to 0x26cb82 (0x56347d bytes) 000016[1] 1722795205.008112 mapread 0x1ed000 thru 0x1f28de (0x58df bytes) 000018[1] 1722795205.016392 read_OD 0x23a000 thru 0x246fff (0xd000 bytes) 000019[1] 1722795205.018457 mapread 0x107000 thru 0x10a9a4 (0x39a5 bytes) 000020[1] 1722795205.022388 trunc from 0x26cb83 to 0x37eeff (0x11237d bytes) 000021[0] 1722795205.030247 read_OD 0x2d5000 thru 0x2e3fff (0xf000 bytes) 000022[1] 1722795205.035983 mapread 0x320000 thru 0x327007 (0x7008 bytes) 000023[1] 1722795205.042503 write_OD 0x427000 thru 0x429fff (0x3000 bytes) 000025[0] 1722795205.057602 mapread 0x325000 thru 0x32b227 (0x6228 bytes) 000027[0] 1722795205.060602 trunc from 0x42a000 to 0x22627a (0x203d85 bytes) 000029[0] 1722795205.067388 read_OD 0x02c000 thru 0x034fff (0x9000 bytes) 000030[0] 1722795205.072759 read_OD 0x194000 thru 0x1a2fff (0xf000 bytes) 000035[1] 1722795205.076342 read_OD 0x08e000 thru 0x09afff (0xd000 bytes) truncating to largest ever: 0x6b07c1 000036[1] 1722795205.079167 trunc from 0x22627b to 0x6b07c0 (0x48a546 bytes) truncating to largest ever: 0x72814a 000037[0] 1722795205.086364 trunc from 0x6b07c1 to 0x728149 (0x77989 bytes) 000038[1] 1722795205.094991 trunc from 0x72814a to 0x718f2e (0xf21b bytes) 000039[0] 1722795205.099373 trunc from 0x718f2f to 0x221b61 (0x4f73cd bytes) 000040[1] 1722795205.104650 mapwrite 0x71b000 thru 0x71d8fd (0x28fe bytes) 000041[0] 1722795205.166416 read_OD 0x2f7000 thru 0x305fff (0xf000 bytes) 000045[1] 1722795205.180554 write_OD 0x805000 thru 0x807fff (0x3000 bytes) 000047[0] 1722795205.196483 mapwrite 0x019000 thru 0x025d71 (0xcd72 bytes) truncating to largest ever: 0x772ddf 000054[0] 1722795205.264689 trunc from 0x808000 to 0x772dde (0x95221 bytes) 000056[1] 1722795205.269867 mapread 0x655000 thru 0x65d3f9 (0x83fa bytes) 000057[0] 1722795205.294852 mapwrite 0x366000 thru 0x3683cc (0x23cd bytes) 000061[1] 1722795205.318582 trunc from 0x772ddf to 0x1b3bf6 (0x5bf1e8 bytes) 000062[1] 1722795205.328291 read_OD 0x154000 thru 0x162fff (0xf000 bytes) 000065[1] 1722795205.335725 write_OD 0x449000 thru 0x452fff (0xa000 bytes) 000066[1] 1722795205.352532 trunc from 0x453000 to 0x74fa0b (0x2fca0c bytes) 000067[1] 1722795205.362731 trunc from 0x74fa0c to 0x736f21 (0x18aea bytes) 000069[1] 1722795205.367986 mapwrite 0x497000 thru 0x498302 (0x1303 bytes) 000072[1] 1722795205.420987 mapwrite 0x629000 thru 0x62d861 (0x4862 bytes) 000073[0] 1722795205.439683 trunc from 0x736f22 to 0x35efd6 (0x3d7f4b bytes) 000074[1] 1722795205.451941 mapwrite 0x9b7000 thru 0x9bcdaf (0x5db0 bytes) 000076[0] 1722795205.491378 trunc from 0x9bcdb0 to 0x33ffd5 (0x67cdda bytes) 000077[1] 1722795205.500707 trunc from 0x33ffd6 to 0x431767 (0xf1792 bytes) 000078[1] 1722795205.507026 mapread 0x3f3000 thru 0x3f3eb0 (0xeb1 bytes) 000080[0] 1722795205.509027 trunc from 0x431768 to 0x5376fc (0x105f95 bytes) 000081[0] 1722795205.517525 read_OD 0x085000 thru 0x08afff (0x6000 bytes) 000082[0] 1722795205.521544 mapwrite 0x22f000 thru 0x23d8e4 (0xe8e5 bytes) 000087[0] 1722795205.585093 read_OD 0x324000 thru 0x325fff (0x2000 bytes) 000088[1] 1722795205.593191 write_OD 0x64e000 thru 0x651fff (0x4000 bytes) 000090[1] 1722795205.607964 mapread 0x536000 thru 0x53b19d (0x519e bytes) 000091[0] 1722795205.614953 trunc from 0x652000 to 0x330469 (0x321b96 bytes) 000093[0] 1722795205.628322 write_OD 0x0fe000 thru 0x102fff (0x5000 bytes) 000097[1] 1722795205.643385 mapwrite 0x0c2000 thru 0x0c62cb (0x42cc bytes) 000099[1] 1722795205.668584 read_OD 0x311000 thru 0x31ffff (0xf000 bytes) 000101[0] 1722795205.672611 read_OD 0x01f000 thru 0x023fff (0x5000 bytes) 000104[0] 1722795205.675054 write_OD 0x169000 thru 0x171fff (0x9000 bytes) 000106[0] 1722795205.692064 mapread 0x221000 thru 0x22a7f6 (0x97f7 bytes) 000108[0] 1722795205.713179 mapwrite 0x274000 thru 0x27ec66 (0xac67 bytes) 000110[1] 1722795205.770869 write_OD 0x624000 thru 0x62afff (0x7000 bytes) 000112[0] 1722795205.788724 mapread 0x5fc000 thru 0x606b74 (0xab75 bytes) 000118[0] 1722795205.808294 write_OD 0x125000 thru 0x132fff (0xe000 bytes) 000122[1] 1722795205.827151 mapwrite 0x458000 thru 0x45ae91 (0x2e92 bytes) 000125[0] 1722795205.877679 mapwrite 0x37e000 thru 0x381ea5 (0x3ea6 bytes) 000126[0] 1722795205.905714 mapread 0x48f000 thru 0x490cba (0x1cbb bytes) 000131[0] 1722795205.920811 read_OD 0x5f0000 thru 0x5f1fff (0x2000 bytes) 000135[1] 1722795205.925474 trunc from 0x62b000 to 0x227c3a (0x4033c5 bytes) 000136[0] 1722795205.934898 mapwrite 0x2a7000 thru 0x2b59b9 (0xe9ba bytes) 000140[1] 1722795205.988334 mapwrite 0x211000 thru 0x2136cb (0x26cc bytes) 000144[0] 1722795206.022132 mapwrite 0x2ca000 thru 0x2cfd86 (0x5d87 bytes) 000146[1] 1722795206.049020 read_OD 0x2bc000 thru 0x2befff (0x3000 bytes) 000147[0] 1722795206.054519 mapread 0x0c9000 thru 0x0d8404 (0xf405 bytes) 000151[0] 1722795206.078968 mapwrite 0x0ca000 thru 0x0d3806 (0x9807 bytes) 000153[1] 1722795206.130227 read_OD 0x141000 thru 0x142fff (0x2000 bytes) 000154[1] 1722795206.135257 trunc from 0x2cfd87 to 0x65431f (0x384599 bytes) 000158[1] 1722795206.147552 mapread 0x0da000 thru 0x0dad59 (0xd5a bytes) 000159[1] 1722795206.186043 mapread 0x408000 thru 0x409516 (0x1517 bytes) 000162[0] 1722795206.190616 read_OD 0x230000 thru 0x23efff (0xf000 bytes) 000163[0] 1722795206.202838 trunc from 0x654320 to 0x5ab345 (0xa8fda bytes) 000168[1] 1722795206.222585 mapread 0x1c9000 thru 0x1d4658 (0xb659 bytes) 000169[1] 1722795206.243693 mapwrite 0x42f000 thru 0x436950 (0x7951 bytes) 000173[1] 1722795206.285970 mapread 0x22c000 thru 0x22e0dd (0x20de bytes) 000176[1] 1722795206.298815 read_OD 0x1a8000 thru 0x1adfff (0x6000 bytes) 000177[0] 1722795206.303365 read_OD 0x25a000 thru 0x266fff (0xd000 bytes) 000180[1] 1722795206.307320 write_OD 0x202000 thru 0x203fff (0x2000 bytes) 000182[1] 1722795206.327905 trunc from 0x5ab346 to 0x25a40f (0x350f36 bytes) 000184[0] 1722795206.340451 mapwrite 0x36f000 thru 0x37e0e8 (0xf0e9 bytes) 000186[0] 1722795206.393344 write_OD 0x2c1000 thru 0x2cbfff (0xb000 bytes) 000187[0] 1722795206.410669 write_OD 0x7c5000 thru 0x7ccfff (0x8000 bytes) 000188[1] 1722795206.429225 mapread 0x332000 thru 0x33ae3e (0x8e3f bytes) 000190[0] 1722795206.474911 mapread 0x395000 thru 0x397c7d (0x2c7e bytes) 000191[0] 1722795206.498905 trunc from 0x7cd000 to 0x150505 (0x67cafa bytes) 000193[1] 1722795206.514932 read_OD 0x05f000 thru 0x06cfff (0xe000 bytes) 000194[1] 1722795206.523363 trunc from 0x150506 to 0x5291a4 (0x3d8c9f bytes) 000196[0] 1722795206.531560 write_OD 0x956000 thru 0x956fff (0x1000 bytes) 000199[1] 1722795206.552645 write_OD 0x302000 thru 0x304fff (0x3000 bytes) 000205[1] 1722795206.572456 mapread 0x698000 thru 0x6a10f0 (0x90f1 bytes) 000208[1] 1722795206.629907 mapwrite 0x8b1000 thru 0x8b7807 (0x6808 bytes) 000209[0] 1722795206.683166 write_OD 0x831000 thru 0x832fff (0x2000 bytes) 000211[1] 1722795206.700720 mapwrite 0x098000 thru 0x0a6c0b (0xec0c bytes) 000213[1] 1722795206.744050 read_OD 0x851000 thru 0x855fff (0x5000 bytes) 000217[1] 1722795206.749500 mapread 0x0fb000 thru 0x106ca5 (0xbca6 bytes) 000219[0] 1722795206.800468 mapread 0x15f000 thru 0x166812 (0x7813 bytes) 000220[0] 1722795206.821004 mapwrite 0x427000 thru 0x43182d (0xa82e bytes) 000221[1] 1722795206.860744 mapwrite 0x488000 thru 0x4978c4 (0xf8c5 bytes) 000222[0] 1722795206.898410 mapread 0x2e2000 thru 0x2eceac (0xaead bytes) 000223[0] 1722795206.914616 write_OD 0x92c000 thru 0x934fff (0x9000 bytes) 000225[1] 1722795206.932154 mapwrite 0x917000 thru 0x91e71a (0x771b bytes) 000228[1] 1722795206.961785 read_OD 0x3b2000 thru 0x3b5fff (0x4000 bytes) 000229[1] 1722795206.965540 mapread 0x12b000 thru 0x12c6aa (0x16ab bytes) 000230[0] 1722795207.015084 trunc from 0x957000 to 0x42765f (0x52f9a0 bytes) 000231[0] 1722795207.028600 read_OD 0x178000 thru 0x17ffff (0x8000 bytes) 000233[0] 1722795207.034514 mapread 0x06f000 thru 0x06f342 (0x343 bytes) truncating to largest ever: 0x823d12 000235[1] 1722795207.058605 trunc from 0x427660 to 0x823d11 (0x3fc6b2 bytes) 000238[0] 1722795207.080594 trunc from 0x823d12 to 0x228710 (0x5fb601 bytes) 000239[1] 1722795207.089857 mapwrite 0x042000 thru 0x050400 (0xe401 bytes) 000240[0] 1722795207.128981 mapwrite 0x1b4000 thru 0x1bd511 (0x9512 bytes) 000242[0] 1722795207.165562 write_OD 0x682000 thru 0x68cfff (0xb000 bytes) 000245[1] 1722795207.184010 write_OD 0x9a0000 thru 0x9a8fff (0x9000 bytes) 000246[1] 1722795207.201333 mapwrite 0x14b000 thru 0x150fd5 (0x5fd6 bytes) 000249[0] 1722795207.236530 trunc from 0x9a9000 to 0x822dad (0x186252 bytes) 000250[1] 1722795207.247266 mapwrite 0x828000 thru 0x82bc22 (0x3c23 bytes) truncating to largest ever: 0x968a9a 000251[1] 1722795207.275046 trunc from 0x82bc23 to 0x968a99 (0x13ce77 bytes) 000252[0] 1722795207.279170 trunc from 0x968a9a to 0x902982 (0x66117 bytes) 000253[0] 1722795207.285646 write_OD 0x4c0000 thru 0x4cbfff (0xc000 bytes) 000255[1] 1722795207.304383 mapwrite 0x8b5000 thru 0x8c3fab (0xefac bytes) 000256[1] 1722795207.341673 mapwrite 0x686000 thru 0x6880e2 (0x20e3 bytes) truncating to largest ever: 0x9953ce 000262[0] 1722795207.372064 trunc from 0x902983 to 0x9953cd (0x92a4b bytes) 000264[0] 1722795207.382818 write_OD 0x86c000 thru 0x879fff (0xe000 bytes) 000265[0] 1722795207.397180 mapread 0x3e3000 thru 0x3e889e (0x589f bytes) 000267[0] 1722795207.432429 read_OD 0x1dc000 thru 0x1e7fff (0xc000 bytes) 000268[0] 1722795207.435355 read_OD 0x364000 thru 0x36dfff (0xa000 bytes) 000270[1] 1722795207.438041 read_OD 0x5ea000 thru 0x5f8fff (0xf000 bytes) 000271[1] 1722795207.440769 trunc from 0x9953ce to 0x6013e4 (0x393fe9 bytes) 000273[0] 1722795207.455241 read_OD 0x480000 thru 0x484fff (0x5000 bytes) 000275[0] 1722795207.461031 read_OD 0x1ba000 thru 0x1c2fff (0x9000 bytes) 000276[0] 1722795207.463567 read_OD 0x0d8000 thru 0x0e3fff (0xc000 bytes) 000277[1] 1722795207.466170 mapread 0x5dd000 thru 0x5e5c2c (0x8c2d bytes) 000279[1] 1722795207.496552 read_OD 0x5e6000 thru 0x5e6fff (0x1000 bytes) 000281[0] 1722795207.498603 mapread 0x1f8000 thru 0x2055cb (0xd5cc bytes) 000284[1] 1722795207.531851 read_OD 0x3cd000 thru 0x3dbfff (0xf000 bytes) 000285[1] 1722795207.535194 read_OD 0x031000 thru 0x031fff (0x1000 bytes) 000286[0] 1722795207.536945 mapwrite 0x568000 thru 0x575965 (0xd966 bytes) 000287[1] 1722795207.595987 mapwrite 0x476000 thru 0x48222a (0xc22b bytes) 000288[1] 1722795207.646898 mapwrite 0x11b000 thru 0x11e767 (0x3768 bytes) 000291[0] 1722795207.662363 write_OD 0x223000 thru 0x231fff (0xf000 bytes) 000292[1] 1722795207.682691 mapwrite 0x482000 thru 0x48dfe9 (0xbfea bytes) 000293[1] 1722795207.711891 mapread 0x0ba000 thru 0x0bd751 (0x3752 bytes) 000294[0] 1722795207.716894 mapread 0x568000 thru 0x570fbc (0x8fbd bytes) 000295[1] 1722795207.731654 mapwrite 0x82d000 thru 0x831160 (0x4161 bytes) 000297[0] 1722795207.776657 mapread 0x0b5000 thru 0x0c4bdf (0xfbe0 bytes) 000298[0] 1722795207.818704 write_OD 0x40d000 thru 0x40dfff (0x1000 bytes) 000304[0] 1722795207.837206 trunc from 0x831161 to 0x7336d8 (0xfda88 bytes) 000307[0] 1722795207.850411 mapwrite 0x3fa000 thru 0x4032ce (0x92cf bytes) 000308[0] 1722795207.886120 write_OD 0x47f000 thru 0x484fff (0x6000 bytes) 000311[0] 1722795207.902612 mapwrite 0x644000 thru 0x6476f9 (0x36fa bytes) 000312[1] 1722795207.923565 mapread 0x544000 thru 0x550c22 (0xcc23 bytes) 000313[1] 1722795207.942545 read_OD 0x560000 thru 0x561fff (0x2000 bytes) 000319[1] 1722795207.945859 trunc from 0x7336d9 to 0x47f593 (0x2b4145 bytes) 000321[0] 1722795207.953968 write_OD 0x106000 thru 0x10ffff (0xa000 bytes) 000324[0] 1722795207.975184 write_OD 0x3de000 thru 0x3dffff (0x2000 bytes) 000328[0] 1722795207.993353 write_OD 0x26c000 thru 0x26efff (0x3000 bytes) 000330[1] 1722795208.013025 write_OD 0x395000 thru 0x39efff (0xa000 bytes) 000332[1] 1722795208.030256 mapwrite 0x32b000 thru 0x33a2a3 (0xf2a4 bytes) 000337[0] 1722795208.086109 mapread 0x1c7000 thru 0x1c7275 (0x276 bytes) 000338[0] 1722795208.095696 read_OD 0x231000 thru 0x233fff (0x3000 bytes) 000340[1] 1722795208.098085 mapread 0x110000 thru 0x1122f3 (0x22f4 bytes) 000341[0] 1722795208.101489 mapread 0x064000 thru 0x07322f (0xf230 bytes) 000342[1] 1722795208.122558 mapwrite 0x936000 thru 0x940c19 (0xac1a bytes) 000344[0] 1722795208.180770 mapread 0x38d000 thru 0x398984 (0xb985 bytes) 000345[0] 1722795208.233774 write_OD 0x935000 thru 0x939fff (0x5000 bytes) 000346[1] 1722795208.255754 mapread 0x406000 thru 0x409786 (0x3787 bytes) 000347[1] 1722795208.261764 mapwrite 0x236000 thru 0x242b86 (0xcb87 bytes) 000352[1] 1722795208.311625 mapread 0x320000 thru 0x32899b (0x899c bytes) 000354[0] 1722795208.326498 read_OD 0x6ea000 thru 0x6ecfff (0x3000 bytes) 000355[0] 1722795208.332592 mapwrite 0x19d000 thru 0x1a616a (0x916b bytes) 000358[1] 1722795208.406545 trunc from 0x940c1a to 0x812d44 (0x12ded5 bytes) 000360[0] 1722795208.424666 mapwrite 0x74b000 thru 0x759e1f (0xee20 bytes) 000361[0] 1722795208.457393 write_OD 0x930000 thru 0x93bfff (0xc000 bytes) 000362[1] 1722795208.473563 mapread 0x38c000 thru 0x392a6e (0x6a6f bytes) 000364[0] 1722795208.482346 trunc from 0x93c000 to 0x1ed2fa (0x74ed05 bytes) 000365[0] 1722795208.490522 mapwrite 0x581000 thru 0x58b0d5 (0xa0d6 bytes) 000366[0] 1722795208.526266 trunc from 0x58b0d6 to 0x025340 (0x565d95 bytes) 000369[1] 1722795208.540020 trunc from 0x025341 to 0x70c7ad (0x6e746d bytes) 000370[0] 1722795208.546744 mapread 0x2b6000 thru 0x2c5941 (0xf942 bytes) 000374[0] 1722795208.570104 mapread 0x214000 thru 0x21a6fa (0x66fb bytes) 000376[0] 1722795208.579729 read_OD 0x01a000 thru 0x024fff (0xb000 bytes) 000381[0] 1722795208.582534 mapread 0x20c000 thru 0x21bf0d (0xff0e bytes) 000382[0] 1722795208.627388 mapread 0x270000 thru 0x270ac8 (0xac9 bytes) 000384[0] 1722795208.628154 read_OD 0x142000 thru 0x147fff (0x6000 bytes) 000386[0] 1722795208.632451 trunc from 0x70c7ae to 0x959293 (0x24cae6 bytes) 000387[0] 1722795208.645833 mapwrite 0x3dd000 thru 0x3e10df (0x40e0 bytes) 000388[1] 1722795208.713747 mapwrite 0x773000 thru 0x77d2b1 (0xa2b2 bytes) 000390[0] 1722795208.758980 mapwrite 0x33b000 thru 0x343267 (0x8268 bytes) 000392[1] 1722795208.792243 read_OD 0x23b000 thru 0x242fff (0x8000 bytes) 000393[1] 1722795208.797113 read_OD 0x4f8000 thru 0x4fffff (0x8000 bytes) 000398[1] 1722795208.800527 read_OD 0x128000 thru 0x135fff (0xe000 bytes) 000401[1] 1722795208.803478 read_OD 0x427000 thru 0x42dfff (0x7000 bytes) 000404[0] 1722795208.806101 trunc from 0x959294 to 0x6ea6d7 (0x26ebbc bytes) 000405[0] 1722795208.812551 read_OD 0x060000 thru 0x068fff (0x9000 bytes) truncating to largest ever: 0x9f0fe6 000409[1] 1722795208.818680 trunc from 0x6ea6d8 to 0x9f0fe5 (0x30690e bytes) 000410[1] 1722795208.825379 read_OD 0x466000 thru 0x467fff (0x2000 bytes) 000414[0] 1722795208.830614 mapread 0x5b0000 thru 0x5be07c (0xe07d bytes) 000415[0] 1722795208.892538 trunc from 0x9f0fe6 to 0x4eb2d6 (0x505d0f bytes) 000418[0] 1722795208.914186 mapread 0x22c000 thru 0x2328c8 (0x68c9 bytes) 000419[0] 1722795208.921786 write_OD 0x183000 thru 0x191fff (0xf000 bytes) 000420[0] 1722795208.939728 trunc from 0x4eb2d7 to 0x6c2501 (0x1d722b bytes) 000424[0] 1722795208.957325 read_OD 0x57c000 thru 0x57ffff (0x4000 bytes) 000426[0] 1722795208.962953 mapwrite 0x5a3000 thru 0x5ae4df (0xb4e0 bytes) 000428[0] 1722795209.025261 read_OD 0x4e6000 thru 0x4eafff (0x5000 bytes) 000429[0] 1722795209.034878 mapwrite 0x98d000 thru 0x994735 (0x7736 bytes) 000431[0] 1722795209.119774 mapwrite 0x705000 thru 0x71481f (0xf820 bytes) 000432[0] 1722795209.140402 mapread 0x0a3000 thru 0x0aa188 (0x7189 bytes) 000433[0] 1722795209.141983 write_OD 0x03e000 thru 0x03efff (0x1000 bytes) 000434[0] 1722795209.164257 mapwrite 0x115000 thru 0x11f1f2 (0xa1f3 bytes) 000435[0] 1722795209.195566 trunc from 0x994736 to 0x3b3f75 (0x5e07c0 bytes) 000437[0] 1722795209.204973 mapwrite 0x055000 thru 0x0582ce (0x32cf bytes) 000439[1] 1722795209.224822 mapwrite 0x8b4000 thru 0x8b9cca (0x5ccb bytes) 000441[0] 1722795209.297885 write_OD 0x2a6000 thru 0x2b3fff (0xe000 bytes) 000443[1] 1722795209.316782 write_OD 0x23e000 thru 0x242fff (0x5000 bytes) 000445[1] 1722795209.331901 write_OD 0x0f2000 thru 0x0f7fff (0x6000 bytes) 000447[1] 1722795209.349920 write_OD 0x288000 thru 0x28cfff (0x5000 bytes) 000448[0] 1722795209.370228 trunc from 0x8b9ccb to 0x950ba2 (0x96ed8 bytes) 000449[0] 1722795209.378001 read_OD 0x8f1000 thru 0x8fdfff (0xd000 bytes) 000452[1] 1722795209.385889 trunc from 0x950ba3 to 0x9113d8 (0x3f7ca bytes) 000454[1] 1722795209.408811 mapwrite 0x6c4000 thru 0x6d20e1 (0xe0e2 bytes) 000456[1] 1722795209.432856 mapread 0x15f000 thru 0x162d78 (0x3d79 bytes) 000457[1] 1722795209.437025 mapwrite 0x603000 thru 0x60c660 (0x9661 bytes) 000460[1] 1722795209.458782 mapread 0x7f7000 thru 0x802735 (0xb736 bytes) 000461[0] 1722795209.464493 mapwrite 0x341000 thru 0x34bafb (0xaafc bytes) 000462[1] 1722795209.531910 read_OD 0x482000 thru 0x48bfff (0xa000 bytes) 000466[0] 1722795209.545458 trunc from 0x9113d9 to 0x1c0fab (0x75042d bytes) 000467[1] 1722795209.552614 trunc from 0x1c0fac to 0x962103 (0x7a1158 bytes) 000470[0] 1722795209.569794 trunc from 0x962104 to 0x5b77b3 (0x3aa950 bytes) 000472[0] 1722795209.581678 mapwrite 0x2f0000 thru 0x2f0979 (0x97a bytes) 000474[0] 1722795209.601691 read_OD 0x039000 thru 0x03ffff (0x7000 bytes) 000476[0] 1722795209.607150 mapwrite 0x9b6000 thru 0x9b7e35 (0x1e36 bytes) 000478[0] 1722795209.690314 trunc from 0x9b7e36 to 0x4e77ca (0x4d066b bytes) 000480[0] 1722795209.705408 read_OD 0x491000 thru 0x49efff (0xe000 bytes) 000483[1] 1722795209.715360 write_OD 0x432000 thru 0x43bfff (0xa000 bytes) 000486[1] 1722795209.730059 mapwrite 0x193000 thru 0x1988b1 (0x58b2 bytes) 000487[0] 1722795209.750554 mapwrite 0x90b000 thru 0x91abc8 (0xfbc9 bytes) 000489[0] 1722795209.821515 write_OD 0x234000 thru 0x23ffff (0xc000 bytes) 000490[0] 1722795209.847819 mapread 0x0ac000 thru 0x0b2473 (0x6474 bytes) 000491[1] 1722795209.858377 mapread 0x35e000 thru 0x36149c (0x349d bytes) 000496[1] 1722795209.863121 read_OD 0x4b8000 thru 0x4c3fff (0xc000 bytes) 000499[0] 1722795209.865596 write_OD 0x69b000 thru 0x6a8fff (0xe000 bytes) 000500[0] 1722795209.882267 mapwrite 0x7ba000 thru 0x7c812f (0xe130 bytes) All operations completed A-OK! PASS 16a (11s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 14:13:30 (1722795210) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.126321 s, 66.4 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.118846 s, 70.6 MB/s Chance of close/open is 1 in 50 Seed set to 5211 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x357235 000001[0] 1722795211.840073 trunc from 00000000 to 0x357234 (0x357235 bytes) 000003[1] 1722795211.851223 read 0x23e3cd thru 0x23fba1 (0x17d5 bytes) 000005[0] 1722795211.861758 mapwrite 0x56d904 thru 0x56f291 (0x198e bytes) 000006[0] 1722795211.903975 read 0x1e4cbd thru 0x1e6a93 (0x1dd7 bytes) 000007[1] 1722795211.910725 mapread 0x51a75a thru 0x51abd4 (0x47b bytes) 000010[0] 1722795211.917728 read 0x20bbf7 thru 0x20c7bb (0xbc5 bytes) 000011[0] 1722795211.952079 read 0x2f7471 thru 0x2f8580 (0x1110 bytes) 000017[1] 1722795211.977363 mapread 0x350308 thru 0x350349 (0x42 bytes) 000018[1] 1722795211.985566 read 0x0f643d thru 0x0f65ab (0x16f bytes) 000020[1] 1722795212.008599 write 0x25e954 thru 0x25fdd6 (0x1483 bytes) 000021[1] 1722795212.018052 mapread 0x15ca76 thru 0x15cdaa (0x335 bytes) 000023[0] 1722795212.024508 write 0x0a953a thru 0x0aa24d (0xd14 bytes) 000024[1] 1722795212.050650 mapwrite 0x848c54 thru 0x849755 (0xb02 bytes) 000027[1] 1722795212.103350 mapread 0x09d271 thru 0x09d2d6 (0x66 bytes) 000028[0] 1722795212.108115 read 0x074593 thru 0x075f83 (0x19f1 bytes) 000029[0] 1722795212.132872 mapread 0x16ac9f thru 0x16bd38 (0x109a bytes) 000031[1] 1722795212.135849 mapread 0x47c52e thru 0x47cf1f (0x9f2 bytes) 000032[0] 1722795212.140803 read 0x327d29 thru 0x3283ef (0x6c7 bytes) 000034[0] 1722795212.147519 read 0x3d2fbb thru 0x3d31e7 (0x22d bytes) 000038[0] 1722795212.154301 trunc from 0x849756 to 0x0eeb8a (0x75abcb bytes) 000039[1] 1722795212.174040 write 0x8aa5d5 thru 0x8aafbc (0x9e8 bytes) 000040[1] 1722795212.182434 mapwrite 0x16d53c thru 0x16d582 (0x47 bytes) 000042[0] 1722795212.213627 write 0x256943 thru 0x2578ed (0xfab bytes) 000043[0] 1722795212.246207 write 0x78fe27 thru 0x78ffad (0x187 bytes) 000044[1] 1722795212.270589 mapread 0x49b534 thru 0x49bf47 (0xa14 bytes) 000045[1] 1722795212.282876 mapwrite 0x9d225f thru 0x9d424f (0x1ff1 bytes) 000049[1] 1722795212.331800 write 0x4b7f13 thru 0x4b9872 (0x1960 bytes) 000054[1] 1722795212.342083 mapread 0x2355dd thru 0x23650f (0xf33 bytes) 000055[1] 1722795212.347926 write 0x068a40 thru 0x068c27 (0x1e8 bytes) 000056[1] 1722795212.353295 mapwrite 0x93ace5 thru 0x93bfd5 (0x12f1 bytes) 000060[0] 1722795212.379178 mapread 0x20670f thru 0x2079ef (0x12e1 bytes) 000061[0] 1722795212.403255 read 0x42aacf thru 0x42b56b (0xa9d bytes) 000062[0] 1722795212.436786 mapread 0x6f3743 thru 0x6f4c9a (0x1558 bytes) 000066[0] 1722795212.448029 mapread 0x145ca9 thru 0x1474d4 (0x182c bytes) 000068[0] 1722795212.448621 mapwrite 0x72608c thru 0x727eb4 (0x1e29 bytes) truncating to largest ever: 0x8657c8 000070[1] 1722795212.482940 trunc from 0x9d4250 to 0x8657c7 (0x16ea88 bytes) 000071[0] 1722795212.495859 read 0x707fe4 thru 0x708a7f (0xa9c bytes) 000072[0] 1722795212.500563 mapread 0x6ede1c thru 0x6ef445 (0x162a bytes) 000073[0] 1722795212.509170 mapwrite 0x500409 thru 0x501c6c (0x1864 bytes) 000075[1] 1722795212.532423 write 0x1923b3 thru 0x193005 (0xc53 bytes) 000079[0] 1722795212.552046 mapread 0x3e3d47 thru 0x3e41c8 (0x482 bytes) 000080[1] 1722795212.589590 mapwrite 0x3fa4ca thru 0x3fb8b3 (0x13ea bytes) 000083[0] 1722795212.626738 read 0x6b989b thru 0x6ba9e7 (0x114d bytes) 000086[1] 1722795212.637252 mapread 0x48603e thru 0x487a8c (0x1a4f bytes) 000089[0] 1722795212.652999 write 0x638b96 thru 0x6396ad (0xb18 bytes) 000092[1] 1722795212.667184 read 0x2c429c thru 0x2c46b6 (0x41b bytes) 000096[1] 1722795212.702948 mapwrite 0x006cd3 thru 0x0075dc (0x90a bytes) 000097[0] 1722795212.719933 read 0x75599c thru 0x756b1b (0x1180 bytes) 000100[0] 1722795212.731563 mapwrite 0x85a9d0 thru 0x85b8ce (0xeff bytes) 000105[0] 1722795212.791918 trunc from 0x8657c8 to 0x02c768 (0x83905f bytes) 000107[1] 1722795212.808254 read 0x015ee9 thru 0x0176be (0x17d6 bytes) 000108[1] 1722795212.815760 write 0x46ce2b thru 0x46cf72 (0x148 bytes) 000109[1] 1722795212.821386 read 0x29f246 thru 0x2a05d4 (0x138f bytes) 000111[1] 1722795212.826811 write 0x47d173 thru 0x47e51c (0x13aa bytes) 000114[1] 1722795212.833741 write 0x3e51a6 thru 0x3e5b5a (0x9b5 bytes) 000116[0] 1722795212.839964 mapread 0x2ee422 thru 0x2ee9a4 (0x583 bytes) 000118[0] 1722795212.846422 write 0x023663 thru 0x024311 (0xcaf bytes) 000119[0] 1722795212.853056 trunc from 0x47e51d to 0x72f137 (0x2b0c1b bytes) 000120[1] 1722795212.863228 read 0x29862d thru 0x29a061 (0x1a35 bytes) 000121[0] 1722795212.893060 write 0x631cbf thru 0x6324bb (0x7fd bytes) 000123[1] 1722795212.901149 trunc from 0x72f138 to 0x06eaa8 (0x6c068f bytes) 000125[0] 1722795212.919579 trunc from 0x06eaa9 to 0x268250 (0x1f97a8 bytes) 000126[1] 1722795212.926941 write 0x79ee77 thru 0x7a048a (0x1614 bytes) 000127[1] 1722795212.933291 trunc from 0x7a048b to 0x799362 (0x7128 bytes) 000130[0] 1722795212.947588 read 0x32eaa1 thru 0x32ff2a (0x148a bytes) 000133[0] 1722795212.979117 trunc from 0x799363 to 0x73c1a9 (0x5d1b9 bytes) 000134[0] 1722795212.993343 mapread 0x52f13a thru 0x530009 (0xed0 bytes) 000135[0] 1722795213.014655 read 0x648d75 thru 0x64aae0 (0x1d6c bytes) 000137[1] 1722795213.017793 write 0x2d1ad8 thru 0x2d2c65 (0x118e bytes) 000140[0] 1722795213.030771 read 0x2dd24f thru 0x2dd73d (0x4ef bytes) 000142[1] 1722795213.061027 read 0x6c2444 thru 0x6c37d1 (0x138e bytes) 000143[0] 1722795213.102346 trunc from 0x73c1aa to 0x31ea59 (0x41d750 bytes) 000144[0] 1722795213.119855 write 0x5930fe thru 0x594fe4 (0x1ee7 bytes) 000145[1] 1722795213.124324 mapread 0x592ab1 thru 0x593e54 (0x13a4 bytes) 000147[1] 1722795213.136136 mapwrite 0x9b5acd thru 0x9b5af0 (0x24 bytes) 000148[0] 1722795213.165170 mapread 0x275173 thru 0x27558e (0x41c bytes) 000150[1] 1722795213.171124 mapwrite 0x76f835 thru 0x77182a (0x1ff6 bytes) 000153[1] 1722795213.188846 trunc from 0x9b5af1 to 0x2380bb (0x77da35 bytes) 000154[1] 1722795213.194559 read 0x1ad789 thru 0x1add7d (0x5f5 bytes) 000156[1] 1722795213.215726 mapread 0x0b30bc thru 0x0b4aa2 (0x19e7 bytes) 000157[0] 1722795213.220063 mapread 0x04092c thru 0x041f7d (0x1652 bytes) 000158[1] 1722795213.231086 mapread 0x1d74fb thru 0x1d9312 (0x1e18 bytes) 000159[1] 1722795213.235529 mapread 0x127db8 thru 0x129348 (0x1591 bytes) 000162[1] 1722795213.245915 write 0x463fb3 thru 0x464d5c (0xdaa bytes) 000163[0] 1722795213.246909 mapread 0x2c95b4 thru 0x2cb0de (0x1b2b bytes) 000165[1] 1722795213.260584 mapread 0x19f7c9 thru 0x1a12d2 (0x1b0a bytes) 000166[0] 1722795213.264600 trunc from 0x464d5d to 0x689361 (0x224605 bytes) 000168[1] 1722795213.280320 mapread 0x0b9438 thru 0x0bad9b (0x1964 bytes) 000170[1] 1722795213.305932 read 0x4d4ce0 thru 0x4d53e3 (0x704 bytes) 000171[1] 1722795213.334097 trunc from 0x689362 to 0x7cd28c (0x143f2b bytes) 000175[0] 1722795213.357558 write 0x024d05 thru 0x0268b0 (0x1bac bytes) truncating to largest ever: 0x97f9d9 000176[0] 1722795213.364572 trunc from 0x7cd28d to 0x97f9d8 (0x1b274c bytes) 000180[1] 1722795213.378162 read 0x5c1d8a thru 0x5c3470 (0x16e7 bytes) 000181[0] 1722795213.384659 mapread 0x14bad8 thru 0x14c328 (0x851 bytes) 000182[0] 1722795213.409149 read 0x6352f4 thru 0x6368e9 (0x15f6 bytes) 000184[0] 1722795213.421531 mapread 0x1cbea8 thru 0x1cd888 (0x19e1 bytes) 000186[0] 1722795213.450581 read 0x0bbb5f thru 0x0bd5a3 (0x1a45 bytes) 000187[0] 1722795213.450950 mapread 0x3f1d3b thru 0x3f26a1 (0x967 bytes) 000192[1] 1722795213.457809 write 0x935261 thru 0x9365d1 (0x1371 bytes) 000193[0] 1722795213.496983 write 0x2304ce thru 0x230fa4 (0xad7 bytes) 000194[0] 1722795213.531786 trunc from 0x97f9d9 to 0x563725 (0x41c2b3 bytes) 000195[0] 1722795213.543772 write 0x4f884e thru 0x4fa114 (0x18c7 bytes) 000196[1] 1722795213.557387 read 0x2f15ca thru 0x2f1dbf (0x7f6 bytes) 000197[1] 1722795213.590385 write 0x4d7b50 thru 0x4d9604 (0x1ab5 bytes) 000198[1] 1722795213.605929 write 0x6f979d thru 0x6faad6 (0x133a bytes) 000199[1] 1722795213.610106 read 0x527e64 thru 0x529a5b (0x1bf8 bytes) 000203[1] 1722795213.623877 write 0x05c031 thru 0x05c046 (0x16 bytes) truncating to largest ever: 0x9905cc 000205[0] 1722795213.659452 trunc from 0x6faad7 to 0x9905cb (0x295af5 bytes) 000208[1] 1722795213.682701 read 0x71cee4 thru 0x71e287 (0x13a4 bytes) 000209[1] 1722795213.691072 trunc from 0x9905cc to 0x0aee5b (0x8e1770 bytes) 000210[1] 1722795213.699603 read 0x0405c7 thru 0x042219 (0x1c53 bytes) 000214[0] 1722795213.715663 read 0x054d3b thru 0x056685 (0x194b bytes) 000215[1] 1722795213.726231 trunc from 0x0aee5c to 0x148f40 (0x9a0e5 bytes) 000216[0] 1722795213.734879 trunc from 0x148f41 to 0x4eec97 (0x3a5d57 bytes) 000217[0] 1722795213.742134 read 0x160890 thru 0x161894 (0x1005 bytes) 000218[1] 1722795213.767993 write 0x6a77d2 thru 0x6a9057 (0x1886 bytes) 000219[0] 1722795213.784040 mapread 0x1390de thru 0x13a821 (0x1744 bytes) 000224[0] 1722795213.802158 mapwrite 0x6c163a thru 0x6c24f8 (0xebf bytes) 000227[0] 1722795213.837866 mapread 0x4c0d21 thru 0x4c15b4 (0x894 bytes) 000228[0] 1722795213.842963 mapwrite 0x0565ac thru 0x0565c4 (0x19 bytes) 000229[0] 1722795213.856088 mapread 0x2607b6 thru 0x261892 (0x10dd bytes) 000231[1] 1722795213.856585 trunc from 0x6c24f9 to 0x66399a (0x5eb5e bytes) 000235[0] 1722795213.872906 mapwrite 0x826d1a thru 0x826d64 (0x4b bytes) 000236[1] 1722795213.896875 mapread 0x44fea5 thru 0x450ed4 (0x1030 bytes) 000237[0] 1722795213.907490 mapwrite 0x0342ea thru 0x03512c (0xe43 bytes) 000240[0] 1722795213.929322 write 0x10749c thru 0x1093e3 (0x1f48 bytes) 000242[0] 1722795213.936809 trunc from 0x826d65 to 0x1735c7 (0x6b379d bytes) 000243[0] 1722795213.947036 mapwrite 0x2e54c6 thru 0x2e5a60 (0x59b bytes) 000244[0] 1722795213.964871 read 0x17537c thru 0x1756a0 (0x325 bytes) 000245[1] 1722795213.987005 write 0x7b8386 thru 0x7b99b9 (0x1634 bytes) 000246[0] 1722795213.993528 mapwrite 0x128993 thru 0x1292a1 (0x90f bytes) 000247[1] 1722795214.011087 mapwrite 0x386bfd thru 0x387448 (0x84c bytes) 000250[0] 1722795214.058243 trunc from 0x7b99ba to 0x7860f0 (0x338c9 bytes) 000254[0] 1722795214.077393 trunc from 0x7860f1 to 0x743f1f (0x421d1 bytes) 000256[0] 1722795214.082080 trunc from 0x743f20 to 0x59fa9a (0x1a4485 bytes) 000257[1] 1722795214.085378 mapread 0x3d1d46 thru 0x3d1fe6 (0x2a1 bytes) 000259[1] 1722795214.101195 write 0x4d8f36 thru 0x4da49c (0x1567 bytes) 000263[0] 1722795214.123650 read 0x3a3584 thru 0x3a378c (0x209 bytes) 000268[0] 1722795214.162027 mapread 0x480bac thru 0x48210d (0x1562 bytes) 000269[0] 1722795214.182817 write 0x84ebdb thru 0x84fa8e (0xeb4 bytes) 000271[0] 1722795214.192898 mapread 0x026887 thru 0x02808e (0x1808 bytes) 000272[1] 1722795214.199947 read 0x63f4f9 thru 0x6404e2 (0xfea bytes) 000275[0] 1722795214.212592 mapwrite 0x63edb0 thru 0x640a77 (0x1cc8 bytes) 000277[1] 1722795214.248906 read 0x30eed6 thru 0x3108ff (0x1a2a bytes) 000279[1] 1722795214.280491 write 0x9cbae5 thru 0x9cd3d8 (0x18f4 bytes) 000281[1] 1722795214.293091 write 0x7746bd thru 0x77668e (0x1fd2 bytes) 000282[1] 1722795214.312417 write 0x54ce17 thru 0x54d568 (0x752 bytes) 000285[1] 1722795214.319475 write 0x349fa2 thru 0x34a79d (0x7fc bytes) 000287[1] 1722795214.330382 read 0x76157d thru 0x761bd1 (0x655 bytes) 000288[0] 1722795214.330784 write 0x55a484 thru 0x55bb42 (0x16bf bytes) 000289[0] 1722795214.353290 trunc from 0x9cd3d9 to 0x3831ba (0x64a21e bytes) 000290[1] 1722795214.365687 read 0x17bde7 thru 0x17c3f2 (0x60c bytes) 000292[0] 1722795214.394178 write 0x1e5319 thru 0x1e5c9a (0x982 bytes) 000295[1] 1722795214.407401 trunc from 0x3831bb to 0x260a2b (0x12278f bytes) 000296[0] 1722795214.415828 trunc from 0x260a2c to 0x27cb7d (0x1c152 bytes) 000299[1] 1722795214.428407 trunc from 0x27cb7e to 0x80a4ca (0x58d94d bytes) 000302[1] 1722795214.439190 mapread 0x1528d9 thru 0x1537e4 (0xf0c bytes) 000303[0] 1722795214.461195 mapwrite 0x6bacfa thru 0x6bb0b7 (0x3be bytes) 000305[1] 1722795214.510255 read 0x16840d thru 0x168bee (0x7e2 bytes) 000307[0] 1722795214.541012 trunc from 0x80a4cb to 0x30a189 (0x500341 bytes) 000312[0] 1722795214.562297 write 0x433510 thru 0x434fa2 (0x1a93 bytes) 000313[0] 1722795214.566812 mapread 0x19ade9 thru 0x19c844 (0x1a5c bytes) 000314[1] 1722795214.574082 read 0x173568 thru 0x174041 (0xada bytes) 000315[0] 1722795214.596395 mapread 0x1ef153 thru 0x1f0fd4 (0x1e82 bytes) 000317[1] 1722795214.603121 trunc from 0x434fa3 to 0x6419d1 (0x20ca2f bytes) 000319[0] 1722795214.621538 trunc from 0x6419d2 to 0x789dac (0x1483db bytes) truncating to largest ever: 0x9fd0e2 000320[1] 1722795214.629266 trunc from 0x789dad to 0x9fd0e1 (0x273335 bytes) 000322[0] 1722795214.643602 mapwrite 0x5ea5c6 thru 0x5eb459 (0xe94 bytes) 000324[0] 1722795214.671744 mapread 0x27eae0 thru 0x2809d8 (0x1ef9 bytes) 000325[1] 1722795214.693125 read 0x652468 thru 0x654044 (0x1bdd bytes) 000331[1] 1722795214.731556 write 0x1892d3 thru 0x18b066 (0x1d94 bytes) 000332[1] 1722795214.749445 read 0x7adc79 thru 0x7ae616 (0x99e bytes) 000333[0] 1722795214.749818 write 0x7b0c38 thru 0x7b2bde (0x1fa7 bytes) 000335[1] 1722795214.763504 mapwrite 0x85caeb thru 0x85cca2 (0x1b8 bytes) 000340[1] 1722795214.813979 mapwrite 0x14ba2c thru 0x14d77b (0x1d50 bytes) 000341[1] 1722795214.831018 mapread 0x60d6d9 thru 0x60da66 (0x38e bytes) 000345[1] 1722795214.866684 read 0x296499 thru 0x2970f3 (0xc5b bytes) 000348[1] 1722795214.872194 mapwrite 0x7aa5d8 thru 0x7ac3ff (0x1e28 bytes) 000349[1] 1722795214.923075 trunc from 0x9fd0e2 to 0x362c3c (0x69a4a5 bytes) 000352[0] 1722795214.938667 trunc from 0x362c3d to 0x92ec3a (0x5cbffe bytes) 000354[0] 1722795214.953787 trunc from 0x92ec3b to 0x9df14b (0xb0511 bytes) 000356[1] 1722795214.961917 write 0x3ac39c thru 0x3acdb8 (0xa1d bytes) 000359[1] 1722795214.985501 read 0x35d742 thru 0x35ed5d (0x161c bytes) 000362[1] 1722795215.022377 mapread 0x73eff7 thru 0x73fb73 (0xb7d bytes) 000365[1] 1722795215.057971 write 0x5ce5fe thru 0x5ce7ca (0x1cd bytes) 000366[0] 1722795215.064171 read 0x137146 thru 0x137edc (0xd97 bytes) 000369[1] 1722795215.095026 write 0x596290 thru 0x5968fc (0x66d bytes) 000370[0] 1722795215.099621 read 0x13c7bb thru 0x13e7aa (0x1ff0 bytes) 000372[0] 1722795215.100111 mapwrite 0x1efbcc thru 0x1f0df0 (0x1225 bytes) 000373[0] 1722795215.155808 mapread 0x614eac thru 0x616614 (0x1769 bytes) 000374[0] 1722795215.185631 mapread 0x939a8d thru 0x93aceb (0x125f bytes) 000376[0] 1722795215.203465 trunc from 0x9df14c to 0x9ed039 (0xdeee bytes) 000379[0] 1722795215.220377 trunc from 0x9ed03a to 0x7b1205 (0x23be34 bytes) 000381[0] 1722795215.230006 write 0x115606 thru 0x115cea (0x6e5 bytes) 000387[1] 1722795215.240765 mapwrite 0x34d266 thru 0x34d78c (0x527 bytes) 000388[1] 1722795215.266531 read 0x32d951 thru 0x32dfd4 (0x684 bytes) 000390[0] 1722795215.289393 read 0x48a28d thru 0x48b87a (0x15ee bytes) 000394[0] 1722795215.302761 trunc from 0x7b1206 to 0x9c3bdc (0x2129d7 bytes) 000395[1] 1722795215.314702 mapwrite 0x8b6091 thru 0x8b7647 (0x15b7 bytes) 000397[1] 1722795215.336949 mapread 0x50934e thru 0x50b01a (0x1ccd bytes) 000401[1] 1722795215.367031 mapwrite 0x22dcff thru 0x22ddfc (0xfe bytes) 000402[0] 1722795215.396511 read 0x203e1e thru 0x205dab (0x1f8e bytes) 000403[1] 1722795215.429159 trunc from 0x9c3bdd to 0x086c96 (0x93cf46 bytes) 000404[1] 1722795215.452737 write 0x33ccd3 thru 0x33ccef (0x1d bytes) 000405[1] 1722795215.453541 write 0x873972 thru 0x8753e4 (0x1a73 bytes) 000408[1] 1722795215.458492 mapread 0x0688df thru 0x0694c6 (0xbe8 bytes) 000409[0] 1722795215.464324 mapread 0x731477 thru 0x7327bb (0x1345 bytes) 000410[1] 1722795215.475128 mapread 0x71b0b0 thru 0x71bd47 (0xc98 bytes) 000412[0] 1722795215.477300 mapread 0x3afe5f thru 0x3b1468 (0x160a bytes) 000414[1] 1722795215.486196 write 0x5616da thru 0x562d07 (0x162e bytes) 000415[0] 1722795215.493240 mapread 0x05a06f thru 0x05ada5 (0xd37 bytes) 000417[1] 1722795215.499824 read 0x480bbc thru 0x4820fc (0x1541 bytes) 000419[1] 1722795215.514952 write 0x0edee6 thru 0x0ef1aa (0x12c5 bytes) 000421[0] 1722795215.529787 read 0x26f334 thru 0x26fd8e (0xa5b bytes) 000422[1] 1722795215.556908 write 0x33d939 thru 0x33e0a3 (0x76b bytes) 000423[0] 1722795215.563327 trunc from 0x8753e5 to 0x64bfeb (0x2293f9 bytes) 000425[1] 1722795215.579118 trunc from 0x64bfec to 0x8fcca1 (0x2b0cb6 bytes) 000426[1] 1722795215.585535 write 0x414d1a thru 0x416633 (0x191a bytes) 000429[0] 1722795215.611485 mapread 0x7a64c3 thru 0x7a6ad9 (0x617 bytes) 000430[0] 1722795215.636996 mapread 0x727ec0 thru 0x7296b5 (0x17f6 bytes) 000431[1] 1722795215.639097 trunc from 0x8fcca2 to 0x162227 (0x79aa7a bytes) 000433[1] 1722795215.653043 trunc from 0x162228 to 0x808716 (0x6a64ef bytes) 000434[1] 1722795215.661740 mapread 0x2947bd thru 0x294871 (0xb5 bytes) 000436[1] 1722795215.669683 read 0x7d862c thru 0x7d9980 (0x1355 bytes) 000438[1] 1722795215.706167 mapwrite 0x3f3d9d thru 0x3f475d (0x9c1 bytes) 000440[0] 1722795215.722362 trunc from 0x808717 to 0x91b9f8 (0x1132e2 bytes) 000441[0] 1722795215.738101 write 0x9302fa thru 0x930659 (0x360 bytes) 000442[1] 1722795215.739570 mapwrite 0x6b10aa thru 0x6b160f (0x566 bytes) 000448[1] 1722795215.772318 trunc from 0x93065a to 0x442287 (0x4ee3d2 bytes) 000452[1] 1722795215.784347 mapwrite 0x7de4ce thru 0x7dec82 (0x7b5 bytes) 000454[1] 1722795215.804754 write 0x5bf519 thru 0x5bf81e (0x306 bytes) 000455[1] 1722795215.806291 read 0x1ca774 thru 0x1cb1f8 (0xa85 bytes) 000456[0] 1722795215.831932 mapread 0x6c5e23 thru 0x6c6a0a (0xbe8 bytes) 000458[0] 1722795215.861240 mapwrite 0x0e8a96 thru 0x0e9ce1 (0x124c bytes) 000459[1] 1722795215.882874 trunc from 0x7dec83 to 0x545f2c (0x298d56 bytes) 000461[1] 1722795215.895947 trunc from 0x545f2d to 0x0486d1 (0x4fd85b bytes) 000462[1] 1722795215.902229 write 0x4ce14c thru 0x4cf41c (0x12d1 bytes) 000464[1] 1722795215.908172 write 0x8d4303 thru 0x8d5674 (0x1372 bytes) 000472[0] 1722795215.922249 read 0x4b70a3 thru 0x4b8340 (0x129e bytes) 000475[0] 1722795215.936074 read 0x19073d thru 0x1911e4 (0xaa8 bytes) 000477[1] 1722795215.965644 mapread 0x5b455c thru 0x5b4d9c (0x841 bytes) 000478[0] 1722795215.967090 read 0x5c0fc5 thru 0x5c1b56 (0xb92 bytes) 000481[0] 1722795215.969884 mapread 0x0f5906 thru 0x0f634c (0xa47 bytes) 000482[0] 1722795215.970183 mapwrite 0x218771 thru 0x219080 (0x910 bytes) 000483[1] 1722795216.005429 mapwrite 0x664666 thru 0x665a56 (0x13f1 bytes) 000484[0] 1722795216.033096 trunc from 0x8d5675 to 0x583542 (0x352132 bytes) 000486[1] 1722795216.045381 mapread 0x0c5b1f thru 0x0c6a80 (0xf62 bytes) 000489[0] 1722795216.056438 mapwrite 0x11c249 thru 0x11cc69 (0xa21 bytes) 000491[1] 1722795216.077779 trunc from 0x583543 to 0x79aa2d (0x2174eb bytes) 000494[1] 1722795216.089719 read 0x387928 thru 0x387cdb (0x3b4 bytes) 000498[1] 1722795216.113519 write 0x768111 thru 0x769860 (0x1750 bytes) 000500[1] 1722795216.135861 write 0x6c10b5 thru 0x6c1998 (0x8e4 bytes) All operations completed A-OK! PASS 16b (7s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 14:13:37 (1722795217) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 14:13:38 (1722795218) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.129395 s, 64.8 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.119627 s, 70.1 MB/s ldlm.namespaces.MGC192.168.203.110@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b5870000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b5904800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b5870000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b5904800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b5870000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b5904800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.72454 s, 38.5 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.586028 s, 179 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.54738 s, 192 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.0863 s, 96.5 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.82898 s, 57.3 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.86092 s, 122 MB/s PASS 16d (10s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 14:13:48 (1722795228) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0613194 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292659 s, 35.8 MB/s PASS 16e (2s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 14:13:50 (1722795230) TIMEOUT PASS 16f (21s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 14:14:11 (1722795251) TIMEOUT PASS 16g (22s) == sanityn test 16h: mmap read after truncate file ======= 14:14:33 (1722795273) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16h (1s) == sanityn test 16i: read after truncate file ============ 14:14:34 (1722795274) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (2s) == sanityn test 16j: race dio with buffered i/o ========== 14:14:36 (1722795276) 1+0 records in 1+0 records out 26214400 bytes (26 MB) copied, 0.375134 s, 69.9 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.218658 s, 120 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.502745 s, 52.1 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.153815 s, 170 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.428296 s, 61.2 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.135021 s, 194 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.522864 s, 50.1 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.371059 s, 70.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.381775 s, 68.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.461549 s, 56.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.525798 s, 49.9 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.214801 s, 122 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.268643 s, 97.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.535423 s, 49.0 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.565486 s, 46.4 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.339992 s, 77.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.366243 s, 71.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.584943 s, 44.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.623442 s, 42.0 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (7s) == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 14:14:43 (1722795283) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!fsx: : test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported truncating to largest ever: 0x1a7062c fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported truncating to largest ever: 0x1a7062c fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab All operations completed A-OK! + fsx 1 done: rc=0 Sun Aug 4 14:15:07 EDT 2024 All operations completed A-OK! All operations completed A-OK! + fsx 2 done: rc=0 Sun Aug 4 14:15:09 EDT 2024 All operations completed A-OK! + fsx 3 done: rc=0 Sun Aug 4 14:15:09 EDT 2024 All operations completed A-OK! + fsx 4 done: rc=0 Sun Aug 4 14:15:09 EDT 2024 + fsx 5 done: rc=0 Sun Aug 4 14:15:09 EDT 2024 PASS 16k (28s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 14:15:11 (1722795311) fail_loc=0x8000030a PASS 17 (4s) == sanityn test 18: mmap sanity check =========================================================================================== 14:15:15 (1722795315) mmap test1: basic mmap operation (PASS, 0.019491s) mmap test2: MAP_PRIVATE not write back (PASS, 0.013692s) mmap test3: concurrent mmap ops on two nodes (PASS, 2.3392s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 4.0767s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.025363s) mmap test6: check mmap write/read content on two nodes (PASS, 0.032616s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: multi-thread mmap access (PASS, 0.34305s) mmap test11: multi-thread shared mmap (PASS, 0.32263s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0176s) PASS 18 (11s) == sanityn test 19: test concurrent uncached read races ========================================================================= 14:15:26 (1722795326) oleg310-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg310-client: oleg310-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (0s) == sanityn test 20: test extra readahead page left in cache ============================================================== 14:15:27 (1722795327) PASS 20 (2s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 14:15:28 (1722795328) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (1s) == sanityn test 23: others should see updated atime while another read============================================================== 14:15:29 (1722795329) now is 1722795330 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 new atime is 1722795391 PASS 23 (63s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 14:16:32 (1722795392) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 400 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 114.5K 401 114.1K 1% /mnt/lustre[OST:1] filesystem_summary: 229.3K 387 229.0K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 400 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 401 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.3K 387 229.0K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 8.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 31.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 39.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118000 400 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117265 401 116864 1% /mnt/lustre2[OST:1] filesystem_summary: 234851 387 234464 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 400 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 401 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.3K 387 229.0K 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre[MDT:0] filesystem_summary: 502378 387 501991 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118000 400 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117265 401 116864 1% /mnt/lustre2[OST:1] filesystem_summary: 234851 387 234464 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre2 PASS 24a (1s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 14:16:33 (1722795393) PASS 24b (2s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 14:16:35 (1722795395) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (1s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 14:16:36 (1722795396) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 14:16:37 (1722795397) utime: good mknod times 1722795396 <= 1722795397 <= 1722795397 for /mnt/lustre/f26a utime: good mknod times 1722795396 <= 1722795397 <= 1722795397 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (2s) == sanityn test 26b: sync mtime between ost and mds ====== 14:16:39 (1722795399) PASS 26b (4s) == sanityn test 26c: set-in-past on open file is not lost on close ========================================================== 14:16:43 (1722795403) multiop /mnt/lustre/f26c.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 File: '/mnt/lustre/f26c.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2024-08-04 14:16:43.000000000 -0400 Birth: - File: '/mnt/lustre2/f26c.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2024-08-04 14:16:43.000000000 -0400 Birth: - PASS 26c (2s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 14:16:46 (1722795406) 0 PASS 30 (3s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 14:16:50 (1722795410) fail_loc=0x314 PASS 31a (2s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 14:16:52 (1722795412) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg310-server mds-ost sync done. sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.203.110@tcp:/lustre 7542784 39936 7496704 1% /mnt/lustre2 PASS 31b (17s) == sanityn test 31r: open-rename(replace) race =========== 14:17:09 (1722795429) -rw-r--r-- 1 root root 0 Aug 4 14:17 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 mv: cannot stat '/mnt/lustre2/f31r.sanityn.source': Input/output error PASS 31r (4s) == sanityn test 31s: open should not revalidate invalid dentry ========================================================== 14:17:13 (1722795433) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 1211: 12617 Terminated tail -f $tfile PASS 31s (2s) == sanityn test 31t: getattr should not revalidate invalid dentry ========================================================== 14:17:15 (1722795435) PASS 31t (2s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 14:17:18 (1722795438) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 14:17:18 (1722795438) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: dependent transactions should trigger COS ========================================================== 14:17:19 (1722795439) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 14:17:20 (1722795440) SKIP: sanityn test_33e needs >= 2 MDTs SKIP 33e (0s) == sanityn test 34: no lock timeout under IO ============= 14:17:20 (1722795440) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg310-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b5870000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b5904800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800b5870000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8800b5904800.ost_server_uuid in IDLE state after 0 sec oleg310-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b5870000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b5904800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800b5870000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800b5904800.ost_server_uuid in FULL state after 0 sec oleg310-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b5870000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b5904800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800b5870000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8800b5904800.ost_server_uuid in IDLE state after 0 sec oleg310-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b5870000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b5904800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800b5870000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800b5904800.ost_server_uuid in FULL state after 0 sec oleg310-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b5870000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b5904800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800b5870000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8800b5904800.ost_server_uuid in IDLE state after 0 sec oleg310-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b5870000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b5904800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800b5870000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800b5904800.ost_server_uuid in FULL state after 0 sec PASS 34 (86s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 14:18:46 (1722795526) fail_loc=0x80000317 Race attempt 0 Wait for 20073 20213 for 60 sec... total: 4000 open/close in 7.32 seconds: 546.27 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.203.110@tcp:/lustre 7.2G 40M 7.2G 1% /mnt/lustre 192.168.203.110@tcp:/lustre 7.2G 40M 7.2G 1% /mnt/lustre2 PASS 35 (64s) == sanityn test 36: handle ESTALE/open-unlink correctly == 14:19:50 (1722795590) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg310-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.28654 s, 183 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7497728) after_dd(7446528) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.238986 s, 219 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7497728) after_dd(7444480) after(7495680) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.270159 s, 194 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7495680) after_dd(7442432) after(7493632) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.304595 s, 172 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7493632) after_dd(7446528) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.253467 s, 207 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7497728) after_dd(7446528) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.263875 s, 199 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7497728) after_dd(7442432) after(7493632) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.264452 s, 198 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7493632) after_dd(7444480) after(7495680) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.298592 s, 176 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7495680) after_dd(7444480) after(7495680) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.448616 s, 117 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7495680) after_dd(7444480) after(7495680) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.258471 s, 203 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7495680) after_dd(7444480) after(7495680) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.264791 s, 198 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7495680) after_dd(7446528) after(7497728) PASS 36 (226s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 14:23:36 (1722795816) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 - create 9509 (time 1722795827.00 total 10.00 last 950.88) total: 10000 create in 10.76 seconds: 929.05 ops/second PASS 37 (13s) == sanityn test 39a: file mtime does not change after rename ========================================================== 14:23:49 (1722795829) repeat after cancel_lru_locks PASS 39a (2s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 14:23:51 (1722795831) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 14:23:54 (1722795834) repeat after cancel_lru_locks PASS 39c (2s) == sanityn test 39d: sync write should update mtime ====== 14:23:56 (1722795836) fail_loc=0x411 fail_loc=0 PASS 39d (2s) == sanityn test 40a: pdirops: create vs others ======================================================================== 14:23:58 (1722795838) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (8s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 14:24:06 (1722795846) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (7s) == sanityn test 40c: pdirops: link and others ======================================================================== 14:24:13 (1722795853) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (8s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 14:24:21 (1722795861) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (7s) == sanityn test 40e: pdirops: rename and others ======================================================================== 14:24:28 (1722795868) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (7s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 14:24:35 (1722795875) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (4s) == sanityn test 41b: pdirops: create vs create ======================================================================== 14:24:39 (1722795879) open: File exists Conflict PASS 41b (4s) == sanityn test 41c: pdirops: create vs link ======================================================================== 14:24:43 (1722795883) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (5s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 14:24:48 (1722795888) Conflict PASS 41d (5s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 14:24:53 (1722795893) Conflict PASS 41e (4s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 14:24:57 (1722795897) Conflict PASS 41f (5s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 14:25:02 (1722795902) Conflict PASS 41g (5s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 14:25:07 (1722795907) Conflict PASS 41h (6s) == sanityn test 41i: reint_open: create vs create ======== 14:25:13 (1722795913) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (676s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 14:36:29 (1722796589) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (4s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 14:36:33 (1722796593) open: File exists Conflict PASS 42b (6s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 14:36:39 (1722796599) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (5s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 14:36:44 (1722796604) Conflict PASS 42d (5s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 14:36:49 (1722796609) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (5s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 14:36:54 (1722796614) Conflict PASS 42f (5s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 14:36:59 (1722796619) Conflict PASS 42g (4s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 14:37:03 (1722796623) Conflict PASS 42h (5s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 14:37:08 (1722796628) PASS 43a (13s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 14:37:22 (1722796642) Conflict PASS 43b (4s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 14:37:26 (1722796646) Conflict PASS 43c (4s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 14:37:30 (1722796650) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (6s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 14:37:36 (1722796656) Conflict PASS 43e (5s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 14:37:41 (1722796661) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (5s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 14:37:46 (1722796666) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (5s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 14:37:51 (1722796671) Conflict PASS 43h (4s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 14:37:55 (1722796675) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 14:37:56 (1722796676) mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (36s) == sanityn test 43k: unlink vs create ==================== 14:38:32 (1722796712) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (411s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 14:45:23 (1722797123) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Sun Aug 4 14:45:26 EDT 2024 PASS 44a (5s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 14:45:28 (1722797128) open: File exists Conflict PASS 44b (5s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 14:45:33 (1722797133) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (5s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 14:45:38 (1722797138) Conflict PASS 44d (4s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 14:45:42 (1722797142) Conflict PASS 44e (5s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 14:45:47 (1722797147) Conflict PASS 44f (4s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 14:45:51 (1722797151) Conflict PASS 44g (5s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 14:45:56 (1722797156) Conflict PASS 44h (4s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 14:46:00 (1722797160) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 14:46:01 (1722797161) PASS 45a (35s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 14:46:36 (1722797196) Conflict PASS 45b (5s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 14:46:41 (1722797201) Conflict PASS 45c (5s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 14:46:46 (1722797206) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (5s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 14:46:51 (1722797211) Conflict PASS 45e (5s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 14:46:56 (1722797216) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (6s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 14:47:02 (1722797222) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (4s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 14:47:06 (1722797226) Conflict PASS 45h (4s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 14:47:11 (1722797231) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (0s) == sanityn test 45j: read vs rename ====================== 14:47:12 (1722797232) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (385s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 14:53:37 (1722797617) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (4s) == sanityn test 46b: pdirops: link vs create ======================================================================== 14:53:41 (1722797621) open: File exists Conflict PASS 46b (4s) == sanityn test 46c: pdirops: link vs link ======================================================================== 14:53:46 (1722797626) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (6s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 14:53:51 (1722797631) Conflict PASS 46d (5s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 14:53:56 (1722797636) Conflict PASS 46e (6s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 14:54:02 (1722797642) Conflict PASS 46f (5s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 14:54:07 (1722797647) Conflict PASS 46g (6s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 14:54:13 (1722797653) Conflict PASS 46h (5s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 14:54:18 (1722797658) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 14:54:19 (1722797659) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 14:54:20 (1722797660) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (0s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 14:54:20 (1722797660) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 14:54:21 (1722797661) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 14:54:22 (1722797662) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 14:54:23 (1722797663) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (0s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 14:54:23 (1722797663) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 14:54:24 (1722797664) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.00374934 s, 2.7 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 14:54:31 (1722797671) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00719564 s, 22.1 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3514: kill: (31473) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 14:54:35 (1722797675) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00456076 s, 225 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220985 s, 47.5 MB/s 3145728 PASS 51b (14s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 14:54:49 (1722797689) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00339491 s, 302 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 14:54:55 (1722797695) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142726 s, 73.5 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (3s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 14:54:58 (1722797698) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2454 0x996 0x240000400 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2454 0x996 0x240000400 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 14:55:02 (1722797702) llite.lustre-ffff8800b5870000.xattr_cache=1 llite.lustre-ffff8800b5904800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505012 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-08-04 14:55:02.000000000 -0400 Modify: 2024-08-04 14:55:02.000000000 -0400 Change: 2024-08-04 14:55:02.000000000 -0400 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505015 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-08-04 14:55:07.000000000 -0400 Modify: 2024-08-04 14:55:08.000000000 -0400 Change: 2024-08-04 14:55:08.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505018 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-08-04 14:55:13.000000000 -0400 Modify: 2024-08-04 14:55:13.000000000 -0400 Change: 2024-08-04 14:55:13.000000000 -0400 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505017 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-08-04 14:55:13.000000000 -0400 Modify: 2024-08-04 14:55:13.000000000 -0400 Change: 2024-08-04 14:55:13.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505020 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-08-04 14:55:18.000000000 -0400 Modify: 2024-08-04 14:55:18.000000000 -0400 Change: 2024-08-04 14:55:18.000000000 -0400 Birth: - File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505021 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-08-04 14:55:18.000000000 -0400 Modify: 2024-08-04 14:55:18.000000000 -0400 Change: 2024-08-04 14:55:18.000000000 -0400 Birth: - PASS 54 (23s) == sanityn test 55a: rename vs unlink target dir ========= 14:55:25 (1722797725) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 14:55:32 (1722797732) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 14:55:39 (1722797739) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 14:55:51 (1722797751) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) == sanityn test 56a: test llverdev with single large stripe ========================================================== 14:56:00 (1722797760) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1722797760 chunksize: 4096 size: 16777216000000 write offset: 16384000000kB 2.65387 MB/s read offset: 16384000000kB inf MB/s llverdev: data verified successfully PASS 56a (24s) == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 14:56:24 (1722797784) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1722797785 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 4.69516 MB/s read offset: 32768000000kB -nan MB/s llverdev: data verified successfully PASS 56b (29s) == sanityn test 60: Verify data_version behaviour ======== 14:56:54 (1722797814) PASS 60 (1s) == sanityn test 70a: cd directory && rm directory ======== 14:56:55 (1722797815) PASS 70a (1s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 14:56:56 (1722797816) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 14:56:58 (1722797818) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00228464 s, 28.7 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 14:56:59 (1722797819) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00355532 s, 18.4 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (0s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 14:57:00 (1722797820) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (0s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 14:57:00 (1722797820) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00307201 s, 21.3 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 14:57:01 (1722797821) llite.lustre-ffff8800b5870000.xattr_cache=1 llite.lustre-ffff8800b5904800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 14:57:03 (1722797823) llite.lustre-ffff8800b5870000.xattr_cache=1 llite.lustre-ffff8800b5904800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (2s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 14:57:05 (1722797825) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00253297 s, 404 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0021807 s, 470 kB/s 11416: get lock2 11417: get lock1 11417: done 11417: get lock2 11416: done 11416: get lock1 fcntl cmd 7 failed: Resource deadlock avoided 11417: cannot get lock2: Resource deadlock avoided 11417: exit rc=0 11416: done 11416: exit rc=0 PASS 74 (5s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 14:57:10 (1722797830) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0148552 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00763357 s, 137 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4103: 12054 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 14:57:17 (1722797837) opening files: ........ closing files: ........ PASS 76 (14s) == sanityn test 77a: check FIFO NRS policy =============== 14:57:31 (1722797851) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0980818 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00717299 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112412 s, 93.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206223 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158651 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0230659 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153471 s, 68.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180679 s, 58.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163457 s, 64.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197432 s, 53.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0275909 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371819 s, 28.2 MB/s , 0.0340961 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0384641 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0424842 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315423 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392321 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00835165 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0065992 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00937879 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00527173 s, 199 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00904671 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00931279 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00772205 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00695051 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00847778 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00798232 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00851228 s, 123 MB/s , 0.00860582 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00945292 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00974961 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102061 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00914008 s, 115 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg310-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (3s) == sanityn test 77b: check CRR-N NRS policy ============== 14:57:34 (1722797854) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0734026 s, 229 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00527903 s, 199 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108106 s, 97.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158806 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182661 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342721 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354587 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365356 s, 28.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0431697 s, 24.3 MB/s 1048576 bytes (1.0 MB) copied, 0.037997 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0323511 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346258 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0382359 s, 27.4 MB/s , 0.0377892 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342528 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.038008 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0519325 s, 20.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b5904800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00792781 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00696386 s, 151 MB/s 1048576 bytes (1.0 MB) copied, 0.00630495 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00171321 s, 612 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00691327 s, 152 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00832005 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112073 s, 93.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0085911 s, 122 MB/s , 0.00877832 s, 119 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00620319 s, 169 MB/s 1048576 bytes (1.0 MB) copied, 0.00654824 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00917567 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00977235 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00873266 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109984 s, 95.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109353 s, 95.9 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.141845 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202635 s, 51.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219683 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198906 s, 52.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0504933 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0568506 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0541444 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0530366 s, 19.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0613571 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0593347 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0584325 s, 17.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0433227 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0544677 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495846 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0590802 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0507617 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0592951 s, 17.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182878 s, 57.3 MB/s , 0.00816685 s, 128 MB/s 1048576 bytes (1.0 MB) copied, 0.00819948 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104279 s, 101 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222479 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100478 s, 104 MB/s 1048576 bytes (1.0 MB) copied, 0.0121684 s, 86.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214688 s, 48.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231094 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100544 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121022 s, 86.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112172 s, 93.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128215 s, 81.8 MB/s , 0.0115191 s, 91.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011418 s, 91.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128693 s, 81.5 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) == sanityn test 77c: check ORR NRS policy ================ 14:57:37 (1722797857) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0826416 s, 203 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00785188 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00545517 s, 192 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014602 s, 71.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236736 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102667 s, 102 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328399 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274229 s, 38.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0188153 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0347828 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0264982 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232659 s, 45.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255008 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0351354 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378603 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388001 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.035136 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00840948 s, 125 MB/s 1048576 bytes (1.0 MB) copied, 0.0106614 s, 98.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00766559 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00816868 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011955 s, 87.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00996426 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00969365 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111598 s, 94.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109055 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113956 s, 92.0 MB/s , 0.0118375 s, 88.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120485 s, 87.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105481 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113233 s, 92.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113022 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126739 s, 82.7 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0967866 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154006 s, 68.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172144 s, 60.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270643 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034747 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0336527 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0366735 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0385068 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0409908 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0491525 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0482229 s, 21.7 MB/s , 0.0461048 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0455063 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0487927 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0487582 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0441841 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0576812 s, 18.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00932353 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00843693 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00206993 s, 507 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00710127 s, 148 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00489585 s, 214 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00897787 s, 117 MB/s 1+0 records in 1+0 records out , 0.0261719 s, 40.1 MB/s 1048576 bytes (1.0 MB) copied, 0.00184239 s, 569 MB/s 1048576 bytes (1.0 MB) copied, 0.0154646 s, 67.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260476 s, 40.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0101961 s, 103 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0112651 s, 93.1 MB/s , 0.0217931 s, 48.1 MB/s , 0.0271689 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132895 s, 78.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00171577 s, 611 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0952173 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00723869 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00822388 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114259 s, 91.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00595371 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123283 s, 85.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110907 s, 94.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00501893 s, 209 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00537207 s, 195 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243536 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0321275 s, 32.6 MB/s , 0.0252798 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283423 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338366 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017306 s, 60.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354969 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0264364 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103797 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00894353 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00858319 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00894553 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158487 s, 66.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112683 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170135 s, 61.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114908 s, 91.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013316 s, 78.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127825 s, 82.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148266 s, 70.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012273 s, 85.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013224 s, 79.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0138388 s, 75.8 MB/s , 0.0134003 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129015 s, 81.3 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (4s) == sanityn test 77d: check TRR nrs policy ================ 14:57:41 (1722797861) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.101055 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00735891 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012089 s, 86.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0280693 s, 37.4 MB/s , 0.0286724 s, 36.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0360292 s, 29.1 MB/s , 0.0355987 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0349198 s, 30.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0342872 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357887 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378601 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0369479 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0440479 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0438315 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0419083 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0407643 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451216 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00830591 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00924339 s, 113 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0185201 s, 56.6 MB/s , 0.0175799 s, 59.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113434 s, 92.4 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169147 s, 62.0 MB/s , 0.0115406 s, 90.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105593 s, 99.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207671 s, 50.5 MB/s , 0.0118062 s, 88.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120687 s, 86.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124502 s, 84.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130264 s, 80.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013152 s, 79.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132294 s, 79.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154067 s, 68.1 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.083428 s, 201 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00854646 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00903124 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122719 s, 85.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00944698 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242098 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254964 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0339589 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0358396 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261662 s, 40.1 MB/s , 0.0313474 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400825 s, 26.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0694179 s, 15.1 MB/s , 0.0734702 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0743296 s, 14.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0748516 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0694285 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109832 s, 95.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011337 s, 92.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119435 s, 87.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122136 s, 85.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0220998 s, 47.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0204821 s, 51.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195425 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191254 s, 54.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0144796 s, 72.4 MB/s , 0.0146399 s, 71.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158984 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158508 s, 66.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156613 s, 67.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016332 s, 64.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184464 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0204806 s, 51.2 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0797122 s, 210 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00916994 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00993424 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162632 s, 64.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149971 s, 69.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162745 s, 64.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411963 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389221 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0414254 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295081 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0373258 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0373711 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371988 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.071136 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0594365 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0621674 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0788992 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00940694 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119647 s, 87.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00506523 s, 207 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116831 s, 89.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00745766 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015864 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00785196 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00793023 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130966 s, 80.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0093955 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0108589 s, 96.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0110808 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128376 s, 81.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136092 s, 77.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0137608 s, 76.2 MB/s , 0.0131622 s, 79.7 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg310-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (4s) == sanityn test 77e: check TBF NID nrs policy ============ 14:57:45 (1722797865) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.203.10@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.08575 s, 196 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00751544 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00822863 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276999 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233845 s, 44.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280905 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0302286 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290052 s, 36.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.035008 s, 30.0 MB/s , 0.0309479 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0384853 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353358 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0417527 s, 25.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0385448 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0367358 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.035956 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389902 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00775874 s, 135 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00682821 s, 154 MB/s , 0.00690937 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00731464 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00997696 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out , 0.00754452 s, 139 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00877846 s, 119 MB/s , 0.0120671 s, 86.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107772 s, 97.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127135 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0068324 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00978933 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102074 s, 103 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103509 s, 101 MB/s 1048576 bytes (1.0 MB) copied, 0.0103525 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00988992 s, 106 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg310-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0720494 s, 233 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00755547 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125946 s, 83.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120568 s, 87.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346161 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345511 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324138 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400437 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0452006 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0373327 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457178 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378375 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03609 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0403666 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0407493 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0450266 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0465819 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00729231 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00864861 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00674032 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00656655 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00739656 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00936626 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00833941 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00800189 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00844004 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00860916 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00868366 s, 121 MB/s 1048576 bytes (1.0 MB) copied, 0.00871533 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108613 s, 96.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100448 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.010252 s, 102 MB/s 1048576 bytes (1.0 MB) copied, 0.010288 s, 102 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0780363 s, 215 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00637807 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948332 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116816 s, 89.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01589 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019725 s, 53.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00708018 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123902 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00974184 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00996808 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0485396 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0460995 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0438624 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0474783 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0549884 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0415141 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449549 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00812112 s, 129 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00763 s, 137 MB/s , 0.00939141 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0069673 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0084959 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0084185 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0089315 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00967374 s, 108 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00858642 s, 122 MB/s , 0.00897114 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00903659 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102007 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0102282 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110822 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0088309 s, 119 MB/s 1048576 bytes (1.0 MB) copied, 0.0109759 s, 95.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.119582 s, 140 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00951342 s, 110 MB/s , 0.0094984 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00673604 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00944522 s, 111 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0551533 s, 19.0 MB/s , 0.043097 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0535194 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0472647 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0593303 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.057933 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0647446 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0657463 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.065191 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0571712 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0573238 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0644721 s, 16.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.015219 s, 68.9 MB/s , 0.00864632 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139756 s, 75.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132292 s, 79.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106355 s, 98.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0120496 s, 87.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0119416 s, 87.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132232 s, 79.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013677 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0119638 s, 87.6 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0112282 s, 93.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0127094 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125027 s, 83.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133417 s, 78.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133838 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128678 s, 81.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg310-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (7s) == sanityn test 77f: check TBF JobID nrs policy ========== 14:57:52 (1722797872) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg310-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.10211 s, 164 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00801748 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00934052 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984586 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00834193 s, 126 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247145 s, 42.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0309586 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0329846 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251008 s, 41.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0329047 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269573 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371233 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328045 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0441274 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0374875 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.043504 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0498372 s, 21.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123712 s, 84.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00285584 s, 367 MB/s , 0.00607563 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00884844 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0118119 s, 88.8 MB/s , 0.0153843 s, 68.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145256 s, 72.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00673127 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194328 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00191007 s, 549 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00222458 s, 471 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186225 s, 56.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00212383 s, 494 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00229875 s, 456 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0020148 s, 520 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00232814 s, 450 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg310-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.106321 s, 158 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107453 s, 97.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00986781 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018224 s, 57.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258613 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288831 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308762 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295135 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248695 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343814 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0441914 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0408417 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0429709 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0422455 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0478359 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0425356 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.051601 s, 20.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101618 s, 103 MB/s , 0.00850809 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00868618 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00908049 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102373 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120019 s, 87.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195271 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130271 s, 80.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135339 s, 77.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147304 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.029607 s, 35.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158767 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165784 s, 63.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176331 s, 59.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179816 s, 58.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183914 s, 57.0 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg310-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0929242 s, 181 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0091329 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148342 s, 70.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184118 s, 57.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315887 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342771 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354307 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316648 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343725 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03569 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0458229 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0414709 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0401844 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0454075 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446071 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0514981 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.054633 s, 19.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101031 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00633363 s, 166 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0123619 s, 84.8 MB/s , 0.00717934 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00431785 s, 243 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848292 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100012 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105221 s, 99.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0116861 s, 89.7 MB/s , 0.0115509 s, 90.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117292 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109357 s, 95.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00535567 s, 196 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126124 s, 83.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121912 s, 86.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013852 s, 75.7 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg310-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.114462 s, 147 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00832469 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00832468 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0286328 s, 36.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0239558 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183514 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297424 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343919 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279622 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365243 s, 28.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308378 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389552 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0408188 s, 25.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0391163 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361499 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0366597 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0433623 s, 24.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00929048 s, 113 MB/s 1048576 bytes (1.0 MB) copied, 0.00913231 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00982482 s, 107 MB/s 1048576 bytes (1.0 MB) copied, 0.0101894 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109811 s, 95.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208771 s, 50.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218509 s, 48.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208646 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134811 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145448 s, 72.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014562 s, 72.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156668 s, 66.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0160604 s, 65.3 MB/s , 0.0155181 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162163 s, 64.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140345 s, 74.7 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (6s) == sanityn test 77g: Change TBF type directly ============ 14:57:58 (1722797878) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (3s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 14:58:01 (1722797881) oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg310-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg310-client: oleg310-server: ssh exited with exit code 2 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 oleg310-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Unknown error 524 pdsh@oleg310-client: oleg310-server: ssh exited with exit code 12 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 oleg310-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Unknown error 524 pdsh@oleg310-client: oleg310-server: ssh exited with exit code 12 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 oleg310-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Unknown error 524 pdsh@oleg310-client: oleg310-server: ssh exited with exit code 12 PASS 77h (3s) == sanityn test 77i: Change rank of TBF rule ============= 14:58:04 (1722797884) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (6s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 14:58:10 (1722797890) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0694831 s, 241 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00635384 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110894 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228423 s, 45.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241986 s, 43.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0243351 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021296 s, 49.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0284438 s, 36.9 MB/s , 0.0208284 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205365 s, 51.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232886 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0303662 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288094 s, 36.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288839 s, 36.3 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0257242 s, 40.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0321568 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269871 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0068476 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00686313 s, 153 MB/s , 0.00894881 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00682324 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00831908 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00938422 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00931831 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00926978 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100001 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106578 s, 98.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110544 s, 94.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119878 s, 87.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111782 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111882 s, 93.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0127067 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121119 s, 86.6 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94513 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4308 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (50s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 14:59:00 (1722797940) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg310-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg310-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg310-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg310-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg310-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.21756 s, 13.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216096 s, 48.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228375 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00666788 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304982 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127729 s, 82.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345825 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260787 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266438 s, 39.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.173645 s, 6.0 MB/s , 0.1737 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.167782 s, 6.2 MB/s , 0.187743 s, 5.6 MB/s , 0.182934 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178882 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.176859 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.183911 s, 5.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0085973 s, 122 MB/s 1048576 bytes (1.0 MB) copied, 0.00644968 s, 163 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0065081 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00649512 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00776984 s, 135 MB/s 1048576 bytes (1.0 MB) copied, 0.0097905 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918839 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00806095 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0109093 s, 96.1 MB/s 1048576 bytes (1.0 MB) copied, 0.00895021 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00882135 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0083143 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0095208 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00910347 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00938367 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00993728 s, 106 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg310-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 20.0401 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9368 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg310-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg310-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg310-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg310-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg310-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.23055 s, 13.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211188 s, 49.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255112 s, 41.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0267511 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239257 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279609 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00787016 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115933 s, 90.4 MB/s , 0.00922566 s, 114 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.157967 s, 6.6 MB/s , 0.157894 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.18544 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.189917 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178323 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.161469 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.199751 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188081 s, 5.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165442 s, 63.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153689 s, 68.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163883 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174869 s, 60.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0142833 s, 73.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187346 s, 56.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0179567 s, 58.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161125 s, 65.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155154 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155756 s, 67.3 MB/s 1+0 records in 1+0 records out , 0.0182744 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016769 s, 62.5 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0186348 s, 56.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0163895 s, 64.0 MB/s , 0.0172714 s, 60.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184692 s, 56.8 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8414 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9392 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (131s) == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 15:01:11 (1722798071) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg310-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg310-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg310-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg310-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg310-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.23139 s, 13.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243414 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0367173 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0494968 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0582661 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0597442 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0537931 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0718227 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0524091 s, 20.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.160028 s, 6.6 MB/s , 0.159983 s, 6.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.184453 s, 5.7 MB/s , 0.173952 s, 6.0 MB/s , 0.159354 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166509 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.199931 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178038 s, 5.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013957 s, 75.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177507 s, 59.1 MB/s , 0.0174236 s, 60.2 MB/s , 0.0136447 s, 76.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198591 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020816 s, 50.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0220357 s, 47.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210004 s, 49.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0177362 s, 59.1 MB/s , 0.0182703 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016718 s, 62.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178228 s, 58.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187005 s, 56.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194003 s, 54.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0185905 s, 56.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199557 s, 52.5 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg310-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8472 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9363 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg310-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg310-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg310-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg310-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg310-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.22476 s, 13.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335684 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360517 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361691 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0329235 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318771 s, 32.9 MB/s , 0.0193096 s, 54.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0407912 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234026 s, 44.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.155491 s, 6.7 MB/s , 0.155432 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.174267 s, 6.0 MB/s , 0.17569 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.172819 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.183563 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188784 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150756 s, 7.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170633 s, 61.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018564 s, 56.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0186549 s, 56.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188642 s, 55.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150302 s, 69.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157342 s, 66.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.018549 s, 56.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0168621 s, 62.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184423 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168692 s, 62.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0186504 s, 56.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0179435 s, 58.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017901 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187492 s, 55.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213218 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235471 s, 44.5 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg310-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8475 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.928 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (130s) == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 15:03:21 (1722798201) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg310-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.260051 s, 64.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200242 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235888 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186442 s, 56.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0366612 s, 28.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0219892 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267576 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0330214 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014247 s, 73.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398717 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.029058 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0583676 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0485484 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0552461 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.059664 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0598771 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0359004 s, 29.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123736 s, 84.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00265398 s, 395 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132076 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121174 s, 86.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185517 s, 56.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017283 s, 60.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136537 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01603 s, 65.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160662 s, 65.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145093 s, 72.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167648 s, 62.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142321 s, 73.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019647 s, 53.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191135 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175293 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174974 s, 59.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg310-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.0108 s, 20.9 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74068 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.10@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.10@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.258734 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013142 s, 79.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161338 s, 65.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0461862 s, 22.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0478885 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0654747 s, 16.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0671955 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0711592 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0653055 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0707209 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0711643 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0742537 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0697388 s, 15.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.247485 s, 4.2 MB/s , 0.244537 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.284771 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.288907 s, 3.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0127088 s, 82.5 MB/s , 0.0128819 s, 81.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0129007 s, 81.3 MB/s , 0.0172575 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158049 s, 66.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153303 s, 68.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.015907 s, 65.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0150469 s, 69.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019915 s, 52.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173212 s, 60.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173187 s, 60.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178814 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141793 s, 74.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192285 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200253 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020821 s, 50.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.03839 s, 20.8 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74079 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.10@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg310-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.269611 s, 62.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167972 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157482 s, 66.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388276 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0535889 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0540632 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0549607 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0595545 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0547544 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0576802 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0564401 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0510227 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0524185 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0638458 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0567279 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0703051 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0659895 s, 15.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125191 s, 83.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128791 s, 81.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114779 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198977 s, 52.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0138826 s, 75.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168363 s, 62.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0146912 s, 71.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153504 s, 68.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159504 s, 65.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163359 s, 64.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157439 s, 66.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159247 s, 65.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182801 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0186815 s, 56.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0175962 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018839 s, 55.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg310-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96096 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.89001 s, 21.4 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg310-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.452787 s, 37.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182251 s, 57.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0445742 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0403137 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0492865 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.047782 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0571227 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.056213 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0489233 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0481698 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0587109 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.078555 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0702801 s, 14.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.264166 s, 4.0 MB/s , 0.277976 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.327545 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.314901 s, 3.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107851 s, 97.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0111505 s, 94.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0168632 s, 62.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112798 s, 93.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134351 s, 78.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.01737 s, 60.4 MB/s , 0.0138022 s, 76.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152216 s, 68.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150681 s, 69.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159927 s, 65.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150502 s, 69.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156695 s, 66.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170794 s, 61.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169293 s, 61.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175482 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019174 s, 54.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.83712 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74267 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96733 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73999 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg310-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.85477 s, 19.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235641 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022314 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227282 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278098 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.032119 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188079 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213705 s, 49.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108409 s, 96.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15386 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.18515 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.182386 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.183834 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.187637 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.190357 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.192879 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.204028 s, 5.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121226 s, 86.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126891 s, 82.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153997 s, 68.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136986 s, 76.5 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0144255 s, 72.7 MB/s , 0.018462 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149132 s, 70.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00449644 s, 233 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159296 s, 65.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175393 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0170091 s, 61.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0162434 s, 64.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158164 s, 66.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01803 s, 58.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170673 s, 61.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208497 s, 50.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg310-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.7103 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4423 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg310-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.23993 s, 69.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184281 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200637 s, 52.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0421215 s, 24.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413902 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0626101 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0608659 s, 17.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0634019 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0640053 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0605166 s, 17.3 MB/s , 0.0637945 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0739187 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0550991 s, 19.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0612238 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0612011 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0745766 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0643749 s, 16.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0110342 s, 95.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159999 s, 65.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191615 s, 54.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156872 s, 66.8 MB/s , 0.0155036 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156563 s, 67.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172582 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015019 s, 69.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154719 s, 67.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0153673 s, 68.2 MB/s , 0.0173936 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175897 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207807 s, 50.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208262 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209286 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245769 s, 42.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg310-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97439 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73639 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg310-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.452253 s, 37.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275584 s, 38.0 MB/s 1048576 bytes (1.0 MB) copied, 0.015536 s, 67.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.029514 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201128 s, 52.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0230505 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177728 s, 59.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139233 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00928901 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.051773 s, 20.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0745264 s, 14.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0832308 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0662645 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.226366 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.219308 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.330714 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.324592 s, 3.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128601 s, 81.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out , 0.0129881 s, 80.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0172743 s, 60.7 MB/s 1048576 bytes (1.0 MB) copied, 0.00361936 s, 290 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158389 s, 66.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169712 s, 61.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150898 s, 69.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159936 s, 65.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222835 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171947 s, 61.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164609 s, 63.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190854 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179172 s, 58.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0149871 s, 70.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016424 s, 63.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234871 s, 44.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.84844 s, 10.6 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7481 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.81472 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7427 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (352s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 15:09:13 (1722798553) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (3s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 15:09:16 (1722798556) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg310-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.59236 s, 292 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg310-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0158 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (51s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 15:10:07 (1722798607) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.10612 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131354 s, 79.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130559 s, 80.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0401882 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400356 s, 26.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0463725 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0458807 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.054686 s, 19.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0537102 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0414125 s, 25.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451558 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0539374 s, 19.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0487865 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0593828 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0523805 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0522499 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0545655 s, 19.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b5904800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00787146 s, 133 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00707819 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112057 s, 93.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00789706 s, 133 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00736764 s, 142 MB/s , 0.00410157 s, 256 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00706416 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106784 s, 98.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0077151 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00124538 s, 842 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00724222 s, 145 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.009471 s, 111 MB/s , 0.00916519 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108953 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102868 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119071 s, 88.1 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95469 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97229 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.176297 s, 95.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191133 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0257135 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243215 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283781 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266187 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284876 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233143 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131889 s, 79.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122955 s, 85.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258047 s, 40.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.130838 s, 8.0 MB/s , 0.144428 s, 7.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.173931 s, 6.0 MB/s , 0.202174 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.230941 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.245758 s, 4.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133742 s, 78.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157683 s, 66.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146541 s, 71.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156866 s, 66.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141341 s, 74.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196583 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152433 s, 68.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0167231 s, 62.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174226 s, 60.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171785 s, 61.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016227 s, 64.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158336 s, 66.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174706 s, 60.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167479 s, 62.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177445 s, 59.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192356 s, 54.5 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg310-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg310-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 5.02182 s, 20.9 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97265 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (66s) == sanityn test 77o: Changing rank should not panic ====== 15:11:13 (1722798673) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (4s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 15:11:17 (1722798677) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (34s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 15:11:51 (1722798711) mds.MDS.mdt.nrs_policies=tbf oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg310-client: oleg310-server: ssh exited with exit code 36 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 oleg310-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg310-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg310-client: oleg310-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (13s) == sanityn test 77r: Change type of tbf policy at run time ========================================================== 15:12:04 (1722798724) mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 8545 kill md_thread xargs PASS 77r (36s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 15:12:40 (1722798760) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_policies=fifo PASS 78 (3s) == sanityn test 79: xattr: intent error ================== 15:12:43 (1722798763) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272506890 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-08-04 15:12:44.000000000 -0400 Modify: 2024-08-04 15:12:44.000000000 -0400 Change: 2024-08-04 15:12:44.000000000 -0400 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (13s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 15:12:56 (1722798776) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 15:12:57 (1722798777) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 15:12:58 (1722798778) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (2s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 15:13:00 (1722798780) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 15:13:01 (1722798781) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 15:13:02 (1722798782) - open/close 2347 (time 1722798794.22 total 10.00 last 234.66) total: 2500 open/close in 10.61 seconds: 235.64 ops/second snapshot_time 1722798810.589999202 secs.nsecs start_time 1722798783.937335872 secs.nsecs elapsed_time 26.652663330 secs.nsecs open 2505 samples [usecs] 105 3767 1199605 611597947 close 2504 samples [usecs] 67 6100 386156 96545082 mknod 2501 samples [usecs] 186 3720 1069996 492976076 unlink 2500 samples [usecs] 304 28242 1563758 6703843426 mkdir 2 samples [usecs] 807 1079 1886 1815490 rmdir 2 samples [usecs] 3176 3387 6563 21558745 rename 2500 samples [usecs] 229 9842 3093892 6778628422 getattr 3763 samples [usecs] 7 1775 60945 6329623 parallel_rename_file 2500 samples [usecs] 229 9842 3093892 6778628422 crossdir_rename 2500 samples [usecs] 229 9842 3093892 6778628422 PASS 81d (29s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 15:13:31 (1722798811) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6938 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6938 PASS 82 (1s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 15:13:32 (1722798812) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 15:13:33 (1722798813) fail_loc=0x8000060b PASS 84 (7s) == sanityn test 85: Lustre API root cache race =========== 15:13:40 (1722798820) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1722798820 Finishing test test1 at 1722798821 PASS 85 (3s) == sanityn test 90: open/create and unlink striped directory ========================================================== 15:13:43 (1722798823) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 15:13:44 (1722798824) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 15:13:46 (1722798826) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 15:13:47 (1722798827) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6291 0x1893 0x240000400 1 6277 0x1885 0x280000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6292 0x1894 0x240000400 1 6278 0x1886 0x280000400 rc1=0 and rc2=0 PASS 93 (9s) == sanityn test 94: signal vs CP callback race =========== 15:13:56 (1722798836) osc.lustre-OST0000-osc-ffff8800b5870000.idle_timeout=0 osc.lustre-OST0000-osc-ffff8800b5904800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b5870000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b5904800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0439403 s, 186 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 3.00184 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5739: 30223 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.023479 s, 174 kB/s osc.lustre-OST0000-osc-ffff8800b5870000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b5904800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b5870000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b5904800.idle_timeout=debug PASS 94 (10s) == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 15:14:06 (1722798846) llite.lustre-ffff8800b5870000.fast_read=0 llite.lustre-ffff8800b5904800.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0221054 s, 94.9 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff8800b5870000.fast_read=1 llite.lustre-ffff8800b5904800.fast_read=1 PASS 95a (20s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 15:14:26 (1722798866) llite.lustre-ffff8800b5870000.fast_read=0 llite.lustre-ffff8800b5904800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.0027901 s, 4.4 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800b5870000.fast_read=1 llite.lustre-ffff8800b5904800.fast_read=1 PASS 95b (8s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 15:14:34 (1722798874) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 15:14:35 (1722798875) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00264491 s, 1.5 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (3s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 15:14:38 (1722798878) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0393111 s, 53.3 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 15:14:40 (1722798880) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0392092 s, 53.5 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 15:14:42 (1722798882) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00284211 s, 4.2 MB/s PASS 100e (3s) == sanityn test 101a: Discard DoM data on unlink ========= 15:14:45 (1722798885) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000784286 s, 5.2 MB/s PASS 101a (2s) == sanityn test 101b: Discard DoM data on rename ========= 15:14:47 (1722798887) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00081198 s, 5.0 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 15:14:50 (1722798890) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000907967 s, 4.5 MB/s PASS 101c (3s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 15:14:54 (1722798894) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1722798894 lockahead_test: lockahead_test.c:1191: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1722798897 Finishing test test23 at 1722798897 PASS 103 (8s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 15:15:01 (1722798901) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 15:15:02 (1722798902) fail_loc=0x416 PASS 105 (23s) == sanityn test 106a: Verify the btime via statx() ======= 15:15:25 (1722798925) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 15:15:26 (1722798926) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) == sanityn test 106c: Verify statx attributes mask ======= 15:15:27 (1722798927) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) == sanityn test 107a: Basic grouplock conflict =========== 15:15:28 (1722798928) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.098593 s, 106 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 6283 0x188b 0x280000400 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 First grouplock blocks second one PASS 107a (5s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 15:15:33 (1722798933) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0952997 s, 110 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6300 0x189c 0x240000400 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6938 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (9s) == sanityn test 108a: lseek: parallel updates ============ 15:15:42 (1722798942) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00304538 s, 3.3 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.30685 s, 1.9 MB/s PASS 108a (6s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 15:15:48 (1722798948) 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions oleg310-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 4 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions oleg310-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 7 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions oleg310-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 12 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions oleg310-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 13 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions oleg310-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 15 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions oleg310-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 18 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions oleg310-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 20 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions oleg310-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 22 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions oleg310-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 25 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions oleg310-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 30 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions oleg310-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 31 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions oleg310-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 35 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions oleg310-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg310-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 39 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions oleg310-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 40 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions oleg310-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 42 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 192.168.203.110@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 192.168.203.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project 0 0 Stopping client oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg310-server' oleg310-server: oleg310-server.virtnet: executing load_modules_local oleg310-server: Loading modules from /home/green/git/lustre-release/lustre oleg310-server: detected 4 online CPUs by sysfs oleg310-server: Force libcfs to create 2 CPU partitions Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre Starting client: oleg310-client.virtnet: -o user_xattr,flock oleg310-server@tcp:/lustre /mnt/lustre2 PASS 109 (460s) == sanityn test 110: do not grant another lock on resend ========================================================== 15:23:28 (1722799408) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-08-04 15:23:29.000000000 -0400 Modify: 2024-08-04 15:23:29.000000000 -0400 Change: 2024-08-04 15:23:29.000000000 -0400 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg310-client.virtnet oleg310-client.virtnet PASS 110 (65s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 15:24:34 (1722799474) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (2s) == sanityn test 112: update max-inherit in default LMV === 15:24:35 (1722799475) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 15:24:36 (1722799476) lustre-OST0000-osc-ffff88009d153000 active. lustre-OST0001-osc-ffff88009d153000 active. lustre-OST0000-osc-ffff88009d153000 active. lustre-OST0001-osc-ffff88009d153000 active. MGC192.168.203.110@tcp active. PASS 113 (2s) == sanityn test 114: implicit default LMV inherit ======== 15:24:38 (1722799478) SKIP: sanityn test_114 We need at least 2 MDTs for this test SKIP 114 (1s) == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 15:24:39 (1722799479) SKIP: sanityn test_115 ldiskfs only test SKIP 115 (2s) == sanityn test 116: DNE: Set default LMV layout from a remote client ========================================================== 15:24:41 (1722799481) SKIP: sanityn test_116 needs >= 2 MDTs SKIP 116 (1s) cleanup: ====================================================== == sanityn test complete, duration 4499 sec ============== 15:24:42 (1722799482) === sanityn: start cleanup 15:24:43 (1722799483) === Stopping clients: oleg310-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg310-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 15:25:59 (1722799559) ===