-----============= acceptance-small: sanityn ============----- Mon May 11 17:26:47 EDT 2026 oleg236-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 Started clients oleg236-client.virtnet: 192.168.202.136@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt) oleg236-client.virtnet: executing check_config_client /mnt/lustre oleg236-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg236-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff986444f2c000.idle_timeout=debug osc.lustre-OST0000-osc-ffff9864494bb000.idle_timeout=debug osc.lustre-OST0001-osc-ffff986444f2c000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9864494bb000.idle_timeout=debug disable quota as required oleg236-server: oleg236-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0438144 s, 23.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f7874] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 17:27:20 (1778534840) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (6s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 17:27:26 (1778534846) -rw-r--r-- 1 root root 0 May 11 17:27 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (4s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 17:27:30 (1778534850) -rw-r--r-- 1 root root 0 May 11 17:27 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (4s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 17:27:35 (1778534855) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (4s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 17:27:39 (1778534859) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (4s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 17:27:43 (1778534863) total 1 -rwxrwxrwx 1 root root 0 May 11 17:27 f2a -rwxrwxrwx 1 root root 0 May 11 17:27 f2b total 1 -rwxrwxrwx 1 root root 0 May 11 17:27 f2a -rwxrwxrwx 1 root root 0 May 11 17:27 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.00554433 s, 92.3 kB/s PASS 2e (5s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 17:27:48 (1778534868) /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn has type file OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn has perms 0777 OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn is owned by user #500 OK /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn is owned by group #500 OK Can't lstat /mnt/lustre2/d2f.sanityn/remote_dir/f2f.sanityn: No such file or directory PASS 2f (5s) == sanityn test 2g: check blocks update on sync write ==== 17:27:53 (1778534873) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.359585 s, 5.8 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (5s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 17:27:58 (1778534878) PASS 3 (5s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 17:28:03 (1778534883) PASS 4 (5s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 17:28:08 (1778534888) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (4s) == sanityn test 6: remove of open file on other node ============================================================================ 17:28:12 (1778534892) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (4s) == sanityn test 7: remove of open directory on other node ======================================================================= 17:28:16 (1778534896) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (5s) == sanityn test 8: remove of open special file on other node ==================================================================== 17:28:21 (1778534901) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (4s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 17:28:26 (1778534906) PASS 9a (5s) == sanityn test 9b: append to striped sparse file ======== 17:28:30 (1778534910) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312361 s, 33.6 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.0371817 s, 0.1 kB/s Data read (expecting 'foo'): foo PASS 9b (5s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 17:28:35 (1778534915) 1+0 records in 1+0 records out 1 byte copied, 0.00728049 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0104978 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00955699 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0127523 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00411814 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0118807 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00386479 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00350227 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00857192 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00450634 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00522903 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00798811 s, 0.1 kB/s PASS 10a (5s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 17:28:40 (1778534920) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.00800174 s, 384 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0483179 s, 84.8 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.000290546 s, 10.6 MB/s PASS 10b (4s) == sanityn test 11: execution of file opened for write should return error ============================================================== 17:28:44 (1778534924) striped dir -i1 -c2 -H crush /mnt/lustre/d11 multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (5s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 17:28:49 (1778534929) warning: '-runtime' deprecated, use '-t runtime' instead - link 537 (time 1778534940.87 total 10.01 last 53.63) - link 1090 (time 1778534950.88 total 20.02 last 55.24) - link 1540 (time 1778534960.90 total 30.04 last 44.94) - link 1964 (time 1778534970.90 total 40.04 last 42.40) - link 2376 (time 1778534980.93 total 50.08 last 41.05) - link 2789 (time 1778534990.94 total 60.08 last 41.28) - link 3259 (time 1778535000.96 total 70.11 last 46.88) - link 3800 (time 1778535010.97 total 80.11 last 54.06) - link 4314 (time 1778535020.97 total 90.11 last 51.40) total: 4847 link in 100.01 seconds: 48.46 ops/second using seed 1741339973 running for 100 seconds - stat 10000 (time 1778534939 ; total 8 ; last 8) - stat 20000 (time 1778534946 ; total 15 ; last 7) - stat 30000 (time 1778534954 ; total 23 ; last 8) - stat 40000 (time 1778534962 ; total 31 ; last 8) - stat 50000 (time 1778534971 ; total 40 ; last 9) - stat 60000 (time 1778534979 ; total 48 ; last 8) - stat 68670 (time 1778534990 ; total 59 ; last 11) - stat 70000 (time 1778534992 ; total 61 ; last 2) - stat 78565 (time 1778535003 ; total 72 ; last 11) - stat 80000 (time 1778535004 ; total 73 ; last 1) - stat 88874 (time 1778535015 ; total 84 ; last 11) - stat 90000 (time 1778535016 ; total 85 ; last 1) - stat 98047 (time 1778535027 ; total 96 ; last 11) - stat 100000 (time 1778535029 ; total 98 ; last 2) total: 101342 stats in 100 seconds: 1013.419983 stats/second - unlinked 0 (time 1778534981 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile4847) error: No such file or directory total: 4846 unlinks in 86 seconds: 56.348839 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (20996) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (20998) - No such process PASS 12 (142s) == sanityn test 13: test directory page revocation ======= 17:31:11 (1778535071) striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d13 f13 f13 PASS 13 (6s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 17:31:17 (1778535077) striped dir -i0 -c2 -H all_char /mnt/lustre/d14aa.sanityn multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 22194 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (5s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 17:31:22 (1778535082) striped dir -i0 -c2 -H crush /mnt/lustre/d14ab.sanityn open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 22779 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (6s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 17:31:28 (1778535088) striped dir -i0 -c2 -H crush /mnt/lustre/d14b.sanityn truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 23362 Terminated $DIR1/$tdir/sleep 60 PASS 14b (5s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 17:31:33 (1778535093) striped dir -i0 -c2 -H crush /mnt/lustre/d14c.sanityn cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 23950 Terminated $DIR1/$tdir/sleep 60 PASS 14c (5s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 17:31:38 (1778535098) striped dir -i0 -c2 -H all_char /mnt/lustre/d14d.sanityn chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 24538 Terminated $DIR1/$tdir/sleep 60 PASS 14d (5s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 17:31:44 (1778535104) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7519232 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7519232kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (18s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 17:32:01 (1778535121) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.267074 s, 7.9 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.155803 s, 13.5 MB/s Chance of close/open is 1 in 50 Seed set to 5123 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[0] 1778535124.293274 write 0x43c01d thru 0x44b319 (0xf2fd bytes) 000003[0] 1778535124.324504 mapwrite 0x3816f5 thru 0x3877ea (0x60f6 bytes) 000004[0] 1778535124.501351 write 0x0e82cf thru 0x0f6e30 (0xeb62 bytes) 000005[1] 1778535124.511156 mapwrite 0x59256b thru 0x598272 (0x5d08 bytes) 000007[1] 1778535124.640577 read 0x051727 thru 0x05d984 (0xc25e bytes) 000011[0] 1778535124.693871 mapwrite 0x36eb7c thru 0x36f1f5 (0x67a bytes) 000012[1] 1778535124.791874 mapwrite 0x7ac9e5 thru 0x7b43d9 (0x79f5 bytes) 000013[0] 1778535124.882906 mapread 0x3c505a thru 0x3d452e (0xf4d5 bytes) 000014[1] 1778535124.923535 mapread 0x142f5c thru 0x14de59 (0xaefe bytes) 000015[0] 1778535124.962875 mapwrite 0x81f2b7 thru 0x825f7f (0x6cc9 bytes) 000016[1] 1778535125.066399 mapread 0x3d9025 thru 0x3e19af (0x898b bytes) 000017[0] 1778535125.095805 write 0x6682c6 thru 0x677e81 (0xfbbc bytes) 000018[1] 1778535125.113654 write 0x66cf2c thru 0x670243 (0x3318 bytes) 000019[0] 1778535125.183634 read 0x397932 thru 0x398e3f (0x150e bytes) 000020[1] 1778535125.193847 mapread 0x0431c3 thru 0x049802 (0x6640 bytes) 000021[1] 1778535125.214081 mapwrite 0x141d2a thru 0x150968 (0xec3f bytes) 000022[1] 1778535125.344251 mapwrite 0x6b45f1 thru 0x6b6b0e (0x251e bytes) 000023[1] 1778535125.415306 write 0x4d2d02 thru 0x4d84f2 (0x57f1 bytes) truncating to largest ever: 0x7997a9 000024[0] 1778535125.424782 trunc from 0x825f80 to 0x7997a8 (0x8c7d7 bytes) 000025[1] 1778535125.472919 read 0x1d1a37 thru 0x1dc53f (0xab09 bytes) truncating to largest ever: 0x7b0b86 000026[0] 1778535125.499653 trunc from 0x7997a9 to 0x7b0b85 (0x173dd bytes) 000027[0] 1778535125.523638 mapread 0x3e961d thru 0x3ece88 (0x386c bytes) 000029[1] 1778535125.599717 read 0x437835 thru 0x441604 (0x9dd0 bytes) 000030[1] 1778535125.647738 mapread 0x0d397b thru 0x0d8ddd (0x5463 bytes) 000031[1] 1778535125.696042 mapread 0x3644ed thru 0x36ae07 (0x691b bytes) 000033[0] 1778535125.799958 read 0x1c6884 thru 0x1d17cb (0xaf48 bytes) 000034[0] 1778535125.875490 mapwrite 0x5876a3 thru 0x59153c (0x9e9a bytes) 000035[0] 1778535126.108171 mapread 0x4e1596 thru 0x4e4bb1 (0x361c bytes) 000036[1] 1778535126.108807 trunc from 0x7b0b86 to 0x03165d (0x77f528 bytes) 000037[0] 1778535126.198626 read 0x022164 thru 0x03165d (0xf4fa bytes) 000038[1] 1778535126.250179 mapwrite 0x742590 thru 0x74662e (0x409f bytes) 000043[0] 1778535126.379204 mapread 0x0d037c thru 0x0d06a2 (0x327 bytes) 000044[0] 1778535126.404104 write 0x1349c9 thru 0x13a0a8 (0x56e0 bytes) 000047[0] 1778535126.522020 mapwrite 0x18d077 thru 0x192350 (0x52da bytes) 000048[0] 1778535126.582864 mapread 0x08deb7 thru 0x09b39e (0xd4e8 bytes) 000049[0] 1778535126.583761 read 0x485541 thru 0x48dca2 (0x8762 bytes) 000051[0] 1778535126.606158 mapread 0x190942 thru 0x19b7a2 (0xae61 bytes) 000052[1] 1778535126.608143 read 0x564b85 thru 0x56e39a (0x9816 bytes) 000053[0] 1778535126.663910 read 0x5f4ef5 thru 0x5fde35 (0x8f41 bytes) 000054[0] 1778535126.678275 mapwrite 0x3dbb18 thru 0x3dda71 (0x1f5a bytes) 000055[0] 1778535126.740081 write 0x89ba52 thru 0x8a456a (0x8b19 bytes) 000056[1] 1778535126.770317 mapwrite 0x3c4624 thru 0x3d154d (0xcf2a bytes) 000057[1] 1778535126.952408 mapwrite 0x858242 thru 0x85d514 (0x52d3 bytes) 000059[1] 1778535127.153823 mapwrite 0x90deff thru 0x91cf23 (0xf025 bytes) 000060[1] 1778535127.366063 read 0x4242e0 thru 0x4245fb (0x31c bytes) 000061[0] 1778535127.440171 read 0x55d5e5 thru 0x56420c (0x6c28 bytes) 000064[0] 1778535127.535997 trunc from 0x91cf24 to 0x6fdbc1 (0x21f362 bytes) 000066[0] 1778535127.572908 trunc from 0x6fdbc2 to 0x73a815 (0x3cc54 bytes) 000068[1] 1778535127.627366 write 0x628a82 thru 0x635d4b (0xd2ca bytes) 000069[1] 1778535127.705530 write 0x3fea66 thru 0x3fead3 (0x6e bytes) 000071[0] 1778535127.771588 mapread 0x3d225f thru 0x3d85ba (0x635c bytes) 000072[0] 1778535127.849246 read 0x700dbb thru 0x70c003 (0xb249 bytes) 000074[0] 1778535127.882870 write 0x28beb9 thru 0x28d44d (0x1595 bytes) 000075[1] 1778535127.914394 write 0x602e2e thru 0x6048f4 (0x1ac7 bytes) 000078[1] 1778535127.982723 read 0x1d2b3b thru 0x1deedd (0xc3a3 bytes) 000080[1] 1778535128.106774 write 0x059fd7 thru 0x05fbd0 (0x5bfa bytes) 000082[0] 1778535128.137980 mapwrite 0x62e16e thru 0x63c15a (0xdfed bytes) 000087[0] 1778535128.398284 trunc from 0x73a816 to 0x63cffc (0xfd819 bytes) 000088[0] 1778535128.447957 mapread 0x220cf6 thru 0x225445 (0x4750 bytes) 000089[1] 1778535128.498265 mapwrite 0x927816 thru 0x932520 (0xad0b bytes) 000090[0] 1778535128.634620 mapwrite 0x05e7c0 thru 0x06a5b4 (0xbdf5 bytes) 000091[1] 1778535128.763200 mapread 0x7f7fd3 thru 0x803269 (0xb297 bytes) 000092[1] 1778535128.803620 mapwrite 0x2db08a thru 0x2e4dbb (0x9d32 bytes) 000094[0] 1778535128.989388 read 0x754657 thru 0x75af3a (0x68e4 bytes) 000095[1] 1778535129.016086 mapread 0x886e35 thru 0x88ae3d (0x4009 bytes) 000096[1] 1778535129.053823 trunc from 0x932521 to 0x27996b (0x6b8bb5 bytes) 000098[1] 1778535129.131671 write 0x4cb62b thru 0x4cf10b (0x3ae1 bytes) 000099[0] 1778535129.152864 read 0x14a17a thru 0x14fd98 (0x5c1f bytes) 000100[1] 1778535129.190226 write 0x951d51 thru 0x95757e (0x582e bytes) 000101[0] 1778535129.202892 mapread 0x635363 thru 0x6452cf (0xff6d bytes) truncating to largest ever: 0x97a52a 000102[0] 1778535129.405286 trunc from 0x95757f to 0x97a529 (0x22fab bytes) 000103[0] 1778535129.483639 read 0x18b59e thru 0x190b07 (0x556a bytes) 000104[0] 1778535129.574116 trunc from 0x97a52a to 0x2b7bc2 (0x6c2967 bytes) 000105[1] 1778535129.616425 mapwrite 0x51a53c thru 0x52591b (0xb3e0 bytes) 000107[0] 1778535129.759037 mapread 0x378505 thru 0x384d50 (0xc84c bytes) 000108[1] 1778535129.827027 write 0x04f352 thru 0x05defa (0xeba9 bytes) 000111[1] 1778535129.865679 mapwrite 0x0a32f7 thru 0x0ae14c (0xae56 bytes) 000113[1] 1778535129.984062 trunc from 0x52591c to 0x6ad8d1 (0x187fb6 bytes) 000114[0] 1778535130.040092 mapread 0x0ca3a1 thru 0x0d057f (0x61df bytes) 000115[0] 1778535130.114675 mapwrite 0x50336d thru 0x50e4fc (0xb190 bytes) 000117[0] 1778535130.275124 read 0x5daf4e thru 0x5dfed7 (0x4f8a bytes) 000118[1] 1778535130.302884 mapread 0x381144 thru 0x387a74 (0x6931 bytes) 000122[1] 1778535130.356178 read 0x412d8c thru 0x417860 (0x4ad5 bytes) 000123[1] 1778535130.396897 write 0x16e9c7 thru 0x17ad15 (0xc34f bytes) 000124[1] 1778535130.451982 mapread 0x3874d9 thru 0x38a4b6 (0x2fde bytes) 000125[1] 1778535130.468221 write 0x74584f thru 0x7465a5 (0xd57 bytes) 000126[1] 1778535130.469142 mapwrite 0x04e5ed thru 0x05b59b (0xcfaf bytes) 000127[0] 1778535130.690782 write 0x90491d thru 0x90ebce (0xa2b2 bytes) 000128[1] 1778535130.734651 read 0x605f4b thru 0x60a074 (0x412a bytes) 000129[1] 1778535130.765994 write 0x696696 thru 0x697b77 (0x14e2 bytes) 000135[1] 1778535130.888561 trunc from 0x90ebcf to 0x6fdda6 (0x210e28 bytes) 000137[0] 1778535130.936626 write 0x721b3b thru 0x725f70 (0x4436 bytes) 000139[0] 1778535130.974092 mapwrite 0x6277e0 thru 0x62f7cf (0x7ff0 bytes) 000141[1] 1778535131.172530 write 0x0a1971 thru 0x0aa9c9 (0x9059 bytes) 000142[0] 1778535131.214423 read 0x47822d thru 0x47cfc7 (0x4d9b bytes) 000143[1] 1778535131.306852 read 0x4f5b5d thru 0x4f6b9d (0x1041 bytes) 000146[0] 1778535131.362408 trunc from 0x725f71 to 0x5dfb8a (0x1463e6 bytes) 000148[0] 1778535131.431754 mapread 0x074c10 thru 0x082866 (0xdc57 bytes) 000149[1] 1778535131.519854 read 0x033356 thru 0x042078 (0xed23 bytes) 000150[0] 1778535131.582634 mapwrite 0x2c16c5 thru 0x2c92a9 (0x7be5 bytes) 000155[1] 1778535131.807204 mapwrite 0x72b4e7 thru 0x7383b7 (0xced1 bytes) 000157[0] 1778535132.162251 write 0x7aef04 thru 0x7bedce (0xfecb bytes) 000158[0] 1778535132.224305 write 0x5ba38b thru 0x5bb350 (0xfc6 bytes) 000160[0] 1778535132.293554 mapread 0x59b335 thru 0x59cad7 (0x17a3 bytes) 000162[1] 1778535132.344626 mapread 0x1ddd74 thru 0x1e542f (0x76bc bytes) 000165[1] 1778535132.454855 trunc from 0x7bedcf to 0x826c46 (0x67e78 bytes) 000166[1] 1778535132.546881 write 0x0a2aa6 thru 0x0ac565 (0x9ac0 bytes) 000168[0] 1778535132.624593 mapwrite 0x008257 thru 0x015fc3 (0xdd6d bytes) 000172[1] 1778535132.890141 mapread 0x3aeea6 thru 0x3b3bb7 (0x4d12 bytes) 000177[1] 1778535133.080811 write 0x9cb8d8 thru 0x9d532c (0x9a55 bytes) 000180[0] 1778535133.163979 write 0x98c512 thru 0x98f75b (0x324a bytes) 000183[1] 1778535133.312907 mapread 0x4821ae thru 0x483da8 (0x1bfb bytes) 000189[1] 1778535133.440291 trunc from 0x9d532d to 0x928ed2 (0xac45a bytes) 000191[0] 1778535133.501240 mapread 0x08a938 thru 0x0939a9 (0x9072 bytes) 000193[0] 1778535133.594343 write 0x2c748e thru 0x2ca90f (0x3482 bytes) 000194[1] 1778535133.632956 mapwrite 0x0e12fb thru 0x0e92e1 (0x7fe7 bytes) 000195[0] 1778535133.758885 mapwrite 0x3a6e61 thru 0x3ac8ee (0x5a8e bytes) 000196[1] 1778535133.892018 write 0x61fc63 thru 0x627419 (0x77b7 bytes) 000203[1] 1778535134.019147 trunc from 0x928ed3 to 0x55cb4d (0x3cc385 bytes) 000204[1] 1778535134.057584 mapwrite 0x29b16d thru 0x2a6ed4 (0xbd68 bytes) 000206[0] 1778535134.154598 mapread 0x17870b thru 0x17ff99 (0x788f bytes) 000207[0] 1778535134.271623 mapwrite 0x01fb39 thru 0x02bfb1 (0xc479 bytes) 000210[0] 1778535134.498831 mapread 0x00100c thru 0x00de05 (0xcdfa bytes) 000211[0] 1778535134.595304 mapread 0x02ed0a thru 0x0397d3 (0xaaca bytes) 000214[0] 1778535134.682341 trunc from 0x55cb4e to 0x921665 (0x3c4b18 bytes) 000216[0] 1778535134.773528 trunc from 0x921666 to 0x631f3c (0x2ef729 bytes) 000217[1] 1778535134.805414 mapread 0x3940a6 thru 0x39600e (0x1f69 bytes) 000221[1] 1778535134.894211 write 0x5b8111 thru 0x5c62c3 (0xe1b3 bytes) 000223[1] 1778535134.958675 mapwrite 0x20dafa thru 0x21614c (0x8653 bytes) 000224[1] 1778535135.031328 write 0x74fc4a thru 0x755cf7 (0x60ae bytes) 000225[1] 1778535135.032783 write 0x8a36f3 thru 0x8a8a37 (0x5345 bytes) 000226[1] 1778535135.044952 mapwrite 0x250584 thru 0x2537bc (0x3239 bytes) 000227[0] 1778535135.123839 mapread 0x1640fb thru 0x170d1b (0xcc21 bytes) 000228[0] 1778535135.248492 trunc from 0x8a8a38 to 0x3ced14 (0x4d9d23 bytes) 000229[1] 1778535135.286113 mapwrite 0x160df3 thru 0x170129 (0xf337 bytes) 000230[1] 1778535135.517416 mapread 0x3797fa thru 0x381e7a (0x8681 bytes) 000231[0] 1778535135.618138 write 0x6e380f thru 0x6eae29 (0x761b bytes) 000233[0] 1778535135.655734 trunc from 0x6eae2a to 0x66e744 (0x7c6e5 bytes) 000236[0] 1778535135.741645 read 0x4b14ae thru 0x4b7864 (0x63b7 bytes) 000239[1] 1778535135.800197 write 0x750bf0 thru 0x75d38a (0xc79b bytes) 000240[0] 1778535135.842888 mapwrite 0x156fe9 thru 0x15bbad (0x4bc5 bytes) 000241[0] 1778535135.942628 read 0x33a14f thru 0x33b12c (0xfde bytes) 000242[0] 1778535135.982593 read 0x080714 thru 0x08632d (0x5c1a bytes) 000243[0] 1778535136.036006 trunc from 0x75d38b to 0x1e9839 (0x573b51 bytes) 000247[0] 1778535136.087965 read 0x0b5cee thru 0x0b6feb (0x12fe bytes) 000250[1] 1778535136.089066 write 0x278b50 thru 0x27ef5b (0x640c bytes) 000252[0] 1778535136.153787 mapwrite 0x5111c8 thru 0x5194f4 (0x832d bytes) 000253[1] 1778535136.305105 write 0x89b992 thru 0x8a9014 (0xd683 bytes) 000254[1] 1778535136.329113 mapread 0x739642 thru 0x743b69 (0xa528 bytes) 000255[1] 1778535136.422823 read 0x123672 thru 0x12d2d2 (0x9c61 bytes) 000256[1] 1778535136.529310 mapwrite 0x86cb8a thru 0x87209a (0x5511 bytes) 000258[0] 1778535136.585095 write 0x8d393f thru 0x8d9ad4 (0x6196 bytes) 000259[0] 1778535136.629853 mapread 0x615299 thru 0x6219f1 (0xc759 bytes) 000260[0] 1778535136.680425 mapwrite 0x72ec6d thru 0x73b40c (0xc7a0 bytes) 000261[0] 1778535136.846073 write 0x193519 thru 0x1960e0 (0x2bc8 bytes) 000262[0] 1778535136.877037 trunc from 0x8d9ad5 to 0x385987 (0x55414d bytes) 000263[0] 1778535136.933121 mapread 0x192da1 thru 0x1a29fb (0xfc5b bytes) 000264[1] 1778535137.042910 mapwrite 0x7825c4 thru 0x79257c (0xffb9 bytes) 000265[0] 1778535137.251931 read 0x35be38 thru 0x35f3ac (0x3575 bytes) 000266[1] 1778535137.311979 write 0x60e3da thru 0x61470c (0x6333 bytes) 000267[1] 1778535137.337184 trunc from 0x79257d to 0x1c74ad (0x5cb0cf bytes) 000268[0] 1778535137.395577 read 0x0e4a4d thru 0x0e9bcc (0x5180 bytes) 000269[1] 1778535137.442416 mapwrite 0x951902 thru 0x95e846 (0xcf45 bytes) 000270[1] 1778535137.624954 mapwrite 0x7a03c4 thru 0x7aff60 (0xfb9d bytes) 000271[0] 1778535137.828044 mapwrite 0x8613fb thru 0x86ced2 (0xbad8 bytes) 000272[1] 1778535138.038428 mapread 0x906b18 thru 0x90ecf1 (0x81da bytes) 000273[1] 1778535138.116556 trunc from 0x95e847 to 0x411bf1 (0x54cc55 bytes) 000274[1] 1778535138.195713 mapwrite 0x026cef thru 0x033e8b (0xd19d bytes) 000275[0] 1778535138.407688 trunc from 0x411bf2 to 0x060fdc (0x3b0c15 bytes) 000279[0] 1778535138.568814 mapread 0x057229 thru 0x060fdc (0x9db4 bytes) 000280[0] 1778535138.626501 write 0x84585d thru 0x853960 (0xe104 bytes) 000284[1] 1778535138.765846 trunc from 0x853961 to 0x435fbc (0x41d9a4 bytes) 000286[0] 1778535138.878094 write 0x0fd3f5 thru 0x1062b5 (0x8ec1 bytes) 000289[0] 1778535138.986172 trunc from 0x435fbd to 0x904d61 (0x4ceda5 bytes) 000290[0] 1778535139.028131 mapwrite 0x7ae8a6 thru 0x7af530 (0xc8b bytes) 000292[1] 1778535139.230675 trunc from 0x904d62 to 0x299e36 (0x66af2b bytes) 000293[0] 1778535139.305066 read 0x01aa02 thru 0x01e6bb (0x3cba bytes) 000295[1] 1778535139.360106 mapwrite 0x90f2c5 thru 0x910b12 (0x184e bytes) 000298[1] 1778535139.529967 read 0x6dc014 thru 0x6df60a (0x35f7 bytes) 000299[1] 1778535139.596216 mapread 0x1fd874 thru 0x202640 (0x4dcd bytes) 000300[0] 1778535139.664779 trunc from 0x910b13 to 0x8172da (0xf9838 bytes) 000302[1] 1778535139.763295 mapread 0x6eb276 thru 0x6f3854 (0x85df bytes) 000303[1] 1778535139.871997 read 0x65dfe3 thru 0x65ed5d (0xd7b bytes) 000304[0] 1778535139.901330 mapwrite 0x873d1a thru 0x879e2a (0x6111 bytes) 000308[1] 1778535140.218575 trunc from 0x879e2b to 0x7e69df (0x9344b bytes) 000309[0] 1778535140.304170 mapwrite 0x175f8f thru 0x17afc9 (0x503b bytes) 000310[1] 1778535140.447789 read 0x506e39 thru 0x50de3c (0x7004 bytes) 000314[0] 1778535140.508766 write 0x0fe176 thru 0x1041b3 (0x603e bytes) 000317[1] 1778535140.597107 mapread 0x1c887a thru 0x1cf314 (0x6a9b bytes) 000318[0] 1778535140.694498 read 0x530051 thru 0x531548 (0x14f8 bytes) 000320[1] 1778535140.805130 mapwrite 0x27fe7f thru 0x28794a (0x7acc bytes) 000322[1] 1778535140.942729 mapwrite 0x954a7d thru 0x95c566 (0x7aea bytes) 000324[1] 1778535141.060535 mapread 0x92acc6 thru 0x933840 (0x8b7b bytes) 000326[1] 1778535141.062562 mapwrite 0x73aca7 thru 0x7421a8 (0x7502 bytes) 000327[0] 1778535141.184949 write 0x139436 thru 0x14061a (0x71e5 bytes) 000329[1] 1778535141.237254 mapwrite 0x4b132a thru 0x4b700e (0x5ce5 bytes) 000333[0] 1778535141.436418 read 0x51c132 thru 0x5237f4 (0x76c3 bytes) 000334[1] 1778535141.482534 mapread 0x2256a1 thru 0x225efc (0x85c bytes) 000336[0] 1778535141.524702 read 0x3f61a3 thru 0x3ffd44 (0x9ba2 bytes) truncating to largest ever: 0x9c3594 000337[0] 1778535141.551093 trunc from 0x95c567 to 0x9c3593 (0x6702d bytes) 000341[0] 1778535141.662803 trunc from 0x9c3594 to 0x9aa653 (0x18f40 bytes) 000345[1] 1778535141.819980 trunc from 0x9aa654 to 0x529fad (0x4806a6 bytes) 000346[1] 1778535141.886370 trunc from 0x529fae to 0x353807 (0x1d67a6 bytes) 000349[1] 1778535141.957510 write 0x7d8ed9 thru 0x7db503 (0x262b bytes) 000351[1] 1778535142.009737 write 0x53fc4f thru 0x5430eb (0x349d bytes) 000352[0] 1778535142.041429 mapread 0x0b3c6b thru 0x0b5672 (0x1a08 bytes) 000355[0] 1778535142.156131 mapwrite 0x80a18d thru 0x80ff79 (0x5ded bytes) 000356[0] 1778535142.359640 write 0x79770f thru 0x79c3ac (0x4c9e bytes) 000357[1] 1778535142.385128 mapread 0x3ddd57 thru 0x3e4e26 (0x70d0 bytes) 000359[0] 1778535142.537816 read 0x1771ca thru 0x17d2b6 (0x60ed bytes) 000361[0] 1778535142.681234 read 0x01dcc7 thru 0x02458f (0x68c9 bytes) 000363[0] 1778535142.750504 read 0x1f10f9 thru 0x1f8ff1 (0x7ef9 bytes) 000365[1] 1778535142.751976 write 0x8f4c03 thru 0x8fbbd2 (0x6fd0 bytes) 000366[0] 1778535142.773646 mapread 0x22370a thru 0x232d6a (0xf661 bytes) 000370[1] 1778535142.894983 read 0x7055f9 thru 0x714df6 (0xf7fe bytes) 000371[0] 1778535142.937557 trunc from 0x8fbbd3 to 0x806075 (0xf5b5d bytes) 000372[1] 1778535143.054269 trunc from 0x806076 to 0x622824 (0x1e3851 bytes) 000374[1] 1778535143.204303 mapwrite 0x5a6253 thru 0x5aaba6 (0x4954 bytes) 000377[1] 1778535143.348020 mapread 0x216da5 thru 0x21a69f (0x38fb bytes) 000380[1] 1778535143.406666 mapread 0x373784 thru 0x37affd (0x787a bytes) 000382[0] 1778535143.519378 write 0x358fc2 thru 0x360a0f (0x7a4e bytes) 000383[1] 1778535143.566877 read 0x42f531 thru 0x439f62 (0xaa32 bytes) 000384[1] 1778535143.620870 mapread 0x495a70 thru 0x497e6a (0x23fb bytes) 000386[1] 1778535143.656164 write 0x7fa8da thru 0x8032f8 (0x8a1f bytes) 000388[0] 1778535143.725337 trunc from 0x8032f9 to 0x2924cf (0x570e29 bytes) 000390[0] 1778535143.776136 read 0x059144 thru 0x05a66c (0x1529 bytes) 000391[1] 1778535143.811713 mapread 0x24b514 thru 0x257490 (0xbf7d bytes) 000392[1] 1778535143.904796 write 0x594b3d thru 0x595ccb (0x118f bytes) 000393[1] 1778535143.937114 write 0x847cbe thru 0x8510d7 (0x941a bytes) 000394[1] 1778535143.985925 mapread 0x0f2a1a thru 0x0f53e6 (0x29cd bytes) 000395[0] 1778535144.047912 write 0x5fd051 thru 0x5ff46f (0x241f bytes) 000396[1] 1778535144.083099 mapread 0x3d3b6f thru 0x3dd3e1 (0x9873 bytes) 000398[0] 1778535144.177907 write 0x24e2e9 thru 0x2519f0 (0x3708 bytes) 000399[1] 1778535144.377224 mapread 0x6c095d thru 0x6d08bc (0xff60 bytes) 000403[1] 1778535144.537020 write 0x59194b thru 0x592352 (0xa08 bytes) 000404[1] 1778535144.559769 read 0x6b55cc thru 0x6b60fe (0xb33 bytes) 000405[0] 1778535144.603989 read 0x5dd29d thru 0x5e0b87 (0x38eb bytes) 000408[0] 1778535144.684837 read 0x6b17d3 thru 0x6b7235 (0x5a63 bytes) 000409[1] 1778535144.688339 read 0x5d61fd thru 0x5d7a96 (0x189a bytes) 000410[1] 1778535144.722520 read 0x6a078c thru 0x6ab788 (0xaffd bytes) 000411[0] 1778535144.723202 mapread 0x7a8c5b thru 0x7a8cbf (0x65 bytes) 000412[0] 1778535144.723816 trunc from 0x8510d8 to 0x53fecd (0x31120a bytes) 000414[1] 1778535144.863717 trunc from 0x53fece to 0x1372de (0x408bef bytes) 000415[1] 1778535144.911378 trunc from 0x1372df to 0x139e75 (0x2b97 bytes) 000417[0] 1778535144.964910 read 0x02ff3c thru 0x03ec57 (0xed1c bytes) 000418[0] 1778535145.010535 write 0x385b6e thru 0x394db3 (0xf246 bytes) 000419[0] 1778535145.049327 read 0x0b23b7 thru 0x0b884b (0x6495 bytes) 000421[0] 1778535145.053592 trunc from 0x394db4 to 0x51cca2 (0x187eef bytes) 000424[1] 1778535145.150740 trunc from 0x51cca3 to 0x2440ce (0x2d8bd4 bytes) 000427[0] 1778535145.275057 write 0x3de03b thru 0x3e7120 (0x90e6 bytes) 000430[1] 1778535145.332742 mapwrite 0x930c15 thru 0x93ef54 (0xe340 bytes) 000431[1] 1778535145.565474 mapread 0x095458 thru 0x09ff86 (0xab2f bytes) 000432[0] 1778535145.680863 mapread 0x42d375 thru 0x42dd28 (0x9b4 bytes) 000437[0] 1778535145.738313 mapwrite 0x999a7b thru 0x99dfdf (0x4565 bytes) 000438[1] 1778535145.920076 mapwrite 0x45ffe9 thru 0x46f54b (0xf563 bytes) 000440[0] 1778535146.148758 read 0x09539f thru 0x09c7cc (0x742e bytes) 000442[0] 1778535146.307836 mapread 0x3365ca thru 0x342f46 (0xc97d bytes) 000443[1] 1778535146.333256 read 0x7c30a4 thru 0x7caa47 (0x79a4 bytes) 000444[1] 1778535146.399946 mapread 0x797322 thru 0x79c814 (0x54f3 bytes) 000446[0] 1778535146.432115 mapwrite 0x6708bc thru 0x672904 (0x2049 bytes) 000447[0] 1778535146.550392 mapwrite 0x4f7e42 thru 0x50690a (0xeac9 bytes) 000449[0] 1778535146.758902 write 0x9d977c thru 0x9d9fd0 (0x855 bytes) 000451[1] 1778535146.796298 write 0x2d286f thru 0x2d97cc (0x6f5e bytes) 000452[0] 1778535146.826973 mapread 0x36a90f thru 0x372559 (0x7c4b bytes) 000453[1] 1778535146.893885 mapwrite 0x6bad32 thru 0x6c83ba (0xd689 bytes) 000454[0] 1778535147.059542 mapwrite 0x6b0700 thru 0x6b6edc (0x67dd bytes) 000456[0] 1778535147.189114 write 0x210fa9 thru 0x21ecb4 (0xdd0c bytes) 000459[1] 1778535147.223758 write 0x553f28 thru 0x554485 (0x55e bytes) 000461[0] 1778535147.283050 write 0x43ec24 thru 0x442798 (0x3b75 bytes) 000462[0] 1778535147.306323 mapwrite 0x022769 thru 0x022d75 (0x60d bytes) 000464[1] 1778535147.425776 read 0x7d2e7d thru 0x7e17bf (0xe943 bytes) 000465[1] 1778535147.467055 read 0x1338eb thru 0x13dd0f (0xa425 bytes) 000466[0] 1778535147.510221 write 0x8aeff1 thru 0x8bc6ab (0xd6bb bytes) 000467[1] 1778535147.548359 mapwrite 0x0695c8 thru 0x071372 (0x7dab bytes) 000468[0] 1778535147.861971 trunc from 0x9d9fd1 to 0x410387 (0x5c9c49 bytes) 000469[0] 1778535147.933601 write 0x5ae64e thru 0x5bc029 (0xd9dc bytes) 000471[0] 1778535147.962010 mapread 0x1f204a thru 0x1f51bd (0x3174 bytes) 000472[1] 1778535147.993027 mapwrite 0x8d859c thru 0x8dfd48 (0x77ad bytes) 000473[1] 1778535148.134661 trunc from 0x8dfd49 to 0x7cdebb (0x111e8d bytes) 000478[0] 1778535148.179918 trunc from 0x7cdebc to 0x4829e0 (0x34b4db bytes) 000479[0] 1778535148.239912 read 0x3598fb thru 0x3677f5 (0xdefb bytes) 000482[0] 1778535148.297297 mapwrite 0x54775c thru 0x54ea47 (0x72ec bytes) 000484[0] 1778535148.408341 mapread 0x3c3e49 thru 0x3ce641 (0xa7f9 bytes) 000486[0] 1778535148.427865 mapwrite 0x5e6d29 thru 0x5e851c (0x17f4 bytes) 000488[0] 1778535148.546877 write 0x1167b0 thru 0x12465c (0xdead bytes) 000490[1] 1778535148.579956 write 0x00451c thru 0x00ea73 (0xa558 bytes) 000494[1] 1778535148.665420 mapwrite 0x661841 thru 0x670d95 (0xf555 bytes) 000496[1] 1778535148.894675 trunc from 0x670d96 to 0x672b1c (0x1d87 bytes) 000497[1] 1778535148.945921 read 0x215c95 thru 0x21a363 (0x46cf bytes) 000498[0] 1778535149.002998 read 0x291e27 thru 0x296184 (0x435e bytes) 000499[1] 1778535149.056827 mapread 0x666075 thru 0x66f929 (0x98b5 bytes) 000500[0] 1778535149.144921 mapread 0x2ac47a thru 0x2bb57e (0xf105 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 5149 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x5da073 000001[0] 1778535149.826592 trunc from 00000000 to 0x5da072 (0x5da073 bytes) 000004[1] 1778535149.896013 mapread 0x433000 thru 0x43df32 (0xaf33 bytes) 000007[0] 1778535149.937929 read_OD 0x59a000 thru 0x59bfff (0x2000 bytes) 000008[1] 1778535149.956518 write_OD 0x2b3000 thru 0x2b6fff (0x4000 bytes) 000009[0] 1778535150.027747 mapwrite 0x6b7000 thru 0x6c20d2 (0xb0d3 bytes) 000010[0] 1778535150.238396 mapwrite 0x774000 thru 0x77bed2 (0x7ed3 bytes) 000012[0] 1778535150.401456 mapread 0x433000 thru 0x43dd67 (0xad68 bytes) 000013[0] 1778535150.450356 mapwrite 0x39d000 thru 0x39eacb (0x1acc bytes) 000015[0] 1778535150.514371 write_OD 0x930000 thru 0x936fff (0x7000 bytes) 000016[1] 1778535150.575076 read_OD 0x433000 thru 0x43bfff (0x9000 bytes) 000018[1] 1778535150.587066 mapread 0x174000 thru 0x1782fd (0x42fe bytes) 000019[1] 1778535150.609386 mapread 0x92c000 thru 0x930a0d (0x4a0e bytes) 000020[0] 1778535150.640327 read_OD 0x388000 thru 0x391fff (0xa000 bytes) 000022[0] 1778535150.651487 trunc from 0x937000 to 0x147f1c (0x7ef0e3 bytes) 000023[1] 1778535150.721865 read_OD 0x0c5000 thru 0x0c8fff (0x4000 bytes) 000024[0] 1778535150.745325 write_OD 0x734000 thru 0x73bfff (0x8000 bytes) 000026[1] 1778535150.805750 write_OD 0x1b8000 thru 0x1c6fff (0xf000 bytes) 000028[1] 1778535150.865212 trunc from 0x73c000 to 0x1e561c (0x5569e3 bytes) truncating to largest ever: 0x677d27 000030[1] 1778535150.922334 trunc from 0x1e561d to 0x677d26 (0x49270a bytes) 000038[1] 1778535151.059808 read_OD 0x2ab000 thru 0x2b9fff (0xf000 bytes) 000040[1] 1778535151.084816 trunc from 0x677d27 to 0x4daede (0x19ce48 bytes) 000041[0] 1778535151.144424 trunc from 0x4daedf to 0x02f5ff (0x4ab8df bytes) 000042[1] 1778535151.188677 mapread 00000000 thru 0x00312f (0x3130 bytes) 000044[0] 1778535151.239176 read_OD 0x006000 thru 0x00cfff (0x7000 bytes) 000045[0] 1778535151.247933 read_OD 0x002000 thru 0x00bfff (0xa000 bytes) 000051[1] 1778535151.258257 trunc from 0x02f600 to 0x4e7281 (0x4b7c82 bytes) 000052[0] 1778535151.301091 write_OD 0x23a000 thru 0x244fff (0xb000 bytes) 000053[1] 1778535151.350647 mapwrite 0x81b000 thru 0x81eaff (0x3b00 bytes) 000054[1] 1778535151.533005 mapread 0x4c9000 thru 0x4d88c5 (0xf8c6 bytes) 000055[1] 1778535151.551205 read_OD 0x1ee000 thru 0x1effff (0x2000 bytes) 000059[1] 1778535151.577659 mapread 0x313000 thru 0x31989a (0x689b bytes) 000062[1] 1778535151.683937 trunc from 0x81eb00 to 0x66e3b1 (0x1b074e bytes) 000067[0] 1778535151.791263 read_OD 0x633000 thru 0x640fff (0xe000 bytes) 000068[1] 1778535151.825910 mapwrite 0x05d000 thru 0x05fa90 (0x2a91 bytes) 000069[1] 1778535151.965653 trunc from 0x66e3b2 to 0x4c9b9b (0x1a4816 bytes) 000071[1] 1778535152.028736 mapread 0x22a000 thru 0x231ba9 (0x7baa bytes) 000072[0] 1778535152.056031 trunc from 0x4c9b9c to 0x392e4c (0x136d4f bytes) 000074[1] 1778535152.116151 write_OD 0x45e000 thru 0x460fff (0x3000 bytes) 000083[1] 1778535152.176659 mapread 0x124000 thru 0x12af86 (0x6f87 bytes) 000085[1] 1778535152.199257 read_OD 0x0b6000 thru 0x0c0fff (0xb000 bytes) 000086[0] 1778535152.205835 mapwrite 0x9e4000 thru 0x9e6d88 (0x2d89 bytes) 000087[0] 1778535152.411397 write_OD 0x06b000 thru 0x06cfff (0x2000 bytes) truncating to largest ever: 0x81b230 000090[1] 1778535152.463070 trunc from 0x9e6d89 to 0x81b22f (0x1cbb59 bytes) 000092[1] 1778535152.505714 write_OD 0x021000 thru 0x027fff (0x7000 bytes) 000093[0] 1778535152.564179 trunc from 0x81b230 to 0x7e2b15 (0x3871a bytes) 000096[0] 1778535152.641380 mapread 0x081000 thru 0x088b6d (0x7b6e bytes) 000098[0] 1778535152.736329 trunc from 0x7e2b16 to 0x06cca1 (0x775e74 bytes) 000099[1] 1778535152.764937 read_OD 0x050000 thru 0x05afff (0xb000 bytes) 000101[0] 1778535152.793079 mapread 0x028000 thru 0x02ce15 (0x4e16 bytes) 000102[1] 1778535152.812867 write_OD 0x99a000 thru 0x9a3fff (0xa000 bytes) 000103[0] 1778535152.874168 mapread 0x324000 thru 0x3334be (0xf4bf bytes) 000104[1] 1778535153.019138 mapread 0x500000 thru 0x50f68b (0xf68c bytes) 000106[1] 1778535153.162617 mapwrite 0x628000 thru 0x62f772 (0x7773 bytes) 000107[1] 1778535153.285410 mapwrite 0x6ee000 thru 0x6fb4ba (0xd4bb bytes) 000109[1] 1778535153.475239 read_OD 0x567000 thru 0x56cfff (0x6000 bytes) 000111[0] 1778535153.493164 mapwrite 0x9f2000 thru 0x9f9964 (0x7965 bytes) 000113[1] 1778535153.696035 mapwrite 0x2d0000 thru 0x2dc83a (0xc83b bytes) 000114[1] 1778535153.988986 trunc from 0x9f9965 to 0x04446e (0x9b54f6 bytes) 000118[1] 1778535154.023974 mapwrite 0x7d7000 thru 0x7d731b (0x31c bytes) 000119[0] 1778535154.130953 trunc from 0x7d731c to 0x58cae4 (0x24a837 bytes) truncating to largest ever: 0x9a16ad 000123[0] 1778535154.220639 trunc from 0x58cae5 to 0x9a16ac (0x414bc8 bytes) 000124[0] 1778535154.251723 mapread 0x160000 thru 0x169f39 (0x9f3a bytes) 000126[0] 1778535154.329165 mapwrite 0x529000 thru 0x52b9d1 (0x29d2 bytes) 000130[0] 1778535154.414859 mapread 0x29e000 thru 0x29f801 (0x1802 bytes) 000131[1] 1778535154.432011 mapwrite 0x61f000 thru 0x62ba04 (0xca05 bytes) 000133[1] 1778535154.570287 read_OD 0x372000 thru 0x379fff (0x8000 bytes) 000134[1] 1778535154.586664 write_OD 0x3e9000 thru 0x3e9fff (0x1000 bytes) 000135[0] 1778535154.633934 read_OD 0x5d7000 thru 0x5ddfff (0x7000 bytes) 000137[1] 1778535154.656992 mapwrite 0x306000 thru 0x315237 (0xf238 bytes) 000138[1] 1778535154.963484 trunc from 0x9a16ad to 0x37e8a6 (0x622e06 bytes) 000139[0] 1778535155.010716 trunc from 0x37e8a7 to 0x030245 (0x34e661 bytes) 000141[0] 1778535155.058355 mapwrite 0x7f0000 thru 0x7f3a86 (0x3a87 bytes) 000143[1] 1778535155.234341 mapread 0x6ef000 thru 0x6fd49e (0xe49f bytes) 000144[1] 1778535155.349835 write_OD 0x090000 thru 0x099fff (0xa000 bytes) 000145[0] 1778535155.403703 mapread 0x6c5000 thru 0x6cb642 (0x6643 bytes) 000146[1] 1778535155.447279 mapread 0x0c0000 thru 0x0c5a72 (0x5a73 bytes) 000150[0] 1778535155.484919 read_OD 0x510000 thru 0x510fff (0x1000 bytes) 000151[0] 1778535155.490809 mapread 0x129000 thru 0x137048 (0xe049 bytes) 000152[0] 1778535155.629797 read_OD 0x6c3000 thru 0x6c7fff (0x5000 bytes) 000153[1] 1778535155.642414 trunc from 0x7f3a87 to 0x7045f0 (0xef496 bytes) 000154[0] 1778535155.730053 read_OD 0x004000 thru 0x00afff (0x7000 bytes) 000155[1] 1778535155.767999 write_OD 0x01a000 thru 0x024fff (0xb000 bytes) 000156[1] 1778535155.849361 mapwrite 0x399000 thru 0x39f91d (0x691e bytes) 000157[0] 1778535155.953856 mapwrite 0x9a5000 thru 0x9a6714 (0x1715 bytes) 000158[0] 1778535156.156227 mapwrite 0x558000 thru 0x562381 (0xa382 bytes) 000161[0] 1778535156.207069 trunc from 0x9a6715 to 0x1432bc (0x863458 bytes) 000162[1] 1778535156.267477 mapread 0x105000 thru 0x10cfb0 (0x7fb1 bytes) 000163[0] 1778535156.331557 write_OD 0x99a000 thru 0x99efff (0x5000 bytes) 000164[1] 1778535156.387134 read_OD 0x3ec000 thru 0x3f7fff (0xc000 bytes) 000166[1] 1778535156.398274 write_OD 0x286000 thru 0x28afff (0x5000 bytes) 000167[1] 1778535156.448269 mapread 0x554000 thru 0x5576ba (0x36bb bytes) 000168[1] 1778535156.551126 mapread 0x635000 thru 0x63688b (0x188c bytes) 000169[1] 1778535156.566606 mapwrite 0x7bf000 thru 0x7c5162 (0x6163 bytes) 000170[1] 1778535156.725267 mapread 0x05e000 thru 0x062ac5 (0x4ac6 bytes) 000172[0] 1778535156.765026 mapwrite 0x723000 thru 0x729f9a (0x6f9b bytes) 000176[1] 1778535156.912632 write_OD 0x734000 thru 0x741fff (0xe000 bytes) 000177[0] 1778535156.969274 write_OD 0x0fe000 thru 0x0fefff (0x1000 bytes) 000178[0] 1778535157.021322 trunc from 0x99f000 to 0x86abfa (0x134405 bytes) 000179[0] 1778535157.058867 mapread 0x7fe000 thru 0x80933b (0xb33c bytes) 000181[1] 1778535157.138688 trunc from 0x86abfb to 0x791cd3 (0xd8f27 bytes) 000182[0] 1778535157.170046 mapwrite 0x0fa000 thru 0x0fb826 (0x1827 bytes) 000184[0] 1778535157.277799 mapwrite 0x264000 thru 0x265a14 (0x1a15 bytes) 000185[1] 1778535157.351974 trunc from 0x791cd4 to 0x493d8d (0x2fdf46 bytes) 000187[0] 1778535157.407372 mapwrite 0x823000 thru 0x82b549 (0x854a bytes) 000188[1] 1778535157.557872 write_OD 0x073000 thru 0x081fff (0xf000 bytes) 000189[1] 1778535157.602216 trunc from 0x82b54a to 0x55ae80 (0x2d06c9 bytes) 000192[0] 1778535157.646170 mapread 0x00a000 thru 0x00fba0 (0x5ba1 bytes) 000193[1] 1778535157.704097 write_OD 0x39c000 thru 0x3a9fff (0xe000 bytes) 000194[1] 1778535157.773936 write_OD 0x3e7000 thru 0x3ebfff (0x5000 bytes) 000195[1] 1778535157.843308 write_OD 0x46c000 thru 0x46efff (0x3000 bytes) 000196[0] 1778535157.896285 read_OD 0x2ae000 thru 0x2b8fff (0xb000 bytes) 000199[1] 1778535157.911293 trunc from 0x55ae81 to 0x64c80c (0xf198c bytes) 000200[1] 1778535157.950810 mapwrite 0x141000 thru 0x14c1d8 (0xb1d9 bytes) 000201[1] 1778535158.062082 write_OD 0x68d000 thru 0x68efff (0x2000 bytes) 000205[1] 1778535158.137077 trunc from 0x68f000 to 0x2d7a86 (0x3b7579 bytes) 000206[0] 1778535158.182314 write_OD 0x003000 thru 0x00efff (0xc000 bytes) 000207[0] 1778535158.257874 mapwrite 0x271000 thru 0x27862a (0x762b bytes) 000210[0] 1778535158.427294 trunc from 0x2d7a87 to 0x7df4e6 (0x507a60 bytes) 000212[0] 1778535158.460153 mapread 0x1b5000 thru 0x1c2ada (0xdadb bytes) 000214[0] 1778535158.654845 trunc from 0x7df4e7 to 0x1da9b5 (0x604b31 bytes) 000216[0] 1778535158.713307 mapread 0x006000 thru 0x013c4a (0xdc4b bytes) 000217[1] 1778535158.794727 write_OD 0x8c6000 thru 0x8cdfff (0x8000 bytes) 000218[1] 1778535158.853919 read_OD 0x57e000 thru 0x581fff (0x4000 bytes) 000219[1] 1778535158.864740 trunc from 0x8ce000 to 0x0bbf46 (0x8120b9 bytes) 000222[1] 1778535158.929052 mapwrite 0x4c4000 thru 0x4d2529 (0xe52a bytes) 000224[0] 1778535159.122501 trunc from 0x4d252a to 0x165344 (0x36d1e5 bytes) 000227[0] 1778535159.178891 mapwrite 0x431000 thru 0x431946 (0x947 bytes) 000228[0] 1778535159.259992 write_OD 0x240000 thru 0x243fff (0x4000 bytes) 000229[1] 1778535159.314677 write_OD 0x0a3000 thru 0x0a6fff (0x4000 bytes) 000230[0] 1778535159.359493 write_OD 0x9dc000 thru 0x9e7fff (0xc000 bytes) 000233[0] 1778535159.423865 mapread 0x0a9000 thru 0x0b8fc9 (0xffca bytes) 000234[1] 1778535159.589102 mapwrite 0x3fc000 thru 0x4015ca (0x55cb bytes) 000235[1] 1778535159.788192 read_OD 0x8d9000 thru 0x8e7fff (0xf000 bytes) 000236[1] 1778535159.816231 read_OD 0x44e000 thru 0x44efff (0x1000 bytes) 000239[0] 1778535159.826975 trunc from 0x9e8000 to 0x14c340 (0x89bcbf bytes) 000241[1] 1778535159.865548 mapwrite 0x6d8000 thru 0x6e37f2 (0xb7f3 bytes) 000243[1] 1778535160.048360 trunc from 0x6e37f3 to 0x3cadf3 (0x3189ff bytes) 000244[0] 1778535160.082870 mapread 0x1d3000 thru 0x1d93ac (0x63ad bytes) 000247[0] 1778535160.162346 mapread 0x2c0000 thru 0x2c3019 (0x301a bytes) 000248[0] 1778535160.187786 mapread 0x23c000 thru 0x247240 (0xb241 bytes) 000251[1] 1778535160.253766 mapwrite 0x33a000 thru 0x33c64f (0x2650 bytes) 000252[1] 1778535160.336909 write_OD 0x509000 thru 0x515fff (0xd000 bytes) truncating to largest ever: 0x9b3936 000258[0] 1778535160.402038 trunc from 0x516000 to 0x9b3935 (0x49d936 bytes) 000259[0] 1778535160.428103 mapread 0x228000 thru 0x234d00 (0xcd01 bytes) 000264[0] 1778535160.538877 trunc from 0x9b3936 to 0x75ad8b (0x258baa bytes) 000266[1] 1778535160.559310 mapread 0x145000 thru 0x14b79f (0x67a0 bytes) 000268[1] 1778535160.618092 read_OD 0x040000 thru 0x04bfff (0xc000 bytes) 000269[1] 1778535160.628101 trunc from 0x75ad8c to 0x10d358 (0x64da33 bytes) 000270[0] 1778535160.676172 mapread 0x052000 thru 0x0568dc (0x48dd bytes) 000273[0] 1778535160.733739 write_OD 0x6de000 thru 0x6dffff (0x2000 bytes) 000275[1] 1778535160.824746 write_OD 0x586000 thru 0x586fff (0x1000 bytes) 000280[0] 1778535160.900145 mapread 0x1f1000 thru 0x1f8a28 (0x7a29 bytes) 000281[1] 1778535160.944564 trunc from 0x6e0000 to 0x3d061b (0x30f9e4 bytes) 000282[1] 1778535160.969340 trunc from 0x3d061c to 0x0f756c (0x2d90af bytes) 000283[1] 1778535160.985537 read_OD 0x0cc000 thru 0x0cffff (0x4000 bytes) 000285[0] 1778535160.994725 write_OD 0x281000 thru 0x286fff (0x6000 bytes) 000288[1] 1778535161.070560 mapwrite 0x2c6000 thru 0x2cb6c4 (0x56c5 bytes) 000289[1] 1778535161.197249 trunc from 0x2cb6c5 to 0x489f2a (0x1be866 bytes) 000292[0] 1778535161.247567 write_OD 0x625000 thru 0x632fff (0xe000 bytes) 000293[1] 1778535161.310085 mapread 0x580000 thru 0x5822bc (0x22bd bytes) 000294[1] 1778535161.324155 mapread 0x3c2000 thru 0x3cd374 (0xb375 bytes) 000298[0] 1778535161.384061 read_OD 0x46b000 thru 0x474fff (0xa000 bytes) 000301[0] 1778535161.395010 trunc from 0x633000 to 0x3c37ca (0x26f835 bytes) 000302[0] 1778535161.432598 write_OD 0x452000 thru 0x45afff (0x9000 bytes) 000303[1] 1778535161.484072 write_OD 0x845000 thru 0x845fff (0x1000 bytes) 000305[0] 1778535161.526678 mapread 0x204000 thru 0x208308 (0x4309 bytes) 000307[0] 1778535161.618687 read_OD 0x393000 thru 0x393fff (0x1000 bytes) 000312[1] 1778535161.630635 write_OD 0x989000 thru 0x98bfff (0x3000 bytes) 000315[1] 1778535161.691565 mapwrite 0x964000 thru 0x9719ea (0xd9eb bytes) 000316[1] 1778535161.771126 write_OD 0x6dc000 thru 0x6e9fff (0xe000 bytes) 000318[1] 1778535161.804258 write_OD 0x0df000 thru 0x0e4fff (0x6000 bytes) 000319[0] 1778535161.835242 trunc from 0x98c000 to 0x4fa9f3 (0x49160c bytes) 000320[1] 1778535161.853687 mapread 0x0d0000 thru 0x0dac9f (0xaca0 bytes) 000321[1] 1778535161.897585 write_OD 0x41f000 thru 0x42dfff (0xf000 bytes) 000323[0] 1778535161.925860 trunc from 0x4fa9f4 to 0x8b8b4e (0x3be15b bytes) 000324[0] 1778535161.959270 mapwrite 0x02b000 thru 0x0314ba (0x64bb bytes) 000325[1] 1778535162.149022 write_OD 0x7f4000 thru 0x7fbfff (0x8000 bytes) 000329[0] 1778535162.267220 trunc from 0x8b8b4f to 0x4f5a9c (0x3c30b2 bytes) 000330[1] 1778535162.303392 read_OD 0x063000 thru 0x064fff (0x2000 bytes) 000331[1] 1778535162.331616 mapread 0x1bb000 thru 0x1bfb5d (0x4b5e bytes) 000332[0] 1778535162.422908 trunc from 0x4f5a9d to 0x0e3eed (0x411baf bytes) 000335[1] 1778535162.485540 mapwrite 0x157000 thru 0x15cbbc (0x5bbd bytes) 000337[0] 1778535162.601407 mapwrite 0x142000 thru 0x14bc89 (0x9c8a bytes) 000338[1] 1778535162.709706 read_OD 0x0f8000 thru 0x100fff (0x9000 bytes) 000340[0] 1778535162.740117 mapwrite 0x3f5000 thru 0x3fb836 (0x6837 bytes) 000343[1] 1778535162.840666 write_OD 0x432000 thru 0x437fff (0x6000 bytes) 000344[1] 1778535162.886516 mapwrite 0x7fc000 thru 0x7fc5a8 (0x5a9 bytes) 000346[0] 1778535163.040799 mapread 0x273000 thru 0x276fd8 (0x3fd9 bytes) 000352[0] 1778535163.101650 mapwrite 0x0cd000 thru 0x0d1003 (0x4004 bytes) 000354[1] 1778535163.188808 read_OD 0x3f4000 thru 0x3fefff (0xb000 bytes) 000359[1] 1778535163.245845 mapwrite 0x68f000 thru 0x697d93 (0x8d94 bytes) 000360[1] 1778535163.401119 write_OD 0x4e4000 thru 0x4e8fff (0x5000 bytes) 000361[1] 1778535163.474593 mapread 0x47c000 thru 0x487c0a (0xbc0b bytes) 000363[0] 1778535163.552214 write_OD 0x1e9000 thru 0x1effff (0x7000 bytes) 000364[1] 1778535163.641243 read_OD 0x0e6000 thru 0x0f2fff (0xd000 bytes) 000366[0] 1778535163.658153 mapread 0x0d0000 thru 0x0dfd93 (0xfd94 bytes) 000368[1] 1778535163.683342 mapwrite 0x360000 thru 0x36144d (0x144e bytes) 000369[1] 1778535163.828908 mapread 0x1da000 thru 0x1e11bc (0x71bd bytes) 000379[0] 1778535163.856972 trunc from 0x7fc5a9 to 0x3b97ad (0x442dfb bytes) 000383[0] 1778535163.919660 write_OD 0x607000 thru 0x60afff (0x4000 bytes) 000386[1] 1778535163.956145 trunc from 0x60b000 to 0x96bc86 (0x360c87 bytes) 000388[0] 1778535164.021114 read_OD 0x7bd000 thru 0x7befff (0x2000 bytes) 000390[0] 1778535164.051599 read_OD 0x4a2000 thru 0x4a5fff (0x4000 bytes) 000391[0] 1778535164.058225 trunc from 0x96bc87 to 0x1a2329 (0x7c995d bytes) 000393[1] 1778535164.114419 read_OD 0x18d000 thru 0x193fff (0x7000 bytes) 000395[0] 1778535164.135156 read_OD 0x073000 thru 0x079fff (0x7000 bytes) 000396[1] 1778535164.145876 trunc from 0x1a232a to 0x71c507 (0x57a1de bytes) 000397[0] 1778535164.211346 mapread 0x29b000 thru 0x2a3e44 (0x8e45 bytes) 000398[1] 1778535164.322894 read_OD 0x272000 thru 0x275fff (0x4000 bytes) 000399[0] 1778535164.333100 mapread 0x4af000 thru 0x4b342a (0x442b bytes) 000400[0] 1778535164.334290 write_OD 0x2a4000 thru 0x2a7fff (0x4000 bytes) 000401[1] 1778535164.420983 trunc from 0x71c508 to 0x17d19f (0x59f368 bytes) 000402[1] 1778535164.472818 write_OD 0x636000 thru 0x638fff (0x3000 bytes) 000403[1] 1778535164.532610 mapread 0x2dd000 thru 0x2ea215 (0xd216 bytes) 000404[0] 1778535164.633475 trunc from 0x639000 to 0x73ec93 (0x105c94 bytes) 000409[0] 1778535164.743264 trunc from 0x73ec94 to 0x07d694 (0x6c15ff bytes) 000411[0] 1778535164.784416 mapread 0x001000 thru 0x00f1ce (0xe1cf bytes) 000414[0] 1778535164.886918 mapwrite 0x89e000 thru 0x8addda (0xfddb bytes) 000415[0] 1778535165.035070 read_OD 0x164000 thru 0x165fff (0x2000 bytes) 000421[1] 1778535165.046633 mapwrite 0x8e4000 thru 0x8e8333 (0x4334 bytes) 000423[0] 1778535165.162672 mapread 0x664000 thru 0x664253 (0x254 bytes) 000425[1] 1778535165.281682 mapread 0x08b000 thru 0x091eb7 (0x6eb8 bytes) 000426[0] 1778535165.303326 trunc from 0x8e8334 to 0x3105f9 (0x5d7d3a bytes) 000428[1] 1778535165.332274 mapwrite 0x7d4000 thru 0x7d8e26 (0x4e27 bytes) 000431[0] 1778535165.453411 read_OD 0x207000 thru 0x20afff (0x4000 bytes) 000432[1] 1778535165.475916 read_OD 0x2ad000 thru 0x2b1fff (0x5000 bytes) 000433[0] 1778535165.487472 write_OD 0x07e000 thru 0x080fff (0x3000 bytes) 000434[1] 1778535165.556662 trunc from 0x7d8e27 to 0x69cb67 (0x13c2bf bytes) 000435[1] 1778535165.614169 mapread 0x2dd000 thru 0x2e2702 (0x5703 bytes) 000436[1] 1778535165.680595 mapread 0x2a5000 thru 0x2aad0e (0x5d0f bytes) 000437[0] 1778535165.712063 mapwrite 0x27d000 thru 0x288bb5 (0xbbb6 bytes) 000438[0] 1778535165.833102 write_OD 0x194000 thru 0x19afff (0x7000 bytes) 000440[1] 1778535165.890512 read_OD 0x0e9000 thru 0x0e9fff (0x1000 bytes) 000441[1] 1778535165.894370 mapread 0x205000 thru 0x206dec (0x1ded bytes) 000446[0] 1778535165.977860 read_OD 0x058000 thru 0x065fff (0xe000 bytes) 000448[1] 1778535165.986241 mapwrite 0x5ad000 thru 0x5baf2e (0xdf2f bytes) 000449[0] 1778535166.204478 mapread 0x338000 thru 0x34066c (0x866d bytes) 000452[1] 1778535166.319493 trunc from 0x69cb68 to 0x96d99f (0x2d0e38 bytes) 000455[0] 1778535166.417409 mapwrite 0x5c8000 thru 0x5d0683 (0x8684 bytes) 000456[0] 1778535166.634860 mapwrite 0x39d000 thru 0x3ac7d1 (0xf7d2 bytes) 000457[0] 1778535166.788989 trunc from 0x96d9a0 to 0x7a82b2 (0x1c56ed bytes) 000459[0] 1778535166.841410 read_OD 0x043000 thru 0x046fff (0x4000 bytes) 000463[0] 1778535166.869967 write_OD 0x9bd000 thru 0x9c7fff (0xb000 bytes) 000471[0] 1778535166.934802 read_OD 0x069000 thru 0x075fff (0xd000 bytes) 000472[1] 1778535166.943675 mapwrite 0x957000 thru 0x9617da (0xa7db bytes) 000473[0] 1778535167.072236 write_OD 0x2aa000 thru 0x2acfff (0x3000 bytes) 000474[0] 1778535167.130164 mapread 0x780000 thru 0x78102b (0x102c bytes) 000475[0] 1778535167.260272 mapread 0x77d000 thru 0x78031e (0x331f bytes) 000477[0] 1778535167.269171 mapread 0x5b7000 thru 0x5c5df3 (0xedf4 bytes) 000479[0] 1778535167.270085 mapwrite 0x77f000 thru 0x78c306 (0xd307 bytes) 000480[0] 1778535167.529273 mapwrite 0x0cd000 thru 0x0da88a (0xd88b bytes) 000481[0] 1778535167.637261 trunc from 0x9c8000 to 0x945891 (0x8276e bytes) 000483[1] 1778535167.672007 read_OD 0x2be000 thru 0x2bffff (0x2000 bytes) 000490[1] 1778535167.695479 mapwrite 0x028000 thru 0x035f4b (0xdf4c bytes) 000491[1] 1778535167.950974 trunc from 0x945892 to 0x04765e (0x8fe233 bytes) 000492[1] 1778535167.997922 mapread 0x047000 thru 0x0470cf (0xd0 bytes) 000493[1] 1778535168.025326 trunc from 0x04765f to 0x1eb370 (0x1a3d12 bytes) 000494[0] 1778535168.075836 mapread 0x147000 thru 0x14c369 (0x536a bytes) 000496[1] 1778535168.137259 trunc from 0x1eb371 to 0x74c75b (0x5613eb bytes) 000497[0] 1778535168.210583 write_OD 0x88a000 thru 0x891fff (0x8000 bytes) All operations completed A-OK! PASS 16a (51s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 17:32:52 (1778535172) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.271573 s, 7.7 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.213583 s, 9.8 MB/s Chance of close/open is 1 in 50 Seed set to 5173 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[0] 1778535174.595639 mapwrite 0x93434b thru 0x9362f7 (0x1fad bytes) 000003[0] 1778535174.753209 mapread 0x03c07d thru 0x03d86b (0x17ef bytes) 000005[0] 1778535174.770248 read 0x57af25 thru 0x57cb6d (0x1c49 bytes) 000006[1] 1778535174.776341 mapwrite 0x42a5aa thru 0x42b86c (0x12c3 bytes) truncating to largest ever: 0x1ca277 000008[1] 1778535174.886294 trunc from 0x9362f8 to 0x1ca276 (0x76c081 bytes) 000009[1] 1778535174.933924 mapwrite 0x778350 thru 0x779888 (0x1539 bytes) 000010[1] 1778535175.048911 mapwrite 0x0b5bb5 thru 0x0b725d (0x16a9 bytes) 000012[0] 1778535175.119084 mapread 0x58d60a thru 0x58d6ce (0xc5 bytes) 000013[1] 1778535175.145350 mapread 0x00d41b thru 0x00e59d (0x1183 bytes) truncating to largest ever: 0x6c7919 000014[1] 1778535175.188523 trunc from 0x779889 to 0x6c7918 (0xb1f70 bytes) truncating to largest ever: 0x6ef9fa 000015[1] 1778535175.276389 trunc from 0x6c7919 to 0x6ef9f9 (0x280e1 bytes) 000017[0] 1778535175.300642 write 0x419051 thru 0x419a6b (0xa1b bytes) 000018[1] 1778535175.381656 trunc from 0x6ef9fa to 0x4ee993 (0x201066 bytes) 000019[0] 1778535175.469980 read 0x37747c thru 0x377cff (0x884 bytes) 000020[1] 1778535175.529360 write 0x5cf844 thru 0x5cfcdd (0x49a bytes) 000021[0] 1778535175.567052 mapwrite 0x9cc9d3 thru 0x9cdf18 (0x1546 bytes) 000023[1] 1778535175.814719 read 0x445da8 thru 0x4467d8 (0xa31 bytes) 000027[0] 1778535175.899933 trunc from 0x9cdf19 to 0x14ac75 (0x8832a3 bytes) 000029[0] 1778535175.950664 trunc from 0x14ac76 to 0x25dce9 (0x113074 bytes) 000030[1] 1778535175.988045 mapwrite 0x21e3e8 thru 0x21eddc (0x9f5 bytes) 000032[1] 1778535176.085351 trunc from 0x25dcea to 0x4620cb (0x2043e2 bytes) 000033[1] 1778535176.135776 mapread 0x22356d thru 0x22459f (0x1033 bytes) 000035[1] 1778535176.175585 mapwrite 0x73e334 thru 0x74000b (0x1cd8 bytes) truncating to largest ever: 0x97bceb 000036[0] 1778535176.243132 trunc from 0x74000c to 0x97bcea (0x23bcdf bytes) 000038[0] 1778535176.318157 read 0x463006 thru 0x463edb (0xed6 bytes) 000039[0] 1778535176.411261 write 0x909945 thru 0x90a7c6 (0xe82 bytes) 000041[1] 1778535176.430997 read 0x52c4e5 thru 0x52d330 (0xe4c bytes) 000042[0] 1778535176.503598 read 0x94480b thru 0x94525d (0xa53 bytes) 000043[0] 1778535176.567674 read 0x070fdb thru 0x071489 (0x4af bytes) 000046[0] 1778535176.713324 trunc from 0x97bceb to 0x7ceb19 (0x1ad1d1 bytes) 000049[1] 1778535176.812422 mapread 0x12711a thru 0x1280b8 (0xf9f bytes) 000050[0] 1778535176.848717 read 0x0891a7 thru 0x08ab8a (0x19e4 bytes) 000052[0] 1778535176.849369 mapwrite 0x362bf6 thru 0x362d0a (0x115 bytes) 000054[1] 1778535176.912481 trunc from 0x7ceb1a to 0x6f6a69 (0xd80b0 bytes) 000055[0] 1778535176.967402 mapread 0x23fa41 thru 0x241161 (0x1721 bytes) 000057[1] 1778535177.032163 write 0x70e453 thru 0x70fa71 (0x161f bytes) 000059[1] 1778535177.033596 mapread 0x462f29 thru 0x46399b (0xa73 bytes) 000060[0] 1778535177.089416 mapread 0x52d69e thru 0x52e092 (0x9f5 bytes) 000062[0] 1778535177.113704 read 0x6d1e83 thru 0x6d23c4 (0x542 bytes) 000068[0] 1778535177.124205 write 0x5c531d thru 0x5c6417 (0x10fb bytes) 000070[0] 1778535177.167995 mapread 0x55b500 thru 0x55c9fd (0x14fe bytes) 000075[0] 1778535177.213377 trunc from 0x70fa72 to 0x75fcd2 (0x50261 bytes) 000077[1] 1778535177.259148 trunc from 0x75fcd3 to 0x82a45c (0xca78a bytes) 000078[1] 1778535177.298196 mapwrite 0x107c0f thru 0x1086fd (0xaef bytes) 000080[1] 1778535177.352266 mapread 0x7313ec thru 0x733114 (0x1d29 bytes) 000082[1] 1778535177.354430 mapwrite 0x1213d6 thru 0x122f16 (0x1b41 bytes) 000084[0] 1778535177.444034 mapread 0x68f964 thru 0x690120 (0x7bd bytes) 000085[1] 1778535177.483709 mapwrite 0x2c5947 thru 0x2c6cc9 (0x1383 bytes) 000086[0] 1778535177.628962 write 0x4c80a5 thru 0x4c8f0d (0xe69 bytes) 000088[0] 1778535177.675123 mapread 0x2b6501 thru 0x2b65d1 (0xd1 bytes) 000096[0] 1778535177.805292 trunc from 0x82a45d to 0x42aed3 (0x3ff589 bytes) 000097[0] 1778535177.868110 write 0x6b1741 thru 0x6b2589 (0xe49 bytes) 000098[1] 1778535177.893254 mapread 0x100dc9 thru 0x102a71 (0x1ca9 bytes) 000099[1] 1778535177.945392 write 0x5cdb62 thru 0x5ce55d (0x9fc bytes) 000100[0] 1778535177.969658 read 0x58e7f3 thru 0x58f551 (0xd5f bytes) 000103[0] 1778535178.076209 mapwrite 0x988c41 thru 0x989da5 (0x1165 bytes) 000104[1] 1778535178.178741 write 0x148aae thru 0x1499da (0xf2d bytes) 000107[1] 1778535178.253892 read 0x5bc37f thru 0x5bd94b (0x15cd bytes) 000108[0] 1778535178.287357 write 0x6eb1e6 thru 0x6eca10 (0x182b bytes) 000109[0] 1778535178.308177 write 0x9eb200 thru 0x9eca3d (0x183e bytes) 000111[1] 1778535178.357990 read 0x475696 thru 0x475b5f (0x4ca bytes) 000112[0] 1778535178.418729 read 0x290d2b thru 0x291b6d (0xe43 bytes) 000119[1] 1778535178.599477 trunc from 0x9eca3e to 0x1256d8 (0x8c7365 bytes) 000120[0] 1778535178.705995 mapread 0x0f1cbe thru 0x0f279d (0xae0 bytes) 000121[1] 1778535178.753067 write 0x702add thru 0x704a80 (0x1fa4 bytes) 000123[1] 1778535178.810198 mapwrite 0x372bec thru 0x372f63 (0x378 bytes) 000125[0] 1778535178.905684 write 0x4516d4 thru 0x45283e (0x116b bytes) 000126[1] 1778535178.952722 mapwrite 0x9e9863 thru 0x9eb70f (0x1ead bytes) 000128[1] 1778535179.120940 write 0x26e00f thru 0x26fd7e (0x1d70 bytes) 000129[1] 1778535179.126987 read 0x28c344 thru 0x28d2ac (0xf69 bytes) 000132[0] 1778535179.209143 mapread 0x1aba9e thru 0x1ac613 (0xb76 bytes) 000133[1] 1778535179.238254 mapread 0x887055 thru 0x8877f2 (0x79e bytes) 000136[0] 1778535179.256092 write 0x07ae97 thru 0x07bb1c (0xc86 bytes) 000137[0] 1778535179.302055 read 0x127b1a thru 0x128371 (0x858 bytes) 000138[1] 1778535179.373840 mapread 0x22e5b0 thru 0x22e8a6 (0x2f7 bytes) 000139[1] 1778535179.436160 read 0x130527 thru 0x130fb5 (0xa8f bytes) 000140[1] 1778535179.541422 write 0x990549 thru 0x9911a4 (0xc5c bytes) truncating to largest ever: 0x9822d5 000141[1] 1778535179.579376 trunc from 0x9eb710 to 0x9822d4 (0x6943b bytes) 000142[1] 1778535179.619996 write 0x95f980 thru 0x95fb0e (0x18f bytes) 000143[1] 1778535179.637623 mapread 0x4cebf7 thru 0x4cfd90 (0x119a bytes) 000144[1] 1778535179.648158 read 0x3cee20 thru 0x3d0d01 (0x1ee2 bytes) 000145[1] 1778535179.702590 mapwrite 0x78b840 thru 0x78beb9 (0x67a bytes) 000147[0] 1778535179.787332 read 0x06aad9 thru 0x06ab30 (0x58 bytes) 000151[1] 1778535179.859094 trunc from 0x9822d5 to 0x27e4e7 (0x703ded bytes) 000152[0] 1778535179.938952 mapwrite 0x5f096b thru 0x5f1e78 (0x150e bytes) 000153[0] 1778535180.110944 trunc from 0x5f1e79 to 0x483aa3 (0x16e3d5 bytes) 000155[0] 1778535180.157715 mapread 0x3fcd00 thru 0x3fd573 (0x874 bytes) 000157[0] 1778535180.182503 mapwrite 0x2cf508 thru 0x2cf749 (0x242 bytes) 000158[0] 1778535180.242278 mapread 0x0a3b8e thru 0x0a3c8b (0xfe bytes) 000159[1] 1778535180.249612 write 0x22f545 thru 0x23040a (0xec6 bytes) 000161[1] 1778535180.293307 mapread 0x13584a thru 0x13689d (0x1054 bytes) 000162[0] 1778535180.325325 mapread 0x45d010 thru 0x45ee4c (0x1e3d bytes) 000163[1] 1778535180.354347 trunc from 0x483aa4 to 0x752b93 (0x2cf0f0 bytes) 000164[0] 1778535180.404145 mapwrite 0x6ff3ec thru 0x6ff747 (0x35c bytes) 000167[0] 1778535180.534348 trunc from 0x752b94 to 0x442a35 (0x31015e bytes) 000169[0] 1778535180.585608 trunc from 0x442a36 to 0x858299 (0x415864 bytes) 000170[0] 1778535180.617120 read 0x594254 thru 0x594e83 (0xc30 bytes) 000171[1] 1778535180.702371 write 0x9679b8 thru 0x968932 (0xf7b bytes) 000172[1] 1778535180.742070 read 0x278ba4 thru 0x27a50d (0x196a bytes) 000173[0] 1778535180.829819 write 0x12b68e thru 0x12c21e (0xb91 bytes) 000174[1] 1778535180.860913 mapread 0x4a6811 thru 0x4a68fa (0xea bytes) 000175[0] 1778535180.866965 write 0x259541 thru 0x259c13 (0x6d3 bytes) 000176[1] 1778535180.903789 mapwrite 0x930d66 thru 0x932a7a (0x1d15 bytes) 000177[1] 1778535180.993399 read 0x76ae69 thru 0x76bbfd (0xd95 bytes) 000178[1] 1778535181.019923 read 0x4068eb thru 0x407dea (0x1500 bytes) 000179[1] 1778535181.107732 write 0x19ca1e thru 0x19d96f (0xf52 bytes) 000182[1] 1778535181.150661 mapread 0x17306c thru 0x173d29 (0xcbe bytes) 000184[1] 1778535181.157084 trunc from 0x968933 to 0x6ea0fb (0x27e837 bytes) 000186[1] 1778535181.209859 mapread 0x4bab51 thru 0x4bab62 (0x12 bytes) 000187[0] 1778535181.213997 write 0x7912cb thru 0x791927 (0x65d bytes) 000191[0] 1778535181.297719 mapwrite 0x4b559c thru 0x4b5e63 (0x8c8 bytes) 000192[1] 1778535181.361280 trunc from 0x791928 to 0x247f51 (0x5499d6 bytes) 000193[1] 1778535181.410839 trunc from 0x247f52 to 0x759413 (0x5114c2 bytes) 000195[1] 1778535181.472187 trunc from 0x759414 to 0x23b0ae (0x51e365 bytes) 000196[0] 1778535181.513967 write 0x8e3c7d thru 0x8e5423 (0x17a7 bytes) 000198[0] 1778535181.581569 trunc from 0x8e5424 to 0x4f2a70 (0x3f29b3 bytes) 000199[1] 1778535181.625600 trunc from 0x4f2a71 to 0x5bcfb5 (0xca545 bytes) 000200[0] 1778535181.686672 write 0x6d4879 thru 0x6d65a5 (0x1d2d bytes) 000204[0] 1778535181.761650 mapread 0x35d274 thru 0x35eced (0x1a7a bytes) 000205[1] 1778535181.813308 mapread 0x4543d0 thru 0x455137 (0xd68 bytes) 000207[0] 1778535181.866242 trunc from 0x6d65a6 to 0x085154 (0x651451 bytes) 000208[0] 1778535181.928895 mapread 0x049002 thru 0x049a5d (0xa5c bytes) 000209[1] 1778535181.936900 mapwrite 0x62bc9c thru 0x62c9b5 (0xd1a bytes) truncating to largest ever: 0x9b7235 000210[0] 1778535182.062103 trunc from 0x62c9b6 to 0x9b7234 (0x38a87f bytes) 000212[0] 1778535182.130235 trunc from 0x9b7235 to 0x8449c8 (0x17286c bytes) 000213[1] 1778535182.154977 mapwrite 0x57499f thru 0x5756af (0xd11 bytes) 000214[1] 1778535182.230269 mapread 0x7c6833 thru 0x7c7f35 (0x1703 bytes) 000215[0] 1778535182.264614 read 0x49aee1 thru 0x49b21b (0x33b bytes) 000216[1] 1778535182.289701 mapread 0x0200fd thru 0x021a34 (0x1938 bytes) 000217[1] 1778535182.333086 mapwrite 0x322ac4 thru 0x323e80 (0x13bd bytes) 000218[1] 1778535182.377569 mapread 0x18c500 thru 0x18de62 (0x1963 bytes) 000221[0] 1778535182.400151 write 0x981888 thru 0x9822fa (0xa73 bytes) 000222[1] 1778535182.412858 mapwrite 0x03896d thru 0x03a420 (0x1ab4 bytes) 000223[0] 1778535182.497412 mapread 0x3af1fc thru 0x3b0bc4 (0x19c9 bytes) 000226[0] 1778535182.579299 mapread 0x024b5f thru 0x02675b (0x1bfd bytes) 000231[1] 1778535182.655319 trunc from 0x9822fb to 0x65db73 (0x324787 bytes) 000234[0] 1778535182.759845 read 0x2d0784 thru 0x2d11de (0xa5b bytes) 000235[0] 1778535182.792187 mapread 0x0bfc28 thru 0x0c17bb (0x1b94 bytes) 000236[1] 1778535182.812308 read 0x080e3b thru 0x081d13 (0xed9 bytes) 000237[0] 1778535182.925262 mapread 0x22639d thru 0x227433 (0x1097 bytes) 000240[0] 1778535182.966382 trunc from 0x65db74 to 0x1eae80 (0x472cf3 bytes) 000241[1] 1778535183.037005 trunc from 0x1eae81 to 0x22ef74 (0x440f4 bytes) 000242[0] 1778535183.093638 mapwrite 0x5c46c8 thru 0x5c610e (0x1a47 bytes) 000247[1] 1778535183.252411 read 0x41b6d3 thru 0x41bb11 (0x43f bytes) 000249[1] 1778535183.308227 trunc from 0x5c610f to 0x8b8c91 (0x2f2b83 bytes) 000250[1] 1778535183.355207 mapread 0x7e9e23 thru 0x7eac10 (0xdee bytes) truncating to largest ever: 0x9f9cee 000252[0] 1778535183.435069 trunc from 0x8b8c92 to 0x9f9ced (0x14105c bytes) 000255[1] 1778535183.607739 trunc from 0x9f9cee to 0x6a499b (0x355352 bytes) 000256[0] 1778535183.708070 trunc from 0x6a499c to 0x3e75f5 (0x2bd3a6 bytes) 000257[0] 1778535183.777624 write 0x0d61c3 thru 0x0d62d9 (0x117 bytes) 000259[1] 1778535183.831022 trunc from 0x3e75f6 to 0x544430 (0x15ce3b bytes) 000263[1] 1778535183.934161 trunc from 0x544431 to 0x26ddd9 (0x2d6657 bytes) 000264[0] 1778535183.967494 write 0x3c3440 thru 0x3c4302 (0xec3 bytes) 000265[1] 1778535183.997655 read 0x14cb18 thru 0x14d575 (0xa5e bytes) 000267[1] 1778535184.087550 trunc from 0x3c4303 to 0x5b7957 (0x1f3655 bytes) 000268[0] 1778535184.126023 read 0x2ee260 thru 0x2ef8b1 (0x1652 bytes) 000269[0] 1778535184.185871 trunc from 0x5b7958 to 0x1ed4e1 (0x3ca476 bytes) 000271[1] 1778535184.258100 mapwrite 0x3adb0b thru 0x3afa67 (0x1f5d bytes) 000273[0] 1778535184.452862 write 0x739408 thru 0x739d8b (0x984 bytes) 000274[1] 1778535184.490145 mapread 0x6d60fe thru 0x6d6841 (0x744 bytes) 000275[1] 1778535184.524574 read 0x631416 thru 0x632b75 (0x1760 bytes) 000279[0] 1778535184.595270 trunc from 0x739d8c to 0x8580cd (0x11e342 bytes) 000280[0] 1778535184.622926 read 0x6c15dc thru 0x6c32df (0x1d04 bytes) 000281[1] 1778535184.666961 mapread 0x1e68e8 thru 0x1e8733 (0x1e4c bytes) 000282[1] 1778535184.724014 mapread 0x342d16 thru 0x3442f7 (0x15e2 bytes) 000283[0] 1778535184.754094 trunc from 0x8580ce to 0x5b1baf (0x2a651e bytes) 000285[1] 1778535184.789696 trunc from 0x5b1bb0 to 0x7d168d (0x21fade bytes) 000286[0] 1778535184.832960 write 0x5e2741 thru 0x5e3e3c (0x16fc bytes) 000287[1] 1778535184.867376 read 0x7a9feb thru 0x7abaaf (0x1ac5 bytes) 000288[1] 1778535184.912126 read 0x34a620 thru 0x34a879 (0x25a bytes) 000289[0] 1778535185.018794 mapread 0x384cc4 thru 0x38663c (0x1979 bytes) 000291[1] 1778535185.070873 trunc from 0x7d168e to 0x3196eb (0x4b7fa2 bytes) 000292[1] 1778535185.130382 write 0x7cb7ee thru 0x7cce7f (0x1692 bytes) 000294[0] 1778535185.136114 mapread 0x7184c1 thru 0x71a0c2 (0x1c02 bytes) 000295[0] 1778535185.199927 trunc from 0x7cce80 to 0x027ff2 (0x7a4e8d bytes) 000296[0] 1778535185.281334 trunc from 0x027ff3 to 0x12f0f5 (0x107103 bytes) 000297[0] 1778535185.321186 mapwrite 0x8ccd84 thru 0x8cd834 (0xab1 bytes) 000300[1] 1778535185.482942 mapwrite 0x4ff7ec thru 0x5012c2 (0x1ad7 bytes) 000304[0] 1778535185.683281 mapread 0x514b50 thru 0x51685e (0x1d0f bytes) 000306[1] 1778535185.779045 write 0x9778f9 thru 0x979430 (0x1b38 bytes) 000308[1] 1778535185.838061 trunc from 0x979431 to 0x7830a7 (0x1f6389 bytes) 000309[0] 1778535185.888601 mapread 0x489a28 thru 0x48b76b (0x1d44 bytes) 000310[0] 1778535185.967802 write 0x54aa67 thru 0x54c976 (0x1f10 bytes) 000313[0] 1778535186.080890 write 0x8a7a77 thru 0x8a88b6 (0xe40 bytes) 000315[1] 1778535186.096641 mapwrite 0x589011 thru 0x58978b (0x77b bytes) 000316[1] 1778535186.239120 read 0x6c921d thru 0x6c98b4 (0x698 bytes) 000317[0] 1778535186.310102 read 0x75ecf8 thru 0x760afd (0x1e06 bytes) 000318[1] 1778535186.350758 write 0x0e1b2f thru 0x0e2b6f (0x1041 bytes) 000322[1] 1778535186.430221 mapwrite 0x8604ab thru 0x862394 (0x1eea bytes) 000324[0] 1778535186.563660 write 0x329ab0 thru 0x32b9d5 (0x1f26 bytes) 000325[1] 1778535186.647037 mapread 0x39c661 thru 0x39ded5 (0x1875 bytes) 000326[0] 1778535186.687304 read 0x49f348 thru 0x4a0162 (0xe1b bytes) 000330[0] 1778535186.799047 mapread 0x4fd979 thru 0x4fe6df (0xd67 bytes) 000334[1] 1778535186.800502 trunc from 0x8a88b7 to 0x7a47f0 (0x1040c6 bytes) 000335[1] 1778535186.904413 mapwrite 0x0df6a6 thru 0x0e1637 (0x1f92 bytes) 000336[0] 1778535187.016680 write 0x9f43dd thru 0x9f5938 (0x155c bytes) 000337[0] 1778535187.054889 write 0x5046da thru 0x504a36 (0x35d bytes) 000339[0] 1778535187.099105 mapwrite 0x6d3270 thru 0x6d4f24 (0x1cb5 bytes) 000343[1] 1778535187.234078 mapwrite 0x81df0a thru 0x81e1f8 (0x2ef bytes) 000347[0] 1778535187.436518 trunc from 0x9f5939 to 0x8e7315 (0x10e623 bytes) 000349[0] 1778535187.512431 read 0x63b63b thru 0x63cfe7 (0x19ad bytes) 000350[0] 1778535187.607705 trunc from 0x8e7316 to 0x6c82bf (0x21f056 bytes) 000351[1] 1778535187.638630 mapwrite 0x643a29 thru 0x6451e1 (0x17b9 bytes) 000354[0] 1778535187.807044 write 0x005128 thru 0x006cec (0x1bc5 bytes) 000355[0] 1778535187.858371 mapwrite 0x47318b thru 0x473694 (0x50a bytes) 000358[1] 1778535187.965544 mapread 0x639a19 thru 0x639dc7 (0x3af bytes) 000360[0] 1778535188.027022 trunc from 0x6c82c0 to 0x5c929e (0xff021 bytes) 000361[1] 1778535188.066249 write 0x8e580f thru 0x8e5898 (0x8a bytes) 000362[1] 1778535188.130709 trunc from 0x8e5899 to 0x0216d0 (0x8c41c8 bytes) 000363[0] 1778535188.198602 write 0x9ff9b7 thru 0x9fffff (0x649 bytes) 000365[1] 1778535188.272047 trunc from 0xa00000 to 0x05b6b5 (0x9a494a bytes) 000367[1] 1778535188.335114 trunc from 0x05b6b6 to 0x113fcf (0xb891a bytes) 000369[1] 1778535188.356233 write 0x2ff973 thru 0x2ffff7 (0x685 bytes) 000370[1] 1778535188.359002 write 0x366804 thru 0x36837a (0x1b77 bytes) 000371[0] 1778535188.383143 write 0x549aa6 thru 0x54b4ef (0x1a4a bytes) 000372[0] 1778535188.418598 mapwrite 0x690e3b thru 0x6913c1 (0x587 bytes) 000373[0] 1778535188.565407 trunc from 0x6913c2 to 0x7adec6 (0x11cb05 bytes) 000374[0] 1778535188.670713 write 0x2462ea thru 0x2479b8 (0x16cf bytes) 000379[0] 1778535188.728092 trunc from 0x7adec7 to 0x2562d4 (0x557bf2 bytes) 000380[1] 1778535188.774942 trunc from 0x2562d5 to 0x8a5dbb (0x64fae7 bytes) 000382[0] 1778535188.913989 write 0x88835d thru 0x8898fd (0x15a1 bytes) 000385[0] 1778535189.042279 mapread 0x2a0026 thru 0x2a0a9a (0xa75 bytes) 000387[1] 1778535189.116227 trunc from 0x8a5dbc to 0x4355cf (0x4707ec bytes) 000389[1] 1778535189.146075 mapwrite 0x08b74f thru 0x08ca81 (0x1333 bytes) 000390[1] 1778535189.192389 write 0x13ed63 thru 0x14062b (0x18c9 bytes) 000391[1] 1778535189.200853 write 0x994f35 thru 0x995712 (0x7de bytes) 000392[0] 1778535189.203105 trunc from 0x995713 to 0x5e757d (0x3ae195 bytes) 000393[1] 1778535189.273260 mapread 0x1438e6 thru 0x143afa (0x215 bytes) 000394[1] 1778535189.312504 read 0x02f4de thru 0x030eee (0x1a11 bytes) 000395[0] 1778535189.439634 read 0x52a256 thru 0x52b121 (0xecc bytes) 000398[1] 1778535189.464037 mapread 0x06b93c thru 0x06d851 (0x1f16 bytes) 000399[1] 1778535189.465283 mapwrite 0x021800 thru 0x0229e7 (0x11e8 bytes) 000400[1] 1778535189.567636 mapread 0x517a56 thru 0x519459 (0x1a04 bytes) 000404[0] 1778535189.588144 mapread 0x3dbf18 thru 0x3dccb2 (0xd9b bytes) 000406[1] 1778535189.633235 mapread 0x168988 thru 0x168e8d (0x506 bytes) 000407[0] 1778535189.645732 mapread 0x06b0f6 thru 0x06b7d0 (0x6db bytes) 000408[1] 1778535189.668416 mapread 0x4c8bf2 thru 0x4ca38d (0x179c bytes) 000410[0] 1778535189.688517 trunc from 0x5e757e to 0x247ea5 (0x39f6d8 bytes) 000413[1] 1778535189.772116 mapread 0x11f15a thru 0x11f24a (0xf1 bytes) 000414[0] 1778535189.854974 write 0x20680c thru 0x20715b (0x950 bytes) 000415[0] 1778535189.893374 write 0x99cbb3 thru 0x99e380 (0x17ce bytes) 000417[0] 1778535189.914823 read 0x1387de thru 0x1389d2 (0x1f5 bytes) 000418[0] 1778535190.040420 mapread 0x755c08 thru 0x7578c9 (0x1cc2 bytes) 000419[0] 1778535190.044726 read 0x2e9bdd thru 0x2eaf68 (0x138c bytes) 000420[1] 1778535190.069898 mapwrite 0x0e8511 thru 0x0e8820 (0x310 bytes) 000421[1] 1778535190.253179 read 0x482606 thru 0x482f0b (0x906 bytes) 000422[0] 1778535190.291253 read 0x1990df thru 0x19a31a (0x123c bytes) 000423[1] 1778535190.322634 write 0x5e01f6 thru 0x5e02b8 (0xc3 bytes) 000424[0] 1778535190.433818 trunc from 0x99e381 to 0x4e6d27 (0x4b7659 bytes) 000425[0] 1778535190.503083 read 0x244101 thru 0x2442cb (0x1cb bytes) 000426[1] 1778535190.572639 mapread 0x064ff7 thru 0x066509 (0x1513 bytes) 000427[0] 1778535190.659759 write 0x054164 thru 0x054398 (0x235 bytes) 000428[1] 1778535190.728851 mapwrite 0x5d80ee thru 0x5d9bf1 (0x1b04 bytes) 000429[0] 1778535190.887299 mapwrite 0x73e504 thru 0x73f7a2 (0x129f bytes) 000430[0] 1778535191.058249 read 0x205739 thru 0x20604b (0x913 bytes) 000431[1] 1778535191.104757 trunc from 0x73f7a3 to 0x897d1c (0x15857a bytes) 000432[1] 1778535191.200718 mapread 0x2fc8a7 thru 0x2fe6e4 (0x1e3e bytes) 000435[1] 1778535191.268633 write 0x61d7af thru 0x61e05c (0x8ae bytes) 000436[1] 1778535191.282356 read 0x516217 thru 0x5181e4 (0x1fce bytes) 000438[0] 1778535191.334640 mapread 0x00caab thru 0x00e193 (0x16e9 bytes) 000439[0] 1778535191.441994 mapwrite 0x614702 thru 0x61500e (0x90d bytes) 000440[0] 1778535191.527633 trunc from 0x897d1d to 0x594e30 (0x302eec bytes) 000441[1] 1778535191.555334 mapread 0x20d2a5 thru 0x20e118 (0xe74 bytes) 000442[1] 1778535191.602410 read 0x23a090 thru 0x23af2c (0xe9d bytes) 000443[1] 1778535191.654244 mapread 0x1eac7b thru 0x1eb994 (0xd1a bytes) 000444[1] 1778535191.773576 mapwrite 0x5c5ffd thru 0x5c67f8 (0x7fc bytes) 000446[1] 1778535191.914585 read 0x204928 thru 0x205bef (0x12c8 bytes) 000447[0] 1778535191.985759 write 0x05e3ae thru 0x05eb0b (0x75e bytes) 000448[1] 1778535192.026213 read 0x1b17fc thru 0x1b2c1d (0x1422 bytes) 000449[0] 1778535192.062817 read 0x01415c thru 0x0150a4 (0xf49 bytes) 000451[0] 1778535192.114837 mapwrite 0x83851c thru 0x839968 (0x144d bytes) 000452[0] 1778535192.369188 trunc from 0x839969 to 0x121862 (0x718106 bytes) 000453[0] 1778535192.423948 write 0x52576b thru 0x5273a9 (0x1c3f bytes) 000455[1] 1778535192.485181 read 0x12c386 thru 0x12d64a (0x12c5 bytes) 000457[1] 1778535192.583523 read 0x14b2ed thru 0x14bb39 (0x84d bytes) 000458[1] 1778535192.621911 mapread 0x17426a thru 0x174dc8 (0xb5f bytes) 000462[1] 1778535192.646532 mapwrite 0x55909e thru 0x5595f3 (0x556 bytes) 000463[1] 1778535192.781288 mapwrite 0x929475 thru 0x92acc3 (0x184f bytes) 000465[1] 1778535192.928527 trunc from 0x92acc4 to 0x82c0a4 (0xfec1f bytes) 000466[0] 1778535192.950280 write 0x50a437 thru 0x50bef0 (0x1aba bytes) 000469[0] 1778535193.006382 mapread 0x3e5a38 thru 0x3e6740 (0xd09 bytes) 000470[0] 1778535193.061972 mapread 0x823d03 thru 0x82542c (0x172a bytes) 000471[1] 1778535193.120957 trunc from 0x82c0a5 to 0x646da4 (0x1e5300 bytes) 000472[1] 1778535193.215564 write 0x7992ad thru 0x799bea (0x93e bytes) 000474[0] 1778535193.256328 mapwrite 0x0b8ed1 thru 0x0b9a52 (0xb82 bytes) 000475[0] 1778535193.355402 trunc from 0x799beb to 0x652eef (0x146cfb bytes) 000476[1] 1778535193.408475 mapwrite 0x92f6ba thru 0x9302e0 (0xc27 bytes) 000477[0] 1778535193.507003 trunc from 0x9302e1 to 0x79b3d2 (0x194f0e bytes) 000479[1] 1778535193.586462 mapread 0x2fde31 thru 0x2ff3db (0x15ab bytes) 000480[0] 1778535193.642355 mapwrite 0x7069ae thru 0x707f08 (0x155b bytes) 000481[0] 1778535193.718792 mapwrite 0x38c5b6 thru 0x38cb85 (0x5d0 bytes) 000482[0] 1778535193.792808 read 0x517eb0 thru 0x5198f0 (0x1a41 bytes) 000483[0] 1778535193.857835 mapread 0x6c1d25 thru 0x6c23c8 (0x6a4 bytes) 000485[0] 1778535193.877695 read 0x332e93 thru 0x332ede (0x4c bytes) 000487[1] 1778535193.920187 trunc from 0x79b3d3 to 0x1de244 (0x5bd18e bytes) 000488[0] 1778535193.985853 write 0x2aab9e thru 0x2ac9b8 (0x1e1b bytes) 000489[1] 1778535194.021017 trunc from 0x2ac9b9 to 0x7a6ecc (0x4fa514 bytes) 000490[0] 1778535194.044047 mapread 0x3cc0ef thru 0x3cd5ee (0x1500 bytes) 000491[0] 1778535194.086549 mapwrite 0x17adec thru 0x17af1a (0x12f bytes) 000492[1] 1778535194.169623 write 0x6e70dc thru 0x6e7f12 (0xe37 bytes) 000500[1] 1778535194.400738 write 0x8575c8 thru 0x85898a (0x13c3 bytes) All operations completed A-OK! PASS 16b (26s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 17:33:18 (1778535198) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (3s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 17:33:21 (1778535201) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.314054 s, 6.7 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.19123 s, 11.0 MB/s ldlm.namespaces.MGC192.168.202.136@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff986444f2c000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff9864494bb000.lru_size=clear ldlm.namespaces.lustre-MDT0001-mdc-ffff986444f2c000.lru_size=clear ldlm.namespaces.lustre-MDT0001-mdc-ffff9864494bb000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff986444f2c000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff9864494bb000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff986444f2c000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff9864494bb000.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 7.05173 s, 14.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.14002 s, 49.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.82717 s, 57.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.25186 s, 24.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.31866 s, 19.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.43358 s, 43.1 MB/s PASS 16d (33s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 17:33:54 (1778535234) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.190889 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0893456 s, 11.7 MB/s PASS 16e (5s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 17:33:59 (1778535239) TIMEOUT PASS 16f (24s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 17:34:24 (1778535264) TIMEOUT PASS 16g (25s) == sanityn test 16i: read after truncate file ============ 17:34:49 (1778535289) striped dir -i0 -c2 -H all_char /mnt/lustre/d16i.sanityn ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (5s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 17:34:54 (1778535294) fail_loc=0x8000030a PASS 17 (7s) == sanityn test 18: mmap sanity check =========================================================================================== 17:35:01 (1778535301) mmap test1: basic mmap operation (PASS, 0.068045s) mmap test2: MAP_PRIVATE not write back (PASS, 0.039695s) mmap test3: concurrent mmap ops on two nodes (PASS, 12.534s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.5923s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.10203s) mmap test6: check mmap write/read content on two nodes (PASS, 0.097603s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.054s) PASS 18 (23s) == sanityn test 19: test concurrent uncached read races ========================================================================= 17:35:25 (1778535325) oleg236-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg236-client: oleg236-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (4s) == sanityn test 20: test extra readahead page left in cache ============================================================== 17:35:28 (1778535328) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d20.sanityn PASS 20 (5s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 17:35:33 (1778535333) striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d21.sanityn rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (5s) == sanityn test 23: others should see updated atime while another read============================================================== 17:35:38 (1778535338) now is 1778535340 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 new atime is 1778535401 PASS 23 (67s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 17:36:45 (1778535405) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 12288 7526400 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 12288 7526400 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 483.8K 383 483.4K 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 484.9K 336 484.6K 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 115.3K 439 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.4K 441 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.5K 719 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 483.8K 383 483.4K 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 484.9K 336 484.6K 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 115.3K 439 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.4K 441 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.5K 719 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.6M 2.1G 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3.6G 7.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 5.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 12.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 495416 383 495033 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 496533 336 496197 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 118071 439 117632 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118137 441 117696 1% /mnt/lustre2[OST:1] filesystem_summary: 236047 719 235328 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 12288 7526400 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 483.8K 383 483.4K 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 484.9K 336 484.6K 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 115.3K 439 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.4K 441 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.5K 719 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 495416 383 495033 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 496533 336 496197 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 118071 439 117632 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118137 441 117696 1% /mnt/lustre[OST:1] filesystem_summary: 236047 719 235328 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 495416 383 495033 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 496533 336 496197 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 118071 439 117632 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118137 441 117696 1% /mnt/lustre2[OST:1] filesystem_summary: 236047 719 235328 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 12288 7526400 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 12288 7526400 1% /mnt/lustre2 PASS 24a (6s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 17:36:51 (1778535411) PASS 24b (5s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 17:36:56 (1778535416) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (5s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 17:37:01 (1778535421) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] Can't lstat /mnt/lustre2/d25b.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] Can't lstat /mnt/lustre2/d25b.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25b.sanityn/f1] PASS 25b (5s) == sanityn test 26a: allow mtime to get older ============ 17:37:06 (1778535426) utime: good mknod times 1778535426 <= 1778535427 <= 1778535427 for /mnt/lustre/f26a utime: good mknod times 1778535426 <= 1778535427 <= 1778535427 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (5s) == sanityn test 26b: sync mtime between ost and mds ====== 17:37:11 (1778535431) PASS 26b (7s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 17:37:20 (1778535440) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d30.sanityn 0 PASS 30 (7s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 17:37:27 (1778535447) striped dir -i1 -c2 -H all_char /mnt/lustre/d31a.sanityn fail_loc=0x314 PASS 31a (7s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 17:37:34 (1778535454) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg236-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete striped dir -i1 -c2 -H crush /mnt/lustre/d31b.sanityn fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.202.136@tcp:/lustre 7542784 13312 7521280 1% /mnt/lustre2 PASS 31b (18s) == sanityn test 31r: open-rename(replace) race =========== 17:37:53 (1778535473) -rw-r--r-- 1 root root 0 May 11 17:37 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (7s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 17:38:01 (1778535481) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (2s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 17:38:04 (1778535484) Stopping /mnt/lustre-mds1 (opts:) on oleg236-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg236-server: oleg236-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 pdsh@oleg236-client: oleg236-server: ssh exited with exit code 1 Started lustre-MDT0000 sync_count 2 sync_count 0 PASS 33c (30s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 17:38:34 (1778535514) CoS count 2 CoS count 2 CoS count 2 CoS count 2 CoS count 2 CoS count 4 CoS count 3 CoS count 4 PASS 33d (94s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 17:40:08 (1778535608) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (2s) == sanityn test 34: no lock timeout under IO ============= 17:40:10 (1778535610) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg236-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff986444f2c000.ost_server_uuid,osc.lustre-OST0000-osc-ffff9864494bb000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff986444f2c000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff9864494bb000.ost_server_uuid in IDLE state after 0 sec oleg236-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff986444f2c000.ost_server_uuid,osc.lustre-OST0001-osc-ffff9864494bb000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff986444f2c000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff9864494bb000.ost_server_uuid in IDLE state after 0 sec oleg236-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff986444f2c000.ost_server_uuid,osc.lustre-OST0000-osc-ffff9864494bb000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff986444f2c000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff9864494bb000.ost_server_uuid in IDLE state after 0 sec oleg236-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff986444f2c000.ost_server_uuid,osc.lustre-OST0001-osc-ffff9864494bb000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff986444f2c000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff9864494bb000.ost_server_uuid in IDLE state after 0 sec oleg236-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff986444f2c000.ost_server_uuid,osc.lustre-OST0000-osc-ffff9864494bb000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff986444f2c000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff9864494bb000.ost_server_uuid in IDLE state after 0 sec oleg236-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff986444f2c000.ost_server_uuid,osc.lustre-OST0001-osc-ffff9864494bb000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff986444f2c000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff9864494bb000.ost_server_uuid in IDLE state after 0 sec PASS 34 (105s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 17:41:56 (1778535716) striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d35.sanityn fail_loc=0x80000317 Race attempt 0 Wait for 52294 52376 for 60 sec... - open/close 991 (time 1778535730.32 total 10.00 last 99.07) - open/close 1950 (time 1778535740.32 total 20.01 last 95.83) - open/close 2955 (time 1778535750.33 total 30.02 last 100.45) total: 4000 open/close in 39.55 seconds: 101.13 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.202.136@tcp:/lustre 7.2G 13M 7.2G 1% /mnt/lustre 192.168.202.136@tcp:/lustre 7.2G 13M 7.2G 1% /mnt/lustre2 PASS 35 (67s) == sanityn test 36: handle ESTALE/open-unlink correctly == 17:43:03 (1778535783) striped dir -i0 -c2 -H all_char /mnt/lustre/d36.sanityn Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg236-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.935453 s, 56.0 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.798494 s, 65.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7525376) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.640027 s, 81.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.763351 s, 68.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7523328) after_dd(7470080) after(7521280) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.769532 s, 68.1 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7521280) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.517871 s, 101 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.641176 s, 81.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.451709 s, 116 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7523328) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.541015 s, 96.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.459765 s, 114 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.550682 s, 95.2 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7525376) after_dd(7470080) after(7521280) PASS 36 (274s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 17:47:37 (1778536057) striped dir -i1 -c2 -H all_char /mnt/lustre/d37.sanityn multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 - create 2568 (time 1778536069.04 total 10.00 last 256.78) - create 5122 (time 1778536079.04 total 20.00 last 255.39) - create 7333 (time 1778536089.04 total 30.00 last 221.08) - create 9959 (time 1778536099.04 total 40.00 last 262.53) total: 10000 create in 40.16 seconds: 249.01 ops/second PASS 37 (45s) == sanityn test 39a: test from 11063 ============================================================================================ 17:48:22 (1778536102) repeat after cancel_lru_locks PASS 39a (3s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 17:48:25 (1778536105) repeat after cancel_lru_locks PASS 39b (5s) == sanityn test 39c: check truncate mtime update ================================================================================ 17:48:30 (1778536110) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 17:48:33 (1778536113) fail_loc=0x411 fail_loc=0 PASS 39d (3s) == sanityn test 40a: pdirops: create vs others ======================================================================== 17:48:36 (1778536116) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (11s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 17:48:47 (1778536127) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (10s) == sanityn test 40c: pdirops: link and others ======================================================================== 17:48:57 (1778536137) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (10s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 17:49:07 (1778536147) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (10s) == sanityn test 40e: pdirops: rename and others ======================================================================== 17:49:17 (1778536157) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (9s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 17:49:26 (1778536166) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (7s) == sanityn test 41b: pdirops: create vs create ======================================================================== 17:49:33 (1778536173) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 open: File exists Conflict PASS 41b (6s) == sanityn test 41c: pdirops: create vs link ======================================================================== 17:49:39 (1778536179) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=4 link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (6s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 17:49:45 (1778536185) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41d (7s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 17:49:52 (1778536192) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41e (6s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 17:49:58 (1778536198) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41f (6s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 17:50:04 (1778536204) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 41g (6s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 17:50:10 (1778536210) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=4 Conflict PASS 41h (6s) == sanityn test 41i: reint_open: create vs create ======== 17:50:16 (1778536216) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (843s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 18:04:19 (1778537059) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (8s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 18:04:28 (1778537068) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 open: File exists Conflict PASS 42b (8s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 18:04:36 (1778537076) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (7s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 18:04:43 (1778537083) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 Conflict PASS 42d (7s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 18:04:50 (1778537090) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (8s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 18:04:58 (1778537098) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 42f (8s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 18:05:06 (1778537106) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 Conflict PASS 42g (7s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 18:05:13 (1778537113) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 42h (8s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 18:05:21 (1778537121) PASS 43a (85s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 18:06:46 (1778537206) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 Conflict PASS 43b (7s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 18:06:53 (1778537213) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43c (7s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 18:07:00 (1778537220) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (8s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 18:07:08 (1778537228) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43e (7s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 18:07:15 (1778537235) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (7s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 18:07:22 (1778537242) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (7s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 18:07:29 (1778537249) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43h (7s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 18:07:36 (1778537256) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 43i (7s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 18:07:43 (1778537263) mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (67s) == sanityn test 43k: unlink vs create ==================== 18:08:50 (1778537330) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (608s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 18:18:58 (1778537938) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=4 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=2 mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Mon May 11 18:19:02 EDT 2026 PASS 44a (6s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 18:19:05 (1778537945) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 open: File exists Conflict PASS 44b (7s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 18:19:11 (1778537951) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (7s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 18:19:18 (1778537958) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44d (7s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 18:19:25 (1778537965) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44e (7s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 18:19:32 (1778537972) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44f (6s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 18:19:38 (1778537978) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=10 Conflict PASS 44g (7s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 18:19:45 (1778537985) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 44h (7s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 18:19:52 (1778537992) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 lfs mkdir: dirstripe error on '/mnt/lustre2/f44i.sanityn': stripe already set lfs setdirstripe: cannot create dir '/mnt/lustre2/f44i.sanityn': File exists Conflict PASS 44i (6s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 18:19:58 (1778537998) PASS 45a (61s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 18:20:59 (1778538059) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 Conflict PASS 45b (6s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 18:21:05 (1778538065) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=9 Conflict PASS 45c (5s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 18:21:10 (1778538070) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=12 rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (6s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 18:21:16 (1778538076) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=6 Conflict PASS 45e (5s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 18:21:22 (1778538082) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (5s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 18:21:27 (1778538087) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (6s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 18:21:33 (1778538093) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45h (5s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 18:21:38 (1778538098) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 45i (5s) == sanityn test 45j: read vs rename ====================== 18:21:44 (1778538104) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (497s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 18:30:00 (1778538600) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=5 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=3 mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (5s) == sanityn test 46b: pdirops: link vs create ======================================================================== 18:30:05 (1778538605) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=9 open: File exists Conflict PASS 46b (5s) == sanityn test 46c: pdirops: link vs link ======================================================================== 18:30:10 (1778538610) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=7 link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (6s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 18:30:16 (1778538616) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 46d (5s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 18:30:21 (1778538621) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 46e (5s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 18:30:26 (1778538626) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 46f (6s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 18:30:32 (1778538632) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 46g (5s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 18:30:37 (1778538637) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 46h (5s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 18:30:42 (1778538642) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 lfs mkdir: dirstripe error on '/mnt/lustre2/f46i.sanityn': stripe already set lfs setdirstripe: cannot create dir '/mnt/lustre2/f46i.sanityn': File exists Conflict PASS 46i (6s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 18:30:48 (1778538648) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 mkdir: cannot create directory '/mnt/lustre2/f47a.sanityn': File exists Conflict PASS 47a (5s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 18:30:53 (1778538653) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=3 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 open: File exists Conflict PASS 47b (6s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 18:30:59 (1778538659) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 link: cannot create link '/mnt/lustre2/f47c.sanityn' to '/mnt/lustre2/f47c.sanityn-2': File exists Conflict PASS 47c (6s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 18:31:05 (1778538665) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=4 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 Conflict PASS 47d (5s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 18:31:10 (1778538670) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=2 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 mv: cannot overwrite directory '/mnt/lustre2/f47e.sanityn' with non-directory Conflict PASS 47e (5s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 18:31:15 (1778538675) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=4 ldlm.namespaces.mdt-lustre-MDT0001_UUID.lock_count=1 Conflict PASS 47f (5s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 18:31:21 (1778538681) ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=1 Conflict PASS 47g (6s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 18:31:27 (1778538687) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.0277141 s, 369 kB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 18:31:34 (1778538694) 0+1 records in 0+1 records out 158 bytes copied, 0.0036514 s, 43.3 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (263747) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 18:31:38 (1778538698) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00275434 s, 372 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015368 s, 68.2 MB/s 3145728 PASS 51b (15s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 18:31:53 (1778538713) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00410494 s, 249 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 18:31:59 (1778538719) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156007 s, 67.2 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (4s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 18:32:03 (1778538723) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 1248 0x4e0 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 1248 0x4e0 0 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 18:32:07 (1778538727) llite.lustre-ffff986444f2c000.xattr_cache=1 llite.lustre-ffff9864494bb000.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205289281031 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:32:07.000000000 -0400 Modify: 2026-05-11 18:32:07.000000000 -0400 Change: 2026-05-11 18:32:07.000000000 -0400 Birth: 2026-05-11 18:32:08.000000000 -0400 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 162129603798762684 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:32:13.000000000 -0400 Modify: 2026-05-11 18:32:13.000000000 -0400 Change: 2026-05-11 18:32:13.000000000 -0400 Birth: 2026-05-11 18:32:14.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205289281034 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:32:18.000000000 -0400 Modify: 2026-05-11 18:32:18.000000000 -0400 Change: 2026-05-11 18:32:18.000000000 -0400 Birth: 2026-05-11 18:32:19.000000000 -0400 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 162129603798762685 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:32:18.000000000 -0400 Modify: 2026-05-11 18:32:19.000000000 -0400 Change: 2026-05-11 18:32:19.000000000 -0400 Birth: 2026-05-11 18:32:19.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205289281035 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:32:24.000000000 -0400 Modify: 2026-05-11 18:32:24.000000000 -0400 Change: 2026-05-11 18:32:24.000000000 -0400 Birth: 2026-05-11 18:32:25.000000000 -0400 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 162129603798762687 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:32:24.000000000 -0400 Modify: 2026-05-11 18:32:24.000000000 -0400 Change: 2026-05-11 18:32:24.000000000 -0400 Birth: 2026-05-11 18:32:25.000000000 -0400 PASS 54 (24s) == sanityn test 55a: rename vs unlink target dir ========= 18:32:31 (1778538751) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (8s) == sanityn test 55b: rename vs unlink source dir ========= 18:32:39 (1778538759) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 18:32:46 (1778538766) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (13s) == sanityn test 55d: rename file vs link ================= 18:32:59 (1778538779) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (9s) == sanityn test 60: Verify data_version behaviour ======== 18:33:08 (1778538788) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d60.sanityn PASS 60 (3s) == sanityn test 70a: cd directory && rm directory ======== 18:33:11 (1778538791) PASS 70a (2s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 18:33:13 (1778538793) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 18:33:15 (1778538795) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00205288 s, 31.9 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 18:33:16 (1778538796) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00266293 s, 24.6 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 18:33:17 (1778538797) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 18:33:18 (1778538798) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00738863 s, 8.9 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 18:33:19 (1778538799) llite.lustre-ffff986444f2c000.xattr_cache=1 llite.lustre-ffff9864494bb000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 18:33:21 (1778538801) llite.lustre-ffff986444f2c000.xattr_cache=1 llite.lustre-ffff9864494bb000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (2s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 18:33:23 (1778538803) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00249094 s, 411 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00306569 s, 334 kB/s 273921: get lock2 273925: get lock1 273921: done 273921: get lock1 273925: done 273925: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 273925: cannot get lock2: Resource deadlock avoided 273925: exit rc=0 273921: done 273921: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 18:33:29 (1778538809) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0156273 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119714 s, 87.6 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 274505 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 18:33:36 (1778538816) striped dir -i0 -c2 -H crush /mnt/lustre/d76.sanityn opening files: ................ closing files: ................ PASS 76 (40s) == sanityn test 77a: check FIFO NRS policy =============== 18:34:16 (1778538856) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.11408 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222442 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296434 s, 35.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0689715 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0821629 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0922307 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0876358 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0787526 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0799666 s, 13.1 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff9864494bb000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff9864494bb000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.094565 s, 11.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0909523 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102181 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0964548 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11193 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112849 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109807 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125898 s, 8.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135455 s, 77.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146132 s, 71.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260589 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279129 s, 37.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248069 s, 42.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217701 s, 48.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293352 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.036334 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216714 s, 48.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330346 s, 31.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299147 s, 35.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215301 s, 48.7 MB/sldlm.namespaces.lustre-OST0001-osc-ffff9864494bb000.lock_unused_count=1 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0362963 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235158 s, 44.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0376633 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285152 s, 36.8 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg236-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg236-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (3s) == sanityn test 77b: check CRR-N NRS policy ============== 18:34:19 (1778538859) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.101083 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0450609 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0609302 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0601919 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0641641 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0580285 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0664159 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0781423 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0722122 s, 14.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0757201 s, 13.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0742522 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0784863 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.071451 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0703748 s, 14.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9864494bb000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0807167 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0667357 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0765382 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115055 s, 91.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233329 s, 44.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587821 s, 178 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173366 s, 60.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00545122 s, 192 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242394 s, 43.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217435 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251934 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835139 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257125 s, 40.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142812 s, 73.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172885 s, 60.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213973 s, 49.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208034 s, 50.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00979587 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165583 s, 63.3 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg236-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg236-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.103968 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0454448 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0393965 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0603342 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0581981 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0574844 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0577102 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0538358 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0374815 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0658488 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0821743 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0879062 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0841196 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0831427 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0660571 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0706525 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0695743 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129967 s, 80.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016422 s, 63.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193627 s, 54.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175241 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115744 s, 90.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021868 s, 48.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228661 s, 45.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178889 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222406 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00303423 s, 346 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224589 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00328874 s, 319 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102456 s, 102 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109565 s, 95.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159161 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012397 s, 84.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9864494bb000.lock_unused_count=1 removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg236-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg236-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) == sanityn test 77c: check ORR NRS policy ================ 18:34:23 (1778538863) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.14393 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297764 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329268 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0558766 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0706806 s, 14.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0701444 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.065129 s, 16.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0759434 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0643223 s, 16.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0883578 s, 11.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0664535 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0785236 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0799771 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0700806 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0650745 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0850315 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0980064 s, 10.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210695 s, 49.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211143 s, 49.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180823 s, 58.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0339937 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260511 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296292 s, 35.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288158 s, 36.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293664 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328484 s, 31.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300994 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305378 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329027 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329857 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324558 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0361465 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0413923 s, 25.3 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg236-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg236-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.114242 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025595 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0350905 s, 29.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0613875 s, 17.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0504331 s, 20.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0546826 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0712259 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.079144 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0849827 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0563454 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0654343 s, 16.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0585395 s, 17.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.076409 s, 13.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105118 s, 10.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0865035 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0841832 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0764901 s, 13.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180806 s, 58.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021181 s, 49.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245029 s, 42.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229248 s, 45.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.037758 s, 27.8 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9864494bb000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236043 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0345575 s, 30.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0346831 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0442291 s, 23.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.046878 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300984 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260167 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0348099 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219349 s, 47.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313294 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297511 s, 35.2 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg236-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg236-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.106778 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198748 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015015 s, 69.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174552 s, 60.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020642 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317446 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219125 s, 47.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0516104 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0657012 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0447082 s, 23.5 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9864494bb000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0821219 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111797 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100973 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11474 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119002 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104828 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135802 s, 7.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9864494bb000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189553 s, 55.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155717 s, 67.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173118 s, 60.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155452 s, 67.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171464 s, 61.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243656 s, 43.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181571 s, 57.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572747 s, 183 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627141 s, 167 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550104 s, 191 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00520728 s, 201 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246952 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253241 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00468278 s, 224 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00358503 s, 292 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890264 s, 118 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg236-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg236-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (5s) == sanityn test 77d: check TRR nrs policy ================ 18:34:28 (1778538868) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.170307 s, 98.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0537766 s, 19.5 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0582595 s, 18.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0521097 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0509492 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0854708 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0939481 s, 11.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.092697 s, 11.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0675263 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0692742 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0805952 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106643 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0912301 s, 11.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0835028 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0745416 s, 14.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0782954 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112607 s, 9.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9864494bb000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9864494bb000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266623 s, 39.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259861 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019387 s, 54.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026411 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295707 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00949567 s, 110 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235943 s, 44.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169632 s, 61.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202445 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306711 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259687 s, 40.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241066 s, 43.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282567 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163168 s, 64.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288036 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150528 s, 69.7 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg236-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg236-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.100835 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0340972 s, 30.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299739 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0442435 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0401549 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0466358 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312833 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0591069 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0689966 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100717 s, 10.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0901748 s, 11.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0958777 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103444 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109574 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0842468 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122719 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104116 s, 10.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9864494bb000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff9864494bb000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0091558 s, 115 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149806 s, 70.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695361 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168124 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133451 s, 78.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967287 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168442 s, 62.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104112 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794423 s, 132 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192375 s, 54.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102384 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066002 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00329624 s, 318 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00312412 s, 336 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151392 s, 69.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121114 s, 86.6 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg236-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg236-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.110983 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0479909 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0458268 s, 22.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321216 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.056859 s, 18.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.047463 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0693726 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0879052 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0782784 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0795821 s, 13.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0772755 s, 13.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9864494bb000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0696389 s, 15.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0952201 s, 11.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0681094 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.083009 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0943449 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0850278 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100815 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153329 s, 68.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00371565 s, 282 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148719 s, 70.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127295 s, 82.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184836 s, 56.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9864494bb000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00370114 s, 283 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00361723 s, 290 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586114 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00322331 s, 325 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210369 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990675 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00402914 s, 260 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016712 s, 62.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00293156 s, 358 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00478972 s, 219 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg236-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg236-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (4s) == sanityn test 77e: check TBF NID nrs policy ============ 18:34:32 (1778538872) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.202.36@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.155133 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0774079 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0747621 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0865611 s, 12.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115752 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117849 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114898 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13686 s, 7.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121631 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133205 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135408 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134964 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141093 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133257 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130825 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139344 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125271 s, 8.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9864494bb000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108263 s, 96.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016743 s, 62.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186629 s, 56.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116664 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136506 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198802 s, 52.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214866 s, 48.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248024 s, 42.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113632 s, 92.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973146 s, 108 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9864494bb000.lock_unused_count=1 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174643 s, 60.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126664 s, 82.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165474 s, 63.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00359775 s, 291 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144034 s, 72.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592286 s, 177 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg236-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg236-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.138293 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0412477 s, 25.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0505422 s, 20.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0427467 s, 24.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0828716 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.088421 s, 11.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.078707 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106472 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103335 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0807091 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0766051 s, 13.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0930898 s, 11.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0832461 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0829345 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0803356 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102178 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114407 s, 9.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179288 s, 58.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225294 s, 46.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195491 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149923 s, 69.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210761 s, 49.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152666 s, 68.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212339 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202706 s, 51.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256133 s, 40.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178318 s, 58.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00399361 s, 263 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243982 s, 43.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973504 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00508046 s, 206 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00472484 s, 222 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603141 s, 174 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg236-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg236-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.187945 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0455211 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0804733 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0663736 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0707185 s, 14.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0737951 s, 14.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0753047 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0659159 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.067514 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110001 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116265 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105136 s, 10.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121707 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106331 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110502 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102114 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0499115 s, 21.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9864494bb000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174975 s, 59.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217807 s, 48.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144415 s, 72.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205892 s, 50.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014342 s, 73.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166636 s, 62.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132144 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638211 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02276 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00399051 s, 263 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185713 s, 56.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025529 s, 41.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154965 s, 67.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189773 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173157 s, 60.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614415 s, 171 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg236-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg236-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.136713 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0546691 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.056531 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0743643 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0952254 s, 11.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0940054 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0807653 s, 13.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0759586 s, 13.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0883997 s, 11.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0884818 s, 11.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0820998 s, 12.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0910105 s, 11.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101282 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10316 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12427 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104787 s, 10.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9864494bb000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134201 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163674 s, 64.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192625 s, 54.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023428 s, 44.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260605 s, 40.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253842 s, 41.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238692 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230949 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252723 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288775 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973736 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653392 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161753 s, 64.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0359363 s, 29.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602587 s, 174 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127257 s, 82.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00299525 s, 350 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg236-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg236-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (9s) == sanityn test 77f: check TBF JobID nrs policy ========== 18:34:41 (1778538881) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg236-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.168383 s, 99.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0450415 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.043693 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0648729 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0794485 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0755012 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0850359 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0815201 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0684045 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103843 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109008 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115946 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138561 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14017 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155463 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141518 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115191 s, 9.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9864494bb000.lock_unused_count=1 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160614 s, 65.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170639 s, 61.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196329 s, 53.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148491 s, 70.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827163 s, 127 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139007 s, 75.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356214 s, 29.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237163 s, 44.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266196 s, 39.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283714 s, 37.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221268 s, 47.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9864494bb000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108965 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864652 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133858 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140209 s, 74.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00273287 s, 384 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg236-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg236-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg236-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.145782 s, 115 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282112 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0348636 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0556051 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0608705 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.096715 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.072787 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0934053 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0840949 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0885821 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121852 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120502 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10601 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105675 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120342 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127621 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130879 s, 8.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9864494bb000.lock_unused_count=1 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183047 s, 57.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183091 s, 57.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021688 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179261 s, 58.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178513 s, 58.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160752 s, 65.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204217 s, 51.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242523 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174135 s, 60.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0045146 s, 232 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226729 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00444795 s, 236 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060667 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108163 s, 96.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074818 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00474902 s, 221 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg236-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg236-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg236-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.1347 s, 125 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286163 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0521442 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0672044 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0816734 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.085994 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100978 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0851692 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100221 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0928362 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0981539 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0787422 s, 13.3 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9864494bb000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0726955 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100219 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111279 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0974574 s, 10.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12544 s, 8.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304064 s, 34.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250595 s, 41.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260496 s, 40.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248598 s, 42.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263658 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222446 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381102 s, 27.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0342753 s, 30.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022466 s, 46.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0384845 s, 27.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435193 s, 24.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0050636 s, 207 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219941 s, 47.7 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff9864494bb000.lock_unused_count=1 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133315 s, 78.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00343079 s, 306 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00354169 s, 296 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg236-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg236-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg236-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.129444 s, 130 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231175 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0386816 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0544336 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0452971 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0919278 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0898997 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117791 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109297 s, 9.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121877 s, 8.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135609 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135154 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153668 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129803 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147551 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16374 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133625 s, 7.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9864494bb000.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff9864494bb000.lock_unused_count=1 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162326 s, 64.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197096 s, 53.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772566 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147961 s, 70.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613413 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162938 s, 64.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018869 s, 55.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021476 s, 48.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101501 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017109 s, 61.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009943 s, 105 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341929 s, 30.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881589 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660513 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142106 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655712 s, 160 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg236-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg236-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (8s) == sanityn test 77g: Change TBF type directly ============ 18:34:49 (1778538889) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (3s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 18:34:52 (1778538892) oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg236-client: oleg236-server: ssh exited with exit code 2 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg236-client: oleg236-server: ssh exited with exit code 12 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg236-client: oleg236-server: ssh exited with exit code 12 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg236-client: oleg236-server: ssh exited with exit code 12 PASS 77h (4s) == sanityn test 77i: Change rank of TBF rule ============= 18:34:56 (1778538896) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (8s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 18:35:04 (1778538904) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.111477 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0643575 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0791405 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0847838 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100902 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0990756 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0903788 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0901899 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0898694 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104894 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117816 s, 8.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107838 s, 9.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117089 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104764 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124727 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110111 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105328 s, 10.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126548 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182518 s, 57.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159738 s, 65.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168331 s, 62.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207597 s, 50.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153985 s, 68.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145887 s, 71.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792167 s, 132 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119071 s, 88.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206503 s, 50.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648019 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0311489 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202651 s, 51.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00507875 s, 206 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102176 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00471535 s, 222 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg236-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg236-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.92555 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4349 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (47s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 18:35:51 (1778538951) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg236-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.653382 s, 25.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0434952 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0349731 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0484919 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.059566 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0447458 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0531939 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0615572 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0559385 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.356024 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.389187 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.376682 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.391603 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.400335 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.395723 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.414828 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.425146 s, 2.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131994 s, 7.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129131 s, 8.1 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136482 s, 7.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133392 s, 7.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140138 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140303 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133097 s, 7.9 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142862 s, 7.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132625 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137095 s, 7.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140546 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140696 s, 7.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141105 s, 7.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138026 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139271 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140515 s, 7.5 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg236-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg236-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6619 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9478 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg236-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.664177 s, 25.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156433 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161799 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173694 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201397 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192561 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19035 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190374 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181449 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.355735 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.372616 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.383323 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.405985 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.396871 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.386416 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.381758 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.392425 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133346 s, 7.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141589 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134982 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135073 s, 7.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141945 s, 7.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137096 s, 7.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137403 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138964 s, 7.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138459 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142866 s, 7.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143421 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141089 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143069 s, 7.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145259 s, 7.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141214 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144523 s, 7.3 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg236-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg236-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6797 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9462 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (136s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 18:38:07 (1778539087) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg236-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.202889 s, 82.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0348374 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206042 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0461379 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0569588 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0427605 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0397195 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0446201 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0633427 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.059758 s, 17.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0915893 s, 11.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0888275 s, 11.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0932636 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110603 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0980489 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106763 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0975683 s, 10.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137813 s, 76.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132007 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135422 s, 77.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178275 s, 58.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213355 s, 49.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242688 s, 43.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826534 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225313 s, 46.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274233 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252634 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0045764 s, 229 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248151 s, 42.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648218 s, 162 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837554 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124404 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680666 s, 154 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg236-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg236-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93367 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73132 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.36@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.36@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.200928 s, 83.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0512292 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0607011 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0740301 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.066696 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0742161 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0630738 s, 16.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.067423 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0748766 s, 14.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0920093 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0753584 s, 13.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143148 s, 7.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154554 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229354 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.250483 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.339162 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.344242 s, 3.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021587 s, 48.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224067 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254774 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267467 s, 39.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205481 s, 51.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275296 s, 38.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222972 s, 47.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269042 s, 39.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154394 s, 67.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336753 s, 31.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621596 s, 169 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010878 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015403 s, 68.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0334793 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00518042 s, 202 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072305 s, 145 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg236-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg236-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93212 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73509 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.36@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg236-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.203261 s, 82.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338733 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0533015 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0579208 s, 18.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0649553 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0908079 s, 11.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.083909 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0800197 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.082182 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0755952 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0840117 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0799434 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0915967 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0751325 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0937443 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0868099 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103025 s, 10.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159777 s, 65.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157824 s, 66.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146025 s, 71.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189189 s, 55.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210389 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137195 s, 76.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150781 s, 69.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221744 s, 47.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206737 s, 50.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173776 s, 60.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00437388 s, 240 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00517337 s, 203 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00534941 s, 196 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00269796 s, 389 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00380765 s, 275 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00350046 s, 300 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg236-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg236-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.94453 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.89322 s, 21.4 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg236-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.349087 s, 48.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323584 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251948 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0380619 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0431407 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312994 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306393 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0431345 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266804 s, 39.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154459 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171372 s, 6.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187384 s, 5.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196697 s, 5.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.343663 s, 3.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.320659 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.34029 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.338668 s, 3.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169817 s, 61.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222996 s, 47.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226892 s, 46.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288946 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177915 s, 58.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187838 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025429 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170603 s, 61.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211363 s, 49.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105427 s, 99.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132953 s, 78.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216019 s, 48.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0364477 s, 28.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0373438 s, 28.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236257 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234002 s, 44.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg236-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg236-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.79524 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74975 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.91975 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73445 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg236-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.662469 s, 25.3 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113278 s, 9.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123479 s, 8.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126344 s, 8.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127708 s, 8.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118499 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112829 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116702 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125356 s, 8.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.324288 s, 3.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.324252 s, 3.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.318636 s, 3.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.326565 s, 3.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.320565 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.324585 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.35196 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.347207 s, 3.0 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00926325 s, 113 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221976 s, 47.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177408 s, 59.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227181 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180771 s, 58.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214475 s, 48.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237615 s, 44.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223342 s, 46.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864193 s, 121 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188515 s, 55.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026283 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242518 s, 43.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00530803 s, 198 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0040929 s, 256 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00362518 s, 289 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00409069 s, 256 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg236-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg236-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6783 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4425 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg236-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.161913 s, 104 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335283 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0529637 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0775465 s, 13.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0818899 s, 12.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0763248 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0782377 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0761518 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0767587 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0904206 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106866 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0840028 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0925371 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0925612 s, 11.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100997 s, 10.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102541 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11329 s, 9.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01048 s, 100 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00950105 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142432 s, 73.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208488 s, 50.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147346 s, 71.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767909 s, 137 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197213 s, 53.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194512 s, 53.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555318 s, 189 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00504882 s, 208 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00365856 s, 287 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199998 s, 52.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100113 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106822 s, 98.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939091 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00295631 s, 355 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg236-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg236-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.9348 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73444 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg236-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.265531 s, 63.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0598436 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0656573 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0651386 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0929476 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112652 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0951466 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0919801 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108832 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117792 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116093 s, 9.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230177 s, 4.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209053 s, 5.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.291088 s, 3.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.275694 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.300937 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.300695 s, 3.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171903 s, 61.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189258 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168932 s, 62.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227869 s, 46.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021537 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183514 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194306 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178295 s, 58.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269611 s, 38.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240239 s, 43.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750105 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00987908 s, 106 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184981 s, 56.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00471598 s, 222 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064607 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00213223 s, 492 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg236-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg236-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.78394 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73651 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.76702 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.72997 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (358s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 18:44:05 (1778539445) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (4s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 18:44:09 (1778539449) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg236-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.0998 s, 338 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg236-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.0154 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 18:44:59 (1778539499) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.110521 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286339 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0652151 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0898354 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0950472 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0948188 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108907 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121364 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119204 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1081 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12317 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107601 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121045 s, 8.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137793 s, 7.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14462 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114953 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116547 s, 9.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9864494bb000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9864494bb000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278957 s, 37.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338332 s, 31.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0348883 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027667 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326815 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321378 s, 32.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315381 s, 33.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.03948 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324047 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0371549 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0459836 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0343091 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0358348 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014138 s, 74.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0375305 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0479265 s, 21.9 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg236-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg236-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93512 s, 21.2 MB/s Write runtime is 5 s, speed is 20.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.97233 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.156375 s, 107 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0511093 s, 20.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0593819 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0683834 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0776547 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0574969 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0629891 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0888099 s, 11.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0592378 s, 17.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0707247 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0534417 s, 19.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139258 s, 7.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139371 s, 7.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.221848 s, 4.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22567 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.346238 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.337191 s, 3.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166006 s, 63.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170472 s, 61.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157698 s, 66.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211273 s, 49.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187047 s, 56.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187834 s, 55.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216176 s, 48.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673279 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00428214 s, 245 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207349 s, 50.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193646 s, 54.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963147 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00389433 s, 269 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291162 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126691 s, 82.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101429 s, 103 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg236-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg236-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.9288 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.96994 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (66s) == sanityn test 77o: Changing rank should not panic ====== 18:46:05 (1778539565) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (4s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 18:46:09 (1778539569) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (40s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 18:46:49 (1778539609) mds.MDS.mdt.nrs_policies=tbf oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg236-client: oleg236-server: ssh exited with exit code 36 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg236-client: oleg236-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (12s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 18:47:01 (1778539621) oleg236-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: Resource temporarily unavailable pdsh@oleg236-client: oleg236-server: ssh exited with exit code 11 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (3s) == sanityn test 79: xattr: intent error ================== 18:47:04 (1778539624) striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d79.sanityn File: /mnt/lustre/d79.sanityn Size: 12288 Blocks: 24 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 162129603798763004 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:47:05.000000000 -0400 Modify: 2026-05-11 18:47:05.000000000 -0400 Change: 2026-05-11 18:47:05.000000000 -0400 Birth: 2026-05-11 18:47:06.000000000 -0400 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (13s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 18:47:17 (1778539637) total: 10 open/close in 0.03 seconds: 291.60 ops/second multiop /mnt/lustre2/d80a.sanityn/f80a.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 lfs migrate: /mnt/lustre/d80a.sanityn/f80a.sanityn migrate failed: Device or resource busy (16) Finish migration, then checking.. PASS 80a (5s) == sanityn test 80b: Accessing directory during migration ========================================================== 18:47:22 (1778539642) start migration thread 302845 accessing the migrating directory for 1 minute... concurrent access 153 failures, 2319 successes /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 302845 Killed ( while true; do mdt_idx=$((RANDOM % MDSCOUNT)); $LFS migrate -m $mdt_idx $migdir1 &> /dev/null; done ) (wd: ~) PASS 80b (61s) == sanityn test 81a: rename and stat under striped directory ========================================================== 18:48:23 (1778539703) stat: cannot statx 'd0/0': No such file or directory File: d0/0 Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205289282299 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:48:24.000000000 -0400 Modify: 2026-05-11 18:48:24.000000000 -0400 Change: 2026-05-11 18:48:24.000000000 -0400 Birth: 2026-05-11 18:48:25.000000000 -0400 PASS 81a (3s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 18:48:26 (1778539706) total: 1000 open/close in 2.27 seconds: 440.06 ops/second /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 306938 Killed ( while true; do touch $DIR1/$tdir; done ) (wd: ~) PASS 81b (31s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 18:48:57 (1778539737) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 18:48:58 (1778539738) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7874 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7874 PASS 82 (2s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 18:49:00 (1778539740) start pid 310164 to create/unlink striped directory start pid 310165 to stat striped directory /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4949: 310164 Terminated ( cd $DIR1; while true; do $LFS mkdir -i1 -c2 $tdir; rmdir $tdir; done ) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4949: 310165 Terminated ( cd $DIR2; while true; do stat $tdir > /dev/null 2>&1; done ) PASS 83 (123s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 18:51:03 (1778539863) fail_loc=0x8000060b PASS 84 (7s) == sanityn test 90: open/create and unlink striped directory ========================================================== 18:51:10 (1778539870) start pid 313524 to open/create under striped directory start pid 313526 to unlink striped directory /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5011: 313524 Terminated ( cd $DIR1; while true; do $LFS mkdir -c$MDSCOUNT $tdir > /dev/null 2>&1; touch $tdir/f{0..3} > /dev/null 2>&1; done ) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5011: 313526 Terminated ( cd $DIR2; while true; do rm -rf $tdir > /dev/null 2>&1; done ) PASS 90 (183s) == sanityn test 91: chmod and unlink striped directory === 18:54:13 (1778540053) start pid 319861 to chmod striped directory start pid 319863 to unlink striped directory /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5048: 319861 Terminated ( cd $DIR1; while true; do $LFS mkdir -c$MDSCOUNT $tdir > /dev/null 2>&1; chmod go+w $tdir > /dev/null 2>&1; done ) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5048: 319863 Terminated ( cd $DIR2; while true; do rm -rf $tdir > /dev/null 2>&1; done ) PASS 91 (184s) == sanityn test 92: create remote directory under orphan directory ========================================================== 18:57:18 (1778540238) fail_loc=0x1408 mkdir: cannot create directory '/mnt/lustre2/d92.sanityn/dir': No such file or directory lfs setdirstripe: cannot resolve path '/mnt/lustre2/d92.sanityn/remote_dir': No such file or directory (2) lfs setdirstripe: '/mnt/lustre2/d92.sanityn/remote_dir' is not on a Lustre filesystem: No such file or directory (2) lfs setdirstripe: cannot create dir '/mnt/lustre2/d92.sanityn/remote_dir': No such file or directory fail_loc=0 PASS 92 (7s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 18:57:24 (1778540244) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2450 0x992 0x280000400 1 2440 0x988 0x2c0000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2450 0x992 0 0 2456 0x998 0 rc1=0 and rc2=0 PASS 93 (14s) == sanityn test 94: signal vs CP callback race =========== 18:57:38 (1778540258) osc.lustre-OST0000-osc-ffff986444f2c000.idle_timeout=0 osc.lustre-OST0000-osc-ffff9864494bb000.idle_timeout=0 osc.lustre-OST0001-osc-ffff986444f2c000.idle_timeout=0 osc.lustre-OST0001-osc-ffff9864494bb000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0926135 s, 88.5 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 3.09227 s, 1.3 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 327253 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0114735 s, 357 kB/s osc.lustre-OST0000-osc-ffff986444f2c000.idle_timeout=debug osc.lustre-OST0000-osc-ffff9864494bb000.idle_timeout=debug osc.lustre-OST0001-osc-ffff986444f2c000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9864494bb000.idle_timeout=debug PASS 94 (11s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 18:57:49 (1778540269) llite.lustre-ffff986444f2c000.fast_read=0 llite.lustre-ffff9864494bb000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.00243896 s, 5.0 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff986444f2c000.fast_read=1 llite.lustre-ffff9864494bb000.fast_read=1 PASS 95b (8s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 18:57:57 (1778540277) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 18:57:58 (1778540278) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00175227 s, 2.3 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (3s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 18:58:01 (1778540281) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0279924 s, 74.9 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 18:58:03 (1778540283) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0231537 s, 90.6 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 18:58:05 (1778540285) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.001737 s, 6.9 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 18:58:07 (1778540287) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000650804 s, 6.3 MB/s PASS 101a (3s) == sanityn test 101b: Discard DoM data on rename ========= 18:58:10 (1778540290) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000633095 s, 6.5 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 18:58:12 (1778540292) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00052663 s, 7.8 MB/s PASS 101c (4s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 18:58:17 (1778540297) striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d103.sanityn fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1778540298 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1778540300 Finishing test test23 at 1778540301 PASS 103 (8s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 18:58:25 (1778540305) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 18:58:26 (1778540306) striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d105.sanityn fail_loc=0x416 PASS 105 (79s) == sanityn test 106a: Verify the btime via statx() ======= 18:59:45 (1778540385) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (2s) == sanityn test 106b: Glimpse RPCs test for statx ======== 18:59:47 (1778540387) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0038665 s, 265 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205289284927 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:59:47.000000000 -0400 Modify: 2026-05-11 18:59:47.000000000 -0400 Change: 2026-05-11 18:59:47.000000000 -0400 Birth: 2026-05-11 18:59:48.000000000 -0400 /mnt/lustre/f106b.sanityn 144115205289284927 -rw-r--r-- 1 0 0 1778540388 1778540387 1778540387 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205289284927 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 18:59:47.000000000 -0400 Modify: 2026-05-11 18:59:47.000000000 -0400 Change: 2026-05-11 18:59:47.000000000 -0400 Birth: 2026-05-11 18:59:48.000000000 -0400 1778540387 1024 1 PASS 106b (3s) == sanityn test 106c: Verify statx attributes mask ======= 18:59:50 (1778540390) PASS 106c (2s) == sanityn test 107a: Basic grouplock conflict =========== 18:59:52 (1778540392) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.0957947 s, 109 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2455 0x997 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 First grouplock blocks second one PASS 107a (5s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 18:59:57 (1778540397) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.095321 s, 110 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2463 0x99f 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7874 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (9s) == sanityn test 108a: lseek: parallel updates ============ 19:00:06 (1778540406) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.0046626 s, 2.1 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 4.398 s, 1.9 MB/s PASS 108a (8s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 19:00:14 (1778540414) 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 192.168.202.136@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.136@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg236-client.virtnet /mnt/lustre (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg236-server' oleg236-server: oleg236-server.virtnet: executing load_modules_local oleg236-server: Loading modules from /home/green/git/lustre-release/lustre oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg236-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg236-server: detected 4 online CPUs by sysfs oleg236-server: Force libcfs to create 2 CPU partitions Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre Starting client: oleg236-client.virtnet: -o user_xattr,flock oleg236-server@tcp:/lustre /mnt/lustre2 PASS 109 (601s) == sanityn test 110: do not grant another lock on resend ========================================================== 19:10:15 (1778541015) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115238826934275 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-11 19:10:15.000000000 -0400 Modify: 2026-05-11 19:10:15.000000000 -0400 Change: 2026-05-11 19:10:15.000000000 -0400 Birth: 2026-05-11 19:10:16.000000000 -0400 fail_loc=0 fail_val=0 Connected clients: oleg236-client.virtnet oleg236-client.virtnet PASS 110 (36s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 19:10:51 (1778541051) fail_loc=0x8000018a /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5892: 396302 Terminated $MULTIOP $DIR2/$tdir/mdt0dir/foodir/file2 Ow4096_c PASS 111 (6s) == sanityn test 112: update max-inherit in default LMV === 19:10:57 (1778541057) lmv.lustre-clilmv-ffff9864437bd800.qos_threshold_rr=90 lmv.lustre-clilmv-ffff98644a7bf000.qos_threshold_rr=90 PASS 112 (3s) == sanityn test 113: check servers of specified fs ======= 19:11:00 (1778541060) lustre-OST0000-osc-ffff9864437bd800 active. lustre-OST0001-osc-ffff9864437bd800 active. lustre-OST0000-osc-ffff9864437bd800 active. lustre-OST0001-osc-ffff9864437bd800 active. MGC192.168.202.136@tcp active. PASS 113 (2s) cleanup: ====================================================== == sanityn test complete, duration 6254 sec ============== 19:11:03 (1778541063) Stopping clients: oleg236-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg236-client.virtnet /mnt/lustre2 opts: