-----============= acceptance-small: sanity-dom ============----- Tue Sep 8 20:40:17 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky use SANITY_DOM_ONLY='' SANITY_DOM_EXCEPT='' oleg228-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanity-dom.*ex': No such file or directory only running tests: sanity sanityn excepting tests: === sanity-dom: start setup 20:40:40 (1788914440) === Starting client oleg228-client.virtnet: -o user_xattr,flock 192.168.202.128@tcp:/lustre /mnt/lustre2 Mount client oleg228-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.202.128@tcp:/lustre /mnt/lustre2 Started clients oleg228-client.virtnet: 192.168.202.128@tcp:/lustre on /mnt/lustre2 type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) oleg228-client.virtnet: executing check_config_client /mnt/lustre oleg228-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg228-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff9ccd89126000.idle_timeout=debug osc.lustre-OST0000-osc-ffff9ccd98ec1800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9ccd89126000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9ccd98ec1800.idle_timeout=debug disable quota as required oleg228-server: oleg228-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanity-dom: finish setup 20:41:34 (1788914494) === running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7617] 192.168.202.128@tcp:/lustre /mnt/lustre2 lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose 0 0 == sanity-dom test sanity: Run sanity with Data-on-MDT files ========================================================== 20:41:38 (1788914498) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky use SANITY_ONLY='36 39 40 41 42d 42e 43 46 56r 101e 119a 131 150a 155a 155b 155c 155d 207 241 251 150b 150bb 150c 150d 150f 150g 150ia 150ib 150ic' SANITY_EXCEPT='' oleg228-server: /home/green/git/lustre-release/lustre/tests/except/sanity.ex - see CLIENT_VERSION > v2_15_63-134-gdacc4b6d38 (34683217 > 34553734) for LU-15963, go 312 - see CLIENT_VERSION > v2_15_63-239-gab4ede45b4 (34683217 > 34553839) for LU-17856, go 56wa 56xe - see CLIENT_VERSION >= v2_15_7-16-gab522d54ad (34683217 >= 34539280) for LU-16203, go 60i - see CLIENT_VERSION > v2_16_54-54-ga2e3a2f5a3 (34683217 > 34616886) for LU-14590, go 0d - see CLIENT_VERSION >= v2_17_50-196-g63ee4af3bb (34683217 >= 34681540) for LU-19620, go 60b - see CLIENT_VERSION >= v2_17_50-34-g089f438ba9 (34683217 >= 34681378) for LU-19692, go 77k - see CLIENT_VERSION >= v2_17_50-71-g2cbe29c282 (34683217 >= 34681415) for LU-19644, go 150ib - see CLIENT_VERSION >= v2_17_50-72-ge1f272796c (34683217 >= 34681416) for LU-19743, go 27z - see CLIENT_VERSION >= v2_17_52-99-g84242c4358 (34683217 >= 34681955) for LU-20010, go 56aa - see CLIENT_VERSION >= v2_17_55-19-g2868627130 (34683217 >= 34682643) for LU-20435, go 430a - need MDS1_VERSION <= 2.16.61-1-g89cf292a8c2 (34683217 <= 34618625) for LU-18938, skip 360 - see MDS1_VERSION <= CLIENT_VERSION (34683217 <= 34683217) for LU-18562, go 270a - need MDS1_VERSION < v2_14_55-100-g8a84c7f9c7 (34683217 < 34486116) for LU-14927, skip 0f - need OST1_VERSION < v2_17_50-410-gbd8e7b9555 (34683217 < 34681754) for LU-12550, skip 216 only running tests: sanity sanityn 36 39 40 41 42d 42e 43 46 56r 101e 119a 131 150a 155a 155b 155c 155d 207 241 251 150b 150bb 150c 150d 150f 150g 150ia 150ib 150ic excepting tests: 42a 42c 42b 118c 118d 407 119i 216 817 411a 130b 130c 130d 130e 130f 130g 360 0f 216 skipping tests SLOW=no: 27m 60i 64b 68 71 135 136 230d 300o 842 51b 51c 51e 834 === sanity: start setup 20:42:14 (1788914534) === Stopping clients: oleg228-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg228-client.virtnet /mnt/lustre2 opts: oleg228-client.virtnet: executing check_config_client /mnt/lustre oleg228-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg228-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff9ccd89126000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9ccd89126000.idle_timeout=debug disable quota as required oleg228-server: oleg228-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanity: finish setup 20:43:08 (1788914588) === running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f10182] preparing for tests involving mounts mke2fs 1.47.3-wc2 (11-Nov-2025) debug=all == sanity test 36a: MDS utime check (mknod, utime) ======= 20:43:15 (1788914595) utime: good mknod times 1788914597 <= 1788914598 <= 1788914598 for /mnt/lustre/f36 utime: good utime mtimes 100000, atime 200000 PASS 36a (10s) == sanity test 36b: OST utime check (open, utime) ======== 20:43:25 (1788914605) utime: good utime mtimes 100000, atime 200000 PASS 36b (11s) == sanity test 36c: non-root MDS utime check (mknod, utime) ========================================================== 20:43:36 (1788914616) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [utime] [/mnt/lustre/d36/f36] utime: good mknod times 1788914618 <= 1788914619 <= 1788914619 for /mnt/lustre/d36/f36 utime: good utime mtimes 100000, atime 200000 PASS 36c (10s) == sanity test 36d: non-root OST utime check (open, utime) ========================================================== 20:43:46 (1788914626) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [utime] [/mnt/lustre/d36/f36] utime: good utime mtimes 100000, atime 200000 PASS 36d (8s) == sanity test 36e: utime on non-owned file (should return error) ========================================================== 20:43:54 (1788914634) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [utime] [/mnt/lustre/d36e.sanity/f36e.sanity] utime: utime(/mnt/lustre/d36e.sanity/f36e.sanity) failed: rc 1: Operation not permitted PASS 36e (8s) == sanity test 36f: utime on file racing with OST BRW write ==================================================================== 20:44:02 (1788914642) fail_loc=0x80000214 Tue Sep 8 20:44:04 EDT 2026 1788914644 Tue Sep 8 20:44:05 EDT 2026 1788914645 PASS 36f (8s) == sanity test 36g: FMD cache expiry =============================================================================== 20:44:11 (1788914651) FMD max age: 30s FMD before: 4 FMD after: 0 PASS 36g (53s) == sanity test 36h: utime on file racing with OST BRW write ==================================================================== 20:45:04 (1788914704) fail_loc=0x80000227 Tue Sep 8 20:45:05 EDT 2026 1788914705 Tue Sep 8 20:45:07 EDT 2026 1788914707 PASS 36h (7s) == sanity test 36i: change mtime on striped directory ==== 20:45:11 (1788914711) SKIP: sanity test_36i needs >= 2 MDTs SKIP 36i (3s) == sanity test 39a: mtime changed on create ============== 20:45:14 (1788914714) Succeed in opening file "/mnt/lustre/f39a.sanity2"(flags=O_CREAT) PASS 39a (9s) == sanity test 39b: mtime change on open, link, unlink, rename ========================================================== 20:45:23 (1788914723) repeat after cancel_lru_locks PASS 39b (8s) == sanity test 39c: mtime change on rename =============== 20:45:31 (1788914731) repeat after cancel_lru_locks PASS 39c (11s) == sanity test 39d: create, utime, stat ================== 20:45:42 (1788914742) repeat after cancel_lru_locks PASS 39d (7s) == sanity test 39e: create, stat, utime, stat ============ 20:45:49 (1788914749) repeat after cancel_lru_locks PASS 39e (8s) == sanity test 39f: create, stat, sleep, utime, stat ===== 20:45:57 (1788914757) repeat after cancel_lru_locks PASS 39f (10s) == sanity test 39g: write, chmod, stat =================== 20:46:07 (1788914767) repeat after cancel_lru_locks PASS 39g (10s) == sanity test 39h: write, utime within one second, stat ========================================================== 20:46:18 (1788914778) write and touch not within one second PASS 39h (11s) == sanity test 39i: write, rename, stat ================== 20:46:28 (1788914788) repeat after cancel_lru_locks PASS 39i (10s) == sanity test 39j: write, rename, close, stat =========== 20:46:38 (1788914798) debug=-1 debug_mb=150 debug=-1 debug_mb=150 fail_loc=0x80000412 multiop /mnt/lustre/f39j.sanity voO_RDWR:w2097152_c TMPPIPE=/tmp/multiop_open_wait_pipe.10182 repeat after cancel_lru_locks fail_loc=0 debug_mb=21 debug_mb=21 PASS 39j (15s) == sanity test 39k: write, utime, close, stat ============ 20:46:54 (1788914814) multiop /mnt/lustre/f39k.sanity voO_RDWR:w2097152_c TMPPIPE=/tmp/multiop_open_wait_pipe.10182 repeat after cancel_lru_locks PASS 39k (11s) == sanity test 39l: directory atime update =============== 20:47:05 (1788914825) PASS 39l (23s) == sanity test 39m: test atime and mtime before 1970 ===== 20:47:28 (1788914848) repeat after cancel_lru_locks PASS 39m (12s) == sanity test 39n: check that O_NOATIME is honored ====== 20:47:40 (1788914860) 1+0 records in 1+0 records out PASS 39n (22s) == sanity test 39o: directory cached attributes updated after create ========================================================== 20:48:02 (1788914882) a b PASS 39o (9s) == sanity test 39p: remote directory cached attributes updated after create ================================================================== 20:48:11 (1788914891) SKIP: sanity test_39p needs >= 2 MDTs SKIP 39p (3s) == sanity test 39r: lazy atime update on OST ============= 20:48:14 (1788914894) SKIP: sanity test_39r ldiskfs only test SKIP 39r (5s) == sanity test 39q: close won't zero out atime =========== 20:48:20 (1788914900) multiop /mnt/lustre/d39q.sanity vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.10182 PASS 39q (11s) == sanity test 39s: relatime is supported ================ 20:48:30 (1788914910) 192.168.202.128@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg228-client.virtnet /mnt/lustre (opts:) Starting client: oleg228-client.virtnet: -o relatime 192.168.202.128@tcp:/lustre /mnt/lustre 1+0 records in 1+0 records out Stopping client oleg228-client.virtnet /mnt/lustre (opts:) Starting client: oleg228-client.virtnet: -o user_xattr,flock 192.168.202.128@tcp:/lustre /mnt/lustre PASS 39s (21s) == sanity test 39u: stat race ============================ 20:48:52 (1788914932) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209134 s, 5.0 MB/s fail_loc=0x80001434 PASS 39u (23s) == sanity test 40: failed open(O_TRUNC) doesn't truncate ======================================================================= 20:49:16 (1788914956) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0119798 s, 342 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [openfile] [-f] [O_WRONLY:O_TRUNC] [/mnt/lustre/f40.sanity] Error in opening file "/mnt/lustre/f40.sanity"(flags=O_WRONLY) 13: Permission denied /mnt/lustre/f40.sanity has type file OK /mnt/lustre/f40.sanity has size 4096 OK PASS 40 (9s) == sanity test 41: test small file write + fstat =============================================================================== 20:49:24 (1788914964) First String: abcdefghijklmnopqr Second String: abcdefghiabcdefghijklmnopqr abcdefghiabcdefghijklmnopqr abcdefghiabcdefghijklmnopqr Pass! PASS 41 (8s) == sanity test 42d: test complete truncate of file with cached dirty data ========================================================== 20:49:32 (1788914972) debug=+cache vm.dirty_writeback_centisecs = 0 vm.dirty_writeback_centisecs = 0 vm.dirty_ratio = 50 vm.dirty_background_ratio = 25 100+0 records in 100+0 records out 102400 bytes (102 kB, 100 KiB) copied, 0.0531841 s, 1.9 MB/s vm.dirty_writeback_centisecs = 500 vm.dirty_background_ratio = 10 vm.dirty_ratio = 30 checking grant......UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3200 2203904 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 3072 3766272 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 2048 3767296 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 5120 7533568 1% /mnt/lustre pass grant check: client:6815744 server:6815744 vm.dirty_writeback_centisecs = 500 vm.dirty_background_ratio = 10 vm.dirty_ratio = 30 PASS 42d (9s) == sanity test 42e: verify sub-RPC writes are not done synchronously ========================================================== 20:49:41 (1788914981) - open/close 757 (time 1788914996.52 total 10.01 last 75.64) total: 1000 open/close in 13.58 seconds: 73.62 ops/second - open/close 674 (time 1788915014.56 total 10.00 last 67.37) total: 924 open/close in 13.69 seconds: 67.49 ops/second 1+0 records in 1+0 records out 484442112 bytes (484 MB, 462 MiB) copied, 22.4217 s, 21.6 MB/s osc.lustre-OST0000-osc-ffff9ccd8865c000.cur_dirty_bytes=0 osc.lustre-OST0000-osc-ffff9ccd8865c000.cur_grant_bytes=488505344 osc.lustre-OST0000-osc-ffff9ccd8865c000.cur_dirty_bytes=1048576 osc.lustre-OST0000-osc-ffff9ccd8865c000.cur_grant_bytes=485621760 osc.lustre-OST0000-osc-ffff9ccd8865c000.rpc_stats=0 osc.lustre-OST0000-osc-ffff9ccd8865c000.rpc_stats= snapshot_time: 1788915188.610930503 secs.nsecs start_time: 1788915179.910925878 secs.nsecs elapsed_time: 8.700004625 secs.nsecs read RPCs in flight: 0 write RPCs in flight: 0 DIO RPCs in flight: 0 pending write pages: 0 pending read pages: 0 read write pages per rpc rpcs % cum % | rpcs % cum % 1: 0 0 0 | 0 0 0 2: 0 0 0 | 0 0 0 4: 0 0 0 | 0 0 0 8: 0 0 0 | 0 0 0 16: 0 0 0 | 38 100 100 read write rpcs in flight rpcs % cum % | rpcs % cum % 1: 0 0 0 | 1 2 2 2: 0 0 0 | 1 2 5 3: 0 0 0 | 1 2 7 4: 0 0 0 | 2 5 13 5: 0 0 0 | 4 10 23 6: 0 0 0 | 10 26 50 7: 0 0 0 | 7 18 68 8: 0 0 0 | 6 15 84 9: 0 0 0 | 5 13 97 10: 0 0 0 | 1 2 100 read write offset rpcs % cum % | rpcs % cum % 0: 0 0 0 | 38 100 100 read write RPC latency (us) count % cum % | count % cum % 0: 0 0 0 | 0 0 0 1: 0 0 0 | 0 0 0 2: 0 0 0 | 0 0 0 4: 0 0 0 | 0 0 0 8: 0 0 0 | 0 0 0 16: 0 0 0 | 0 0 0 32: 0 0 0 | 0 0 0 64: 0 0 0 | 0 0 0 128: 0 0 0 | 0 0 0 256: 0 0 0 | 0 0 0 512: 0 0 0 | 0 0 0 1024: 0 0 0 | 0 0 0 2048: 0 0 0 | 0 0 0 4096: 0 0 0 | 0 0 0 8192: 0 0 0 | 0 0 0 16384: 0 0 0 | 5 13 13 32768: 0 0 0 | 29 76 89 65536: 0 0 0 | 4 10 100 checking grant......UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3456 2205184 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 51200 3707904 2% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 2048 3767296 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 53248 7475200 1% /mnt/lustre pass grant check: client:491913216 server:491913216 PASS 42e (259s) == sanity test 43A: execution of file opened for write should return -ETXTBSY ========================================================== 20:54:00 (1788915240) /home/green/git/lustre-release/lustre/tests/sanity.sh: line 6088: /mnt/lustre/d43A.sanity/f43A.sanity: Text file busy PASS 43A (9s) == sanity test 43a: open(RDWR) of file being executed should return -ETXTBSY ========================================================== 20:54:09 (1788915249) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 5091: 35155 Terminated $DIR/$tdir/sleep 60 (wd: ~) PASS 43a (7s) == sanity test 43b: truncate of file being executed should return -ETXTBSY ========================================================== 20:54:17 (1788915257) truncate: cannot truncate '/mnt/lustre/d43b.sanity/sleep' to length 0: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 5091: 35734 Terminated $DIR/$tdir/sleep 60 (wd: ~) PASS 43b (8s) == sanity test 43c: md5sum of copy into lustre =========== 20:54:25 (1788915265) bash: OK PASS 43c (8s) == sanity test 46: dirtying a previously written page ========================================================================== 20:54:33 (1788915273) vm.dirty_writeback_centisecs = 0 vm.dirty_writeback_centisecs = 0 vm.dirty_ratio = 50 vm.dirty_background_ratio = 25 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.023825 s, 172 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0051727 s, 792 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00168588 s, 2.4 MB/s vm.dirty_writeback_centisecs = 500 vm.dirty_background_ratio = 10 vm.dirty_ratio = 30 vm.dirty_writeback_centisecs = 500 vm.dirty_background_ratio = 10 vm.dirty_ratio = 30 PASS 46 (8s) == sanity test 56r: check lfs find -size works =========== 20:54:42 (1788915282) PASS 56r (11s) == sanity test 101e: check read-ahead for small read(1k) for small files(500k) ========================================================== 20:54:55 (1788915295) Creating 100 500K test files Cancel LRU locks on lustre client to flush the client cache Reset readahead stats llite.lustre-ffff9ccd8865c000.max_cached_mb= users: 5 max_cached_mb: 1846 used_mb: 49 unused_mb: 1797 unevict_mb: 0 reclaim_count: 0 max_read_ahead_mb: 256 used_read_ahead_mb: 0 llite.lustre-ffff9ccd8865c000.read_ahead_stats= snapshot_time 1788915380.134646091 secs.nsecs start_time 1788915364.362297982 secs.nsecs elapsed_time 15.772348109 secs.nsecs hits 12300 samples [pages] misses 200 samples [pages] zero_size_window 100 samples [pages] failed_to_fast_read 200 samples [pages] readahead_pages 100 samples [pages] 123 123 12300 PASS 101e (97s) resend_count is set to 4 4 resend_count is set to 4 4 resend_count is set to 4 4 resend_count is set to 4 4 resend_count is set to 4 4 == sanity test 119a: Short directIO read must return actual read amount ========================================================== 20:56:33 (1788915393) directio on /mnt/lustre/f119a.sanity for 1x524288 bytes PASS PASS 119a (9s) == sanity test 131a: test iov's crossing stripe boundary for writev/readv ========================================================== 20:56:43 (1788915403) PASS 131a (8s) == sanity test 131b: test append writev ================== 20:56:52 (1788915412) /mnt/lustre/f131b.sanity has type file OK /mnt/lustre/f131b.sanity has size 3145728 OK /mnt/lustre/f131b.sanity has type file OK /mnt/lustre/f131b.sanity has size 5767168 OK PASS 131b (10s) == sanity test 131c: test read/write on file w/o objects ========================================================== 20:57:01 (1788915421) Write error: Bad file descriptor (rc = -1, len = 1048576) PASS 131c (8s) == sanity test 131d: test short read ===================== 20:57:09 (1788915429) PASS 131d (10s) == sanity test 131e: test read hitting hole ============== 20:57:19 (1788915439) PASS 131e (9s) == sanity test 150a: truncate/append tests =============== 20:57:29 (1788915449) 1+0 records in 1+0 records out 6096 bytes (6.1 kB, 6.0 KiB) copied, 0.00474418 s, 1.3 MB/s Stopping client oleg228-client.virtnet /mnt/lustre (opts:) Starting client: oleg228-client.virtnet: -o user_xattr,flock 192.168.202.128@tcp:/lustre /mnt/lustre Filesystem 1024-blocks Used Available Capacity Mounted on 192.168.202.128@tcp:/lustre 7542784 7168 7531520 1% /mnt/lustre sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 150a (21s) == sanity test 150b: Verify fallocate (prealloc) functionality ========================================================== 20:57:51 (1788915471) fallocate on zfs doesn't consume space fallocate not supported SKIP: sanity test_150b need >= 2.13.57 and ldiskfs for fallocate SKIP 150b (4s) == sanity test 150bb: Verify fallocate modes both zero space ========================================================== 20:57:55 (1788915475) fallocate on zfs doesn't consume space fallocate not supported SKIP: sanity test_150bb need >= 2.13.57 and ldiskfs for fallocate SKIP 150bb (4s) == sanity test 150c: Verify fallocate Size and Blocks ==== 20:57:59 (1788915479) fallocate on zfs doesn't consume space fallocate not supported SKIP: sanity test_150c need >= 2.13.57 and ldiskfs for fallocate SKIP 150c (4s) == sanity test 150d: Verify fallocate Size and Blocks - Non zero start ========================================================== 20:58:04 (1788915484) fallocate on zfs doesn't consume space fallocate not supported SKIP: sanity test_150d need >= 2.13.57 and ldiskfs for fallocate SKIP 150d (5s) == sanity test 150f: Verify fallocate punch functionality ========================================================== 20:58:08 (1788915488) SKIP: sanity test_150f LU-14160: punch mode is not implemented on OSD ZFS SKIP 150f (5s) == sanity test 150g: Verify fallocate punch on large range ========================================================== 20:58:13 (1788915493) SKIP: sanity test_150g LU-14160: punch mode is not implemented on OSD ZFS SKIP 150g (5s) == sanity test 150ia: Verify fallocate zero-range ZERO functionality ========================================================== 20:58:18 (1788915498) SKIP: sanity test_150ia zero-range mode is not implemented on OSD ZFS SKIP 150ia (5s) == sanity test 150ib: Verify fallocate zero-range PREALLOC functionality ========================================================== 20:58:23 (1788915503) SKIP: sanity test_150ib zero-range mode is not implemented on OSD ZFS SKIP 150ib (4s) == sanity test 150ic: Verify fallocate LARGE zero PREALLOC functionality ========================================================== 20:58:27 (1788915507) SKIP: sanity test_150ic zero-range mode is not implemented on OSD ZFS SKIP 150ic (5s) == sanity test 155a: Verify small file correctness: read cache:on write_cache:on ========================================================== 20:58:33 (1788915513) oleg228-server: error: get_param: param_path 'osd-*/*/writethrough_cache_enable': No such file or directory oleg228-server: error: get_param: param_path 'osd-*/*/writethrough_cache_enable': No such file or directory oleg228-server: error: set_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory oleg228-server: error: set_param: setting 'osd-*/lustre-OST*/read_cache_enable'='1': No such file or directory pdsh@oleg228-client: oleg228-server: ssh exited with exit code 2 oleg228-server: error: set_param: param_path 'osd-*/lustre-OST*/writethrough_cache_enable': No such file or directory oleg228-server: error: set_param: setting 'osd-*/lustre-OST*/writethrough_cache_enable'='1': No such file or directory pdsh@oleg228-client: oleg228-server: ssh exited with exit code 2 1+0 records in 1+0 records out 6096 bytes (6.1 kB, 6.0 KiB) copied, 0.000430222 s, 14.2 MB/s oleg228-server: error: set_param: setting : Invalid argument pdsh@oleg228-client: oleg228-server: ssh exited with exit code 22 oleg228-server: error: set_param: setting : Invalid argument pdsh@oleg228-client: oleg228-server: ssh exited with exit code 22 PASS 155a (17s) == sanity test 155b: Verify small file correctness: read cache:on write_cache:off ========================================================== 20:58:50 (1788915530) oleg228-server: error: get_param: param_path 'osd-*/*/writethrough_cache_enable': No such file or directory oleg228-server: error: get_param: param_path 'osd-*/*/writethrough_cache_enable': No such file or directory oleg228-server: error: set_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory oleg228-server: error: set_param: setting 'osd-*/lustre-OST*/read_cache_enable'='1': No such file or directory pdsh@oleg228-client: oleg228-server: ssh exited with exit code 2 oleg228-server: error: set_param: param_path 'osd-*/lustre-OST*/writethrough_cache_enable': No such file or directory oleg228-server: error: set_param: setting 'osd-*/lustre-OST*/writethrough_cache_enable'='0': No such file or directory pdsh@oleg228-client: oleg228-server: ssh exited with exit code 2 1+0 records in 1+0 records out 6096 bytes (6.1 kB, 6.0 KiB) copied, 0.00526294 s, 1.2 MB/s oleg228-server: error: set_param: setting : Invalid argument pdsh@oleg228-client: oleg228-server: ssh exited with exit code 22 oleg228-server: error: set_param: setting : Invalid argument pdsh@oleg228-client: oleg228-server: ssh exited with exit code 22 PASS 155b (15s) == sanity test 155c: Verify small file correctness: read cache:off write_cache:on ========================================================== 20:59:06 (1788915546) oleg228-server: error: get_param: param_path 'osd-*/*/writethrough_cache_enable': No such file or directory oleg228-server: error: get_param: param_path 'osd-*/*/writethrough_cache_enable': No such file or directory oleg228-server: error: set_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory oleg228-server: error: set_param: setting 'osd-*/lustre-OST*/read_cache_enable'='0': No such file or directory pdsh@oleg228-client: oleg228-server: ssh exited with exit code 2 oleg228-server: error: set_param: param_path 'osd-*/lustre-OST*/writethrough_cache_enable': No such file or directory oleg228-server: error: set_param: setting 'osd-*/lustre-OST*/writethrough_cache_enable'='1': No such file or directory pdsh@oleg228-client: oleg228-server: ssh exited with exit code 2 1+0 records in 1+0 records out 6096 bytes (6.1 kB, 6.0 KiB) copied, 0.000762557 s, 8.0 MB/s oleg228-server: error: set_param: setting : Invalid argument pdsh@oleg228-client: oleg228-server: ssh exited with exit code 22 oleg228-server: error: set_param: setting : Invalid argument pdsh@oleg228-client: oleg228-server: ssh exited with exit code 22 PASS 155c (17s) == sanity test 155d: Verify small file correctness: read cache:off write_cache:off ========================================================== 20:59:22 (1788915562) oleg228-server: error: get_param: param_path 'osd-*/*/writethrough_cache_enable': No such file or directory oleg228-server: error: get_param: param_path 'osd-*/*/writethrough_cache_enable': No such file or directory oleg228-server: error: set_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory oleg228-server: error: set_param: setting 'osd-*/lustre-OST*/read_cache_enable'='0': No such file or directory pdsh@oleg228-client: oleg228-server: ssh exited with exit code 2 oleg228-server: error: set_param: param_path 'osd-*/lustre-OST*/writethrough_cache_enable': No such file or directory oleg228-server: error: set_param: setting 'osd-*/lustre-OST*/writethrough_cache_enable'='0': No such file or directory pdsh@oleg228-client: oleg228-server: ssh exited with exit code 2 1+0 records in 1+0 records out 6096 bytes (6.1 kB, 6.0 KiB) copied, 0.00225821 s, 2.7 MB/s oleg228-server: error: set_param: setting : Invalid argument pdsh@oleg228-client: oleg228-server: ssh exited with exit code 22 oleg228-server: error: set_param: setting : Invalid argument pdsh@oleg228-client: oleg228-server: ssh exited with exit code 22 PASS 155d (18s) == sanity test 207a: can refresh layout at glimpse ======= 20:59:41 (1788915581) 3+0 records in 3+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.0296067 s, 415 kB/s fail_loc=0x170 PASS 207a (12s) == sanity test 207b: can refresh layout at open ========== 20:59:53 (1788915593) 6+0 records in 6+0 records out 24576 bytes (25 kB, 24 KiB) copied, 0.0319726 s, 769 kB/s fail_loc=0x171 checksum is 91ff0dac5df86e798bfef5e573536b08 /mnt/lustre/f207b.sanity PASS 207b (10s) == sanity test 241a: bio vs dio ========================== 21:00:05 (1788915605) 1+0 records in 1+0 records out 40960 bytes (41 kB, 40 KiB) copied, 0.0411599 s, 995 kB/s -rw-r--r-- 1 root root 40960 Sep 8 21:00 /mnt/lustre/f241a.sanity ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=3 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=1 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 ldlm.namespaces.lustre-MDT0000-mdc-ffff9ccd98272000.lock_unused_count=2 PASS 241a (143s) == sanity test 241b: dio vs dio ========================== 21:02:28 (1788915748) 1+0 records in 1+0 records out 40960 bytes (41 kB, 40 KiB) copied, 0.0221193 s, 1.9 MB/s -rw-r--r-- 1 root root 40960 Sep 8 21:02 /mnt/lustre/f241b.sanity PASS 241b (66s) == sanity test 251a: Handling short read and write correctly ========================================================== 21:03:35 (1788915815) fail_loc=0xa0001407 fail_val=1 fail_loc=0xa0001407 fail_val=1 PASS 251a (13s) == sanity test complete, duration 1340 sec =============== 21:03:59 (1788915839) === sanity: start cleanup 21:04:03 (1788915843) === Starting client oleg228-client.virtnet: -o user_xattr,flock 192.168.202.128@tcp:/lustre /mnt/lustre2 Mount client oleg228-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.202.128@tcp:/lustre /mnt/lustre2 Started clients oleg228-client.virtnet: 192.168.202.128@tcp:/lustre on /mnt/lustre2 type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) Stopping clients: oleg228-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg228-client.virtnet /mnt/lustre2 opts: === sanity: finish cleanup 21:04:16 (1788915856) === debug=super ioctl neterror warning dlmtrace error emerg ha rpctrace vfstrace config console lfsck PASS sanity (1368s) == sanity-dom test sanityn: Run sanityn with Data-on-MDT files ========================================================== 21:04:25 (1788915865) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky use SANITYN_ONLY='1 2 4 5 6 7 8 9 10 11 12 14 17 19 20 23 27 39 51a 51c 51d 107' SANITYN_EXCEPT='' oleg228-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory only running tests: sanity sanityn 1 2 4 5 6 7 8 9 10 11 12 14 17 19 20 23 27 39 51a 51c 51d 107 excepting tests: 40a skipping tests SLOW=no: 33a === sanityn: start setup 21:04:53 (1788915893) === Starting client oleg228-client.virtnet: -o user_xattr,flock 192.168.202.128@tcp:/lustre /mnt/lustre2 Mount client oleg228-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.202.128@tcp:/lustre /mnt/lustre2 Started clients oleg228-client.virtnet: 192.168.202.128@tcp:/lustre on /mnt/lustre2 type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) oleg228-client.virtnet: executing check_config_client /mnt/lustre oleg228-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg228-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff9ccd8a0b6000.idle_timeout=debug osc.lustre-OST0000-osc-ffff9ccd98272000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9ccd8a0b6000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9ccd98272000.idle_timeout=debug disable quota as required oleg228-server: oleg228-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 21:05:38 (1788915938) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0348633 s, 30.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f61905] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 21:05:40 (1788915940) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (8s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 21:05:49 (1788915949) -rw-r--r-- 1 root root 0 Sep 8 21:05 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (8s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 21:05:57 (1788915957) -rw-r--r-- 1 root root 0 Sep 8 21:05 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (9s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 21:06:06 (1788915966) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (7s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 21:06:13 (1788915973) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (9s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 21:06:22 (1788915982) total 2 -rwxrwxrwx 1 root root 0 Sep 8 21:05 f2a -rwxrwxrwx 1 root root 0 Sep 8 21:05 f2b total 2 -rwxrwxrwx 1 root root 0 Sep 8 21:05 f2a -rwxrwxrwx 1 root root 0 Sep 8 21:05 f2b running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.0101863 s, 50.3 kB/s PASS 2e (9s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 21:06:31 (1788915991) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (5s) == sanityn test 2g: check blocks update on sync write ==== 21:06:36 (1788915996) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.429276 s, 4.9 MB/s /mnt/lustre/f2g.sanityn has 4109 blocks /mnt/lustre2/f2g.sanityn has 4109 blocks PASS 2g (10s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 21:06:46 (1788916006) PASS 4 (10s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 21:06:57 (1788916017) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (10s) == sanityn test 6: remove of open file on other node ============================================================================ 21:07:07 (1788916027) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (10s) == sanityn test 7: remove of open directory on other node ======================================================================= 21:07:17 (1788916037) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (10s) == sanityn test 8: remove of open special file on other node ==================================================================== 21:07:27 (1788916047) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (9s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 21:07:36 (1788916056) PASS 9a (10s) == sanityn test 9b: append to striped sparse file ======== 21:07:46 (1788916066) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0881389 s, 11.9 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.152811 s, 0.0 kB/s Data read (expecting 'foo'): foo PASS 9b (8s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 21:07:55 (1788916075) 1+0 records in 1+0 records out 1 byte copied, 0.0178806 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0114001 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0114673 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0410037 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0184294 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0112002 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0235786 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0149224 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0153185 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0168619 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0266377 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00860782 s, 0.1 kB/s PASS 10a (11s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 21:08:05 (1788916085) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.0220252 s, 139 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.068934 s, 59.4 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.00153252 s, 2.0 MB/s PASS 10b (8s) == sanityn test 11: execution of file opened for write should return error ============================================================== 21:08:14 (1788916094) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.61905 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 298: /mnt/lustre2/d11/f: Text file busy PASS 11 (8s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 21:08:22 (1788916102) warning: '-runtime' deprecated, use '-t runtime' instead - link 300 (time 1788916115.49 total 10.04 last 29.87) - link 712 (time 1788916125.49 total 20.04 last 41.19) - link 1104 (time 1788916135.53 total 30.08 last 39.06) - link 1461 (time 1788916145.57 total 40.12 last 35.56) - link 1824 (time 1788916155.59 total 50.14 last 36.22) - link 2070 (time 1788916165.62 total 60.17 last 24.53) - link 2315 (time 1788916175.62 total 70.17 last 24.50) - link 2581 (time 1788916185.62 total 80.17 last 26.60) - link 2858 (time 1788916195.62 total 90.17 last 27.69) total: 3148 link in 100.04 seconds: 31.47 ops/second using seed 2986373588 running for 100 seconds - stat 515 (time 1788916117 ; total 11 ; last 11) - stat 3264 (time 1788916128 ; total 22 ; last 11) - stat 6183 (time 1788916139 ; total 33 ; last 11) - stat 8793 (time 1788916150 ; total 44 ; last 11) - stat 10000 (time 1788916154 ; total 48 ; last 4) - stat 11691 (time 1788916165 ; total 59 ; last 11) - stat 12972 (time 1788916176 ; total 70 ; last 11) - stat 14709 (time 1788916187 ; total 81 ; last 11) - stat 16667 (time 1788916198 ; total 92 ; last 11) total: 18366 stats in 100 seconds: 183.660004 stats/second - unlinked 0 (time 1788916156 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile3148) error: No such file or directory total: 3147 unlinks in 103 seconds: 30.553398 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (74605) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (74607) - No such process PASS 12 (164s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 21:11:06 (1788916266) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.61905 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 333: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 5091: 75209 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (9s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 21:11:15 (1788916275) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 5091: 75789 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (10s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 21:11:25 (1788916285) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 348: 76369 Terminated $DIR1/$tdir/sleep 60 PASS 14b (9s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 21:11:34 (1788916294) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 360: 76952 Terminated $DIR1/$tdir/sleep 60 PASS 14c (10s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 21:11:44 (1788916304) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 371: 77535 Terminated $DIR1/$tdir/sleep 60 PASS 14d (12s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 21:11:57 (1788916317) fail_loc=0x8000030a PASS 17 (13s) == sanityn test 19: test concurrent uncached read races ========================================================================= 21:12:10 (1788916330) oleg228-server: error: get_param: param_path 'osd-*/lustre-MDT*/read_cache_enable': No such file or directory pdsh@oleg228-client: oleg228-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (6s) == sanityn test 20: test extra readahead page left in cache ============================================================== 21:12:17 (1788916337) PASS 20 (9s) == sanityn test 23: others should see updated atime while another read============================================================== 21:12:26 (1788916346) now is 1788916349 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.61905 new atime is 1788916410 PASS 23 (72s) == sanityn test 27: align non-overlapping extent locks from request ============================================================= 21:13:38 (1788916418) dd 1 started 4+0 records in 4+0 records out 16793600 bytes (17 MB, 16 MiB) copied, 1.58333 s, 10.6 MB/s dd 2 started 1+0 records in 1+0 records out 15728640 bytes (16 MB, 15 MiB) copied, 0.934545 s, 16.8 MB/s 1+0 records in 1+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0708446 s, 116 kB/s dd 3 finished PASS 27 (16s) == sanityn test 39a: file mtime does not change after rename ========================================================== 21:13:55 (1788916435) repeat after cancel_lru_locks PASS 39a (7s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 21:14:02 (1788916442) repeat after cancel_lru_locks PASS 39b (9s) == sanityn test 39c: check truncate mtime update ================================================================================ 21:14:11 (1788916451) repeat after cancel_lru_locks PASS 39c (9s) == sanityn test 39d: sync write should update mtime ====== 21:14:21 (1788916461) fail_loc=0x411 fail_loc=0 PASS 39d (8s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 21:14:30 (1788916470) 0+1 records in 0+1 records out 158 bytes copied, 0.0300534 s, 5.3 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3690: kill: (83911) - No such process PASS 51a (11s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 21:14:41 (1788916481) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0393742 s, 26.0 kB/s PASS 51c (17s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 21:14:58 (1788916498) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0624926 s, 16.8 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (10s) == sanityn test 107a: Basic grouplock conflict =========== 21:15:09 (1788916509) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.565284 s, 18.5 MB/s /mnt/lustre/f107a.sanityn lcm_layout_gen: 3 lcm_mirror_count: 1 lcm_entry_count: 2 lcme_id: 1 lcme_mirror_id: 0 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 0 lmm_stripe_size: 1048576 lmm_pattern: mdt lmm_layout_gen: 0 lmm_mdt_index: 0 lcme_id: 2 lcme_mirror_id: 0 lcme_flags: init lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x280000400:0x3cc:0x0] } multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.61905 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.61905 First grouplock blocks second one PASS 107a (12s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 21:15:21 (1788916521) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.553178 s, 19.0 MB/s /mnt/lustre/f107b.sanityn lcm_layout_gen: 3 lcm_mirror_count: 1 lcm_entry_count: 2 lcme_id: 1 lcme_mirror_id: 0 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 0 lmm_stripe_size: 1048576 lmm_pattern: mdt lmm_layout_gen: 0 lmm_mdt_index: 0 lcme_id: 2 lcme_mirror_id: 0 lcme_flags: init lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x240000400:0x3cf:0x0] } multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.61905 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (17s) cleanup: ====================================================== == sanityn test complete, duration 674 sec =============== 21:15:41 (1788916541) === sanityn: start cleanup 21:15:43 (1788916543) === Stopping clients: oleg228-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg228-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 21:15:53 (1788916553) === PASS sanityn (697s) == sanity-dom test complete, duration 2143 sec =========== 21:16:02 (1788916562) === sanity-dom: start cleanup 21:16:04 (1788916564) === Stopping clients: oleg228-client.virtnet /mnt/lustre2 (opts:) === sanity-dom: finish cleanup 21:16:08 (1788916568) ===