-----============= acceptance-small: sanity ============----- Wed Jun 3 18:03:38 EDT 2026 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg312-server: /home/green/git/lustre-release/lustre/tests/except/sanity.56aa.ex oleg312-server: /home/green/git/lustre-release/lustre/tests/except/sanity.ex - see CLIENT_VERSION > v2_15_63-134-gdacc4b6d38 (34682169 > 34553734) for LU-15963, go 312 - see CLIENT_VERSION > v2_15_63-239-gab4ede45b4 (34682169 > 34553839) for LU-17856, go 56wa 56xe - see CLIENT_VERSION > v2_16_54-54-ga2e3a2f5a3 (34682169 > 34616886) for LU-14590, go 0d - see CLIENT_VERSION >= v2_17_50-196-g63ee4af3bb (34682169 >= 34681540) for LU-19620, go 60b - see CLIENT_VERSION >= v2_17_50-71-g2cbe29c282 (34682169 >= 34681415) for LU-19644, go 150ib - see CLIENT_VERSION >= v2_17_51-74 (34682169 >= 34681674) for LU-20010, go 56aa - need MDS1_VERSION <= 2.16.61-1-g89cf292a8c2 (34682169 <= 34618625) for LU-18938, skip 360 - see MDS1_VERSION <= CLIENT_VERSION (34682169 <= 34682169) for LU-18562, go 270a - need MDS1_VERSION < v2_14_55-100-g8a84c7f9c7 (34682169 < 34486116) for LU-14927, skip 0f excepting tests: 42a 42c 42b 118c 118d 407 119i 817 411a 130b 130c 130d 130e 130f 130g skipping tests SLOW=no: 27m 60i 64b 68 71 135 136 230d 300o 842 51b 51c 51e 834 === sanity: start setup 18:03:49 (1780524229) === oleg312-client.virtnet: executing check_config_client /mnt/lustre oleg312-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg312-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8f3cc8727800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8f3cc8727800.idle_timeout=debug disable quota as required oleg312-server: oleg312-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanity: finish setup 18:04:03 (1780524243) === running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7562] preparing for tests involving mounts mke2fs 1.47.3-wc2 (11-Nov-2025) debug=all == sanity test 33h: temp file is located on the same MDT as target (crush) ========================================================== 18:04:05 (1780524245) SKIP: sanity test_33h needs >= 2 MDTs SKIP 33h (2s) == sanity test 33hh: temp file is located on the same MDT as target (crush2) ========================================================== 18:04:07 (1780524247) SKIP: sanity test_33hh needs >= 2 MDTs SKIP 33hh (1s) == sanity test 51b: exceed 64k subdirectory nlink limit on create, verify unlink ========================================================== 18:04:09 (1780524249) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 2048 2206592 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 2048 3767296 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 2048 3767296 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 4096 7534592 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 534562 332 534230 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118185 393 117792 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118185 393 117792 1% /mnt/lustre[OST:1] filesystem_summary: 235916 332 235584 1% /mnt/lustre - mkdir 1174 (time 1780524261.43 total 10.00 last 117.39) - mkdir 2525 (time 1780524271.43 total 20.00 last 135.07) - mkdir 4040 (time 1780524281.48 total 30.05 last 150.77) - mkdir 5466 (time 1780524291.48 total 40.05 last 142.55) - mkdir 6861 (time 1780524301.49 total 50.06 last 139.46) - mkdir 8140 (time 1780524311.49 total 60.06 last 127.85) - mkdir 9541 (time 1780524321.49 total 70.06 last 140.09) - mkdir 10000 (time 1780524324.82 total 73.39 last 138.03) - mkdir 11189 (time 1780524334.82 total 83.39 last 118.83) - mkdir 12418 (time 1780524344.82 total 93.40 last 122.89) - mkdir 13614 (time 1780524354.82 total 103.40 last 119.60) - mkdir 14763 (time 1780524364.83 total 113.40 last 114.87) - mkdir 15920 (time 1780524374.83 total 123.40 last 115.67) - mkdir 17096 (time 1780524384.83 total 133.40 last 117.58) - mkdir 18236 (time 1780524394.84 total 143.41 last 113.93) - mkdir 19335 (time 1780524404.84 total 153.41 last 109.84) - mkdir 20000 (time 1780524411.18 total 159.75 last 104.99) - mkdir 20999 (time 1780524421.18 total 169.75 last 99.89) - mkdir 22027 (time 1780524431.19 total 179.76 last 102.69) - mkdir 23121 (time 1780524441.20 total 189.77 last 109.30) - mkdir 24195 (time 1780524451.20 total 199.77 last 107.34) - mkdir 25222 (time 1780524461.20 total 209.78 last 102.67) - mkdir 26270 (time 1780524471.22 total 219.79 last 104.67) - mkdir 27263 (time 1780524481.22 total 229.79 last 99.28) - mkdir 28256 (time 1780524491.22 total 239.80 last 99.26) - mkdir 29258 (time 1780524501.23 total 249.80 last 100.13) - mkdir 30000 (time 1780524508.49 total 257.06 last 102.20) - mkdir 31047 (time 1780524518.50 total 267.07 last 104.59) - mkdir 32095 (time 1780524528.50 total 277.08 last 104.77) total: 32818 mkdir in 284.04 seconds: 115.54 ops/second - mkdir 1160 (time 1780524261.48 total 10.01 last 115.90) - mkdir 2510 (time 1780524271.49 total 20.01 last 134.92) - mkdir 4067 (time 1780524281.51 total 30.04 last 155.35) - mkdir 5467 (time 1780524291.52 total 40.04 last 139.92) - mkdir 6807 (time 1780524301.52 total 50.04 last 134.00) - mkdir 8080 (time 1780524311.52 total 60.04 last 127.28) - mkdir 9515 (time 1780524321.54 total 70.06 last 143.20) - mkdir 10000 (time 1780524325.26 total 73.78 last 130.44) - mkdir 11195 (time 1780524335.26 total 83.79 last 119.45) - mkdir 12421 (time 1780524345.28 total 93.81 last 122.34) - mkdir 13636 (time 1780524355.29 total 103.81 last 121.43) - mkdir 14815 (time 1780524365.29 total 113.82 last 117.88) - mkdir 15971 (time 1780524375.29 total 123.82 last 115.58) - mkdir 17153 (time 1780524385.29 total 133.82 last 118.16) - mkdir 18255 (time 1780524395.30 total 143.82 last 110.17) - mkdir 19409 (time 1780524405.30 total 153.83 last 115.33) - mkdir 20000 (time 1780524411.18 total 159.71 last 100.56) - mkdir 20963 (time 1780524421.18 total 169.71 last 96.29) - mkdir 22002 (time 1780524431.18 total 179.71 last 103.87) - mkdir 23084 (time 1780524441.19 total 189.72 last 108.14) - mkdir 24130 (time 1780524451.19 total 199.72 last 104.56) - mkdir 25148 (time 1780524461.19 total 209.72 last 101.80) - mkdir 26210 (time 1780524471.19 total 219.72 last 106.20) - mkdir 27212 (time 1780524481.20 total 229.73 last 100.14) - mkdir 28218 (time 1780524491.20 total 239.73 last 100.60) - mkdir 29197 (time 1780524501.20 total 249.73 last 97.90) - mkdir 30000 (time 1780524508.89 total 257.41 last 104.45) - mkdir 31059 (time 1780524518.91 total 267.43 last 105.69) - mkdir 32076 (time 1780524528.91 total 277.44 last 101.64) total: 32818 mkdir in 284.48 seconds: 115.36 ops/second UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210176 489728 1718400 23% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 2048 3767296 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 2048 3767296 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 4096 7534592 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 304126 65969 238157 22% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118185 393 117792 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118184 392 117792 1% /mnt/lustre[OST:1] filesystem_summary: 301553 65969 235584 22% /mnt/lustre nlink before: 65638, created before: 65637 - unlinked 0 (time 1780524824 ; total 0 ; last 0) - unlinked 10000 (time 1780524898 ; total 74 ; last 74) - unlinked 20000 (time 1780524974 ; total 150 ; last 76) - unlinked 30000 (time 1780525050 ; total 226 ; last 76) total: 32768 unlinks in 246 seconds: 133.203247 unlinks/second - unlinked 0 (time 1780524824 ; total 0 ; last 0) - unlinked 10000 (time 1780524899 ; total 75 ; last 75) - unlinked 20000 (time 1780524975 ; total 151 ; last 76) - unlinked 30000 (time 1780525051 ; total 227 ; last 76) total: 32768 unlinks in 247 seconds: 132.663971 unlinks/second nlink between: 102 - unlinked 0 (time 1780525072 ; total 0 ; last 0) total: 50 unlinks in 1 seconds: 50.000000 unlinks/second - unlinked 0 (time 1780525072 ; total 0 ; last 0) total: 50 unlinks in 1 seconds: 50.000000 unlinks/second nlink after: 2 rmdir(/mnt/lustre/d51b.sanity/d.1.0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second rmdir(/mnt/lustre/d51b.sanity/d.0.0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2209024 2176 2204800 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 2048 3767296 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 2048 3767296 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 4096 7534592 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 526856 333 526523 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118185 393 117792 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118184 392 117792 1% /mnt/lustre[OST:1] filesystem_summary: 235917 333 235584 1% /mnt/lustre PASS 51b (829s) == sanity test 51f: check many open files limit ========== 18:17:58 (1780525078) ulimit_old=1024 numfree= 526523, max=100000 changed ulimit from 1024 to 100100 - open/keep 2595 (time 1780525090.36 total 10.00 last 259.45) - open/keep 5259 (time 1780525100.36 total 20.00 last 266.39) - open/keep 7892 (time 1780525110.36 total 30.00 last 263.29) - open/keep 10000 (time 1780525117.44 total 37.08 last 297.77) - open/keep 12957 (time 1780525127.44 total 47.08 last 295.66) - open/keep 15569 (time 1780525137.45 total 57.09 last 261.14) - open/keep 18040 (time 1780525147.45 total 67.09 last 247.09) - open/keep 20000 (time 1780525155.17 total 74.81 last 253.66) - open/keep 22962 (time 1780525165.18 total 84.82 last 296.11) - open/keep 25866 (time 1780525175.18 total 94.82 last 290.38) - open/keep 28564 (time 1780525185.18 total 104.82 last 269.74) - open/keep 30000 (time 1780525190.94 total 110.58 last 249.48) total: 32451 open/keep in 120.00 seconds: 270.42 ops/second - closed 4210 (time 1780525210.37 total 10.00 last -2578.86) - closed 9038 (time 1780525220.37 total 20.00 last 482.72) - closed 10000 (time 1780525222.29 total 21.93 last 500.20) - closed 14918 (time 1780525232.29 total 31.93 last 491.72) - closed 20000 (time 1780525242.05 total 41.69 last 520.70) - closed 24660 (time 1780525252.05 total 51.69 last 465.93) - closed 28959 (time 1780525262.05 total 61.69 last 429.85) - closed 30000 (time 1780525264.54 total 64.18 last 418.82) total: 32451 close in 69.30 seconds: 468.28 close/second - unlinked 0 (time 1780525274 ; total 0 ; last 0) - unlinked 10000 (time 1780525306 ; total 32 ; last 32) - unlinked 20000 (time 1780525338 ; total 64 ; last 32) - unlinked 30000 (time 1780525364 ; total 90 ; last 26) unlink(/mnt/lustre/d51f.sanity/f32451) error: No such file or directory total: 32451 unlinks in 97 seconds: 334.546387 unlinks/second PASS 51f (294s) == sanity test 57b: default LOV EAs are stored inside large inodes ============================================================= 18:22:53 (1780525373) SKIP: sanity test_57b ldiskfs only test SKIP 57b (1s) resend_count is set to 4 4 resend_count is set to 4 4 resend_count is set to 4 4 resend_count is set to 4 4 resend_count is set to 4 4 == sanity test 160i: changelog user register/unregister race ========================================================== 18:22:56 (1780525376) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl1' fail_loc=0x10001315 fail_val=1 lustre-MDT0000: clear the changelog for cl1 of all records mdd.lustre-MDT0000.changelog_mask=+hsm lustre-MDT0000: Deregistered changelog user #1 Registered 1 changelog users: 'cl1 cl2' cl2 3 (1) verify changelogs added: 3->7 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 mdd.lustre-MDT0000.changelog_mask=MARK lustre-MDT0000: changelog user 'cl1' not found mdd.lustre-MDT0000.changelog_mask=MARK PASS 160i (12s) == sanity test 230o: dir split =========================== 18:23:09 (1780525389) SKIP: sanity test_230o needs >= 2 MDTs SKIP 230o (1s) == sanity test 230p: dir merge =========================== 18:23:10 (1780525390) SKIP: sanity test_230p needs >= 2 MDTs SKIP 230p (2s) == sanity test 230q: dir auto split ====================== 18:23:12 (1780525392) SKIP: sanity test_230q needs >= 2 MDTs SKIP 230q (1s) == sanity test 230x: dir migration check space =========== 18:23:13 (1780525393) SKIP: sanity test_230x needs >= 2 MDTs SKIP 230x (2s) == sanity test 270a: DoM: basic functionality tests ====== 18:23:15 (1780525395) 192+0 records in 192+0 records out 196608 bytes (197 kB, 192 KiB) copied, 0.0756293 s, 2.6 MB/s 3+0 records in 3+0 records out 196608 bytes (197 kB, 192 KiB) copied, 0.0724053 s, 2.7 MB/s 1984+0 records in 1984+0 records out 2031616 bytes (2.0 MB, 1.9 MiB) copied, 0.709575 s, 2.9 MB/s 31+0 records in 31+0 records out 2031616 bytes (2.0 MB, 1.9 MiB) copied, 0.8179 s, 2.5 MB/s PASS 270a (6s) == sanity test 272b: DoM migration: DOM file to the OST-striped file (plain) ========================================================== 18:23:22 (1780525402) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.10577 s, 19.8 MB/s PASS 272b (4s) == sanity test 272c: DoM migration: DOM file to the OST-striped file (composite) ========================================================== 18:23:26 (1780525406) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0889027 s, 23.6 MB/s PASS 272c (4s) == sanity test 272d: DoM mirroring: OST-striped mirror to DOM file ========================================================== 18:23:30 (1780525410) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0869488 s, 24.1 MB/s lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000401:0x17f3f:0x0]: Invalid argument (22) PASS 272d (4s) == sanity test 300ug: migrate overstriped dirs =========== 18:23:35 (1780525415) SKIP: sanity test_300ug needs >= 2 MDTs SKIP 300ug (1s) == sanity test 808: Check trusted.som xattr not logged in Changelogs ========================================================== 18:23:37 (1780525417) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl3' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0459564 s, 22.8 MB/s lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000: Deregistered changelog user #3 mdd.lustre-MDT0000.changelog_mask=MARK PASS 808 (6s) == sanity test complete, duration 1204 sec =============== 18:23:43 (1780525423) === sanity: start cleanup 18:23:44 (1780525424) === === sanity: finish cleanup 18:23:45 (1780525425) === debug=super ioctl neterror warning dlmtrace error emerg ha rpctrace vfstrace config console lfsck