-----============= acceptance-small: sanity ============----- Thu Jan 22 04:13:54 EST 2026 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg440-server: /home/green/git/lustre-release/lustre/tests/except/sanity.0f.ex oleg440-server: /home/green/git/lustre-release/lustre/tests/except/sanity.ex - see CLIENT_VERSION > v2_15_63-134-gdacc4b6d38 (34681348 > 34553734) for LU-15963, go 312 - need MDS1_VERSION <= 2.14.55-100-g8a84c7f9c7 (34681348 <= 34486116) for LU-14927, skip 0f - see MDS1_VERSION <= CLIENT_VERSION (34681348 <= 34681348) for LU-18562, go 270a - need MDS1_VERSION < v2_14_55-100-g8a84c7f9c7 (34681348 < 34486116) for LU-14927, skip 0f excepting tests: 42a 42c 42b 118c 118d 407 119i 817 411a 130b 130c 130d 130e 130f 130g 312 skipping tests SLOW=no: 27m 60i 64b 68 71 135 136 230d 300o 842 51b === sanity: start setup 04:14:14 (1769073254) === oleg440-client.virtnet: executing check_config_client /mnt/lustre oleg440-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg440-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff913945145800.idle_timeout=debug osc.lustre-OST0001-osc-ffff913945145800.idle_timeout=debug disable quota as required oleg440-server: oleg440-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanity: finish setup 04:14:41 (1769073281) === running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7563] preparing for tests involving mounts mke2fs 1.47.3-wc2 (11-Nov-2025) debug=all == sanity test 60a: llog_test run from kernel module and test llog_reader ========================================================== 04:14:46 (1769073286) SKIP: sanity test_60a missing subtest run-llog.sh SKIP 60a (5s) == sanity test 60b: limit repeated messages from CERROR/CWARN ========================================================== 04:14:51 (1769073291) PASS 60b (7s) == sanity test 60c: unlink file when mds full ============ 04:14:58 (1769073298) create 5000 files - open/close 436 (time 1769073311.42 total 10.04 last 43.44) - open/close 745 (time 1769073321.43 total 20.04 last 30.89) - open/close 1016 (time 1769073331.43 total 30.05 last 27.08) - open/close 1234 (time 1769073341.43 total 40.05 last 21.80) - open/close 1518 (time 1769073351.45 total 50.06 last 28.35) - open/close 1811 (time 1769073361.45 total 60.06 last 29.29) - open/close 2267 (time 1769073371.46 total 70.07 last 45.57) - open/close 2736 (time 1769073381.47 total 80.08 last 46.86) - open/close 3240 (time 1769073391.47 total 90.09 last 50.36) - open/close 3725 (time 1769073401.48 total 100.10 last 48.45) - open/close 4211 (time 1769073411.49 total 110.11 last 48.56) - open/close 4714 (time 1769073421.55 total 120.16 last 50.02) total: 5000 open/close in 126.69 seconds: 39.47 ops/second fail_loc=0x80000137 - unlinked 0 (time 1769073433 ; total 0 ; last 0) total: 5000 unlinks in 70 seconds: 71.428574 unlinks/second fail_loc=0 PASS 60c (213s) == sanity test 60d: test printk console message masking == 04:18:31 (1769073511) printk=0 emerg PASS 60d (9s) == sanity test 60e: no space while new llog is being created ========================================================== 04:18:40 (1769073520) fail_loc=0x15b PASS 60e (9s) == sanity test 60f: change debug_path works ============== 04:18:49 (1769073529) debug_path=/tmp/f60f.sanity fail_loc=0x8000050e ls: cannot access '/tmp/f60f.sanity*': No such file or directory 0 /tmp/f60f.sanity.1769073532.13651 debug_path=/tmp/lustre-log PASS 60f (9s) == sanity test 60g: transaction abort won't cause MDT hung ========================================================== 04:18:58 (1769073538) /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4954: 14397 Killed ( local index=0; while true; do $LFS setdirstripe -i $(($index % $MDSCOUNT)) -c $MDSCOUNT $DIR/$tdir/subdir$index 2> /dev/null; mkdir $DIR/$tdir/subdir$index 2> /dev/null; rmdir $DIR/$tdir/subdir$index 2> /dev/null; index=$((index + 1)); done ) Started LFSCK on the device lustre-MDT0000: scrub namespace /mnt/lustre/d60g.sanity: subdir103 subdir117 subdir139 subdir145 subdir153 subdir16 subdir175 subdir197 subdir203 subdir210 subdir223 subdir228 subdir235 subdir264 subdir272 subdir28 subdir284 subdir297 subdir312 subdir320 subdir327 subdir341 subdir348 subdir355 subdir36 subdir423 subdir429 subdir436 subdir44 subdir456 subdir462 subdir468 subdir473 subdir497 subdir503 subdir530 subdir537 subdir545 subdir551 subdir558 subdir57 subdir571 subdir619 subdir626 subdir632 subdir648 subdir65 subdir689 subdir89 subdir9 subdir96 /mnt/lustre/d60g.sanity/subdir103: /mnt/lustre/d60g.sanity/subdir117: /mnt/lustre/d60g.sanity/subdir139: /mnt/lustre/d60g.sanity/subdir145: /mnt/lustre/d60g.sanity/subdir153: /mnt/lustre/d60g.sanity/subdir16: /mnt/lustre/d60g.sanity/subdir175: /mnt/lustre/d60g.sanity/subdir197: /mnt/lustre/d60g.sanity/subdir203: /mnt/lustre/d60g.sanity/subdir210: /mnt/lustre/d60g.sanity/subdir223: /mnt/lustre/d60g.sanity/subdir228: /mnt/lustre/d60g.sanity/subdir235: /mnt/lustre/d60g.sanity/subdir264: /mnt/lustre/d60g.sanity/subdir272: /mnt/lustre/d60g.sanity/subdir28: /mnt/lustre/d60g.sanity/subdir284: /mnt/lustre/d60g.sanity/subdir297: /mnt/lustre/d60g.sanity/subdir312: /mnt/lustre/d60g.sanity/subdir320: /mnt/lustre/d60g.sanity/subdir327: /mnt/lustre/d60g.sanity/subdir341: /mnt/lustre/d60g.sanity/subdir348: /mnt/lustre/d60g.sanity/subdir355: /mnt/lustre/d60g.sanity/subdir36: /mnt/lustre/d60g.sanity/subdir423: /mnt/lustre/d60g.sanity/subdir429: /mnt/lustre/d60g.sanity/subdir436: /mnt/lustre/d60g.sanity/subdir44: /mnt/lustre/d60g.sanity/subdir456: /mnt/lustre/d60g.sanity/subdir462: /mnt/lustre/d60g.sanity/subdir468: /mnt/lustre/d60g.sanity/subdir473: /mnt/lustre/d60g.sanity/subdir497: /mnt/lustre/d60g.sanity/subdir503: /mnt/lustre/d60g.sanity/subdir530: /mnt/lustre/d60g.sanity/subdir537: /mnt/lustre/d60g.sanity/subdir545: /mnt/lustre/d60g.sanity/subdir551: /mnt/lustre/d60g.sanity/subdir558: /mnt/lustre/d60g.sanity/subdir57: /mnt/lustre/d60g.sanity/subdir571: /mnt/lustre/d60g.sanity/subdir619: /mnt/lustre/d60g.sanity/subdir626: /mnt/lustre/d60g.sanity/subdir632: /mnt/lustre/d60g.sanity/subdir648: /mnt/lustre/d60g.sanity/subdir65: /mnt/lustre/d60g.sanity/subdir689: /mnt/lustre/d60g.sanity/subdir89: /mnt/lustre/d60g.sanity/subdir9: /mnt/lustre/d60g.sanity/subdir96: PASS 60g (120s) == sanity test 60h: striped directory with missing stripes can be accessed ========================================================== 04:20:58 (1769073658) SKIP: sanity test_60h Need at least 2 MDTs SKIP 60h (4s) SKIP: sanity test_60i skipping SLOW test 60i == sanity test 60j: llog_reader reports corruptions ====== 04:21:04 (1769073664) SKIP: sanity test_60j ldiskfs only test SKIP 60j (5s) == sanity test 61a: mmap() writes don't make sync hang ========================================================================== 04:21:09 (1769073669) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0223642 s, 183 kB/s PASS 61a (8s) == sanity test 61b: mmap() of unstriped file is successful ========================================================== 04:21:17 (1769073677) PASS 61b (8s) == sanity test 63a: Verify oig_wait interruption does not crash ================================================================= 04:21:25 (1769073685) /home/green/git/lustre-release/lustre/tests/sanity.sh: line 10793: 19893 Terminated dd if=/dev/zero of=$DIR/f63 bs=8k /home/green/git/lustre-release/lustre/tests/sanity.sh: line 10793: 19903 Terminated dd if=/dev/zero of=$DIR/f63 bs=8k /home/green/git/lustre-release/lustre/tests/sanity.sh: line 10793: 19907 Terminated dd if=/dev/zero of=$DIR/f63 bs=8k /home/green/git/lustre-release/lustre/tests/sanity.sh: line 10793: 19911 Terminated dd if=/dev/zero of=$DIR/f63 bs=8k /home/green/git/lustre-release/lustre/tests/sanity.sh: line 10793: 19915 Terminated dd if=/dev/zero of=$DIR/f63 bs=8k /home/green/git/lustre-release/lustre/tests/sanity.sh: line 10793: 19919 Terminated dd if=/dev/zero of=$DIR/f63 bs=8k /home/green/git/lustre-release/lustre/tests/sanity.sh: line 10793: 19923 Terminated dd if=/dev/zero of=$DIR/f63 bs=8k /home/green/git/lustre-release/lustre/tests/sanity.sh: line 10793: 19927 Terminated dd if=/dev/zero of=$DIR/f63 bs=8k /home/green/git/lustre-release/lustre/tests/sanity.sh: line 10793: 19931 Terminated dd if=/dev/zero of=$DIR/f63 bs=8k /home/green/git/lustre-release/lustre/tests/sanity.sh: line 10793: 19935 Terminated dd if=/dev/zero of=$DIR/f63 bs=8k checking grant......UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 3072 3766272 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 10240 3757056 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7523328 1% /mnt/lustre pass grant check: client:119275520 server:119275520 PASS 63a (73s) == sanity test 63b: async write errors should be returned to fsync ============================================================= 04:22:38 (1769073758) debug=-1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0344437 s, 119 kB/s fail_loc=0x80000406 fsync: Input/output error checking grant......UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 3072 3766272 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 3072 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 6144 7530496 1% /mnt/lustre pass grant check: client:119275520 server:119275520 PASS 63b (18s) == sanity test 64a: verify filter grant calculations (in kernel) =============================================================== 04:22:56 (1769073776) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 3072 3766272 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 3072 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 6144 7530496 1% /mnt/lustre osc.lustre-OST0000-osc-ffff913945145800.cur_grant_bytes=6815744 osc.lustre-OST0000-osc-ffff913945145800.cur_lost_grant_bytes=1703936 osc.lustre-OST0001-osc-ffff913945145800.cur_grant_bytes=107347968 osc.lustre-OST0001-osc-ffff913945145800.cur_lost_grant_bytes=3407872 checking grant......UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 3072 3766272 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 3072 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 6144 7530496 1% /mnt/lustre pass grant check: client:119275520 server:119275520 PASS 64a (11s) SKIP: sanity test_64b skipping SLOW test 64b == sanity test 64c: verify grant shrink ================== 04:23:09 (1769073789) osc.lustre-OST0000-osc-ffff913945145800.cur_grant_bytes=0 checking grant......UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 3072 3766272 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 3072 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 6144 7530496 1% /mnt/lustre pass grant check: client:117571584 server:117571584 PASS 64c (12s) == sanity test 64d: check grant limit exceed ============= 04:23:21 (1769073801) 1000+0 records in 1000+0 records out 1048576000 bytes (1.0 GB, 1000 MiB) copied, 51.6029 s, 20.3 MB/s /home/green/git/lustre-release/lustre/tests/sanity.sh: line 10915: kill: (22805) - No such process checking grant......UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 1027072 2737152 28% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 3072 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 1030144 6501376 14% /mnt/lustre pass grant check: client:599261184 server:599261184 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 64d (90s) == sanity test 64e: check grant consumption (no grant allocation) ========================================================== 04:24:51 (1769073891) debug=+cache Stopping client oleg440-client.virtnet /mnt/lustre (opts:) Starting client: oleg440-client.virtnet: -o user_xattr,flock 192.168.204.140@tcp:/lustre /mnt/lustre fail_loc=0x725 1+0 records in 1+0 records out 589824 bytes (590 kB, 576 KiB) copied, 0.135287 s, 4.4 MB/s fail_loc=0 Stopping client oleg440-client.virtnet /mnt/lustre (opts:) Starting client: oleg440-client.virtnet: -o user_xattr,flock 192.168.204.140@tcp:/lustre /mnt/lustre fail_loc=0x725 fail_loc=0 PASS 64e (22s) == sanity test 64f: check grant consumption (with grant allocation) ========================================================== 04:25:13 (1769073913) debug=+cache Stopping client oleg440-client.virtnet /mnt/lustre (opts:) Starting client: oleg440-client.virtnet: -o user_xattr,flock 192.168.204.140@tcp:/lustre /mnt/lustre 1+0 records in 1+0 records out 643072 bytes (643 kB, 628 KiB) copied, 0.146914 s, 4.4 MB/s Stopping client oleg440-client.virtnet /mnt/lustre (opts:) Starting client: oleg440-client.virtnet: -o user_xattr,flock 192.168.204.140@tcp:/lustre /mnt/lustre llite.lustre-ffff913942dbd000.hybrid_io=0 fail_loc=0x50a fail_val=3 1+0 records in 1+0 records out 643072 bytes (643 kB, 628 KiB) copied, 0.142579 s, 4.5 MB/s fail_loc=0 fail_val=0 PASS 64f (18s) == sanity test 64g: grant shrink on MDT ================== 04:25:31 (1769073931) 1+0 records in 1+0 records out 131072 bytes (131 kB, 128 KiB) copied, 0.0139861 s, 9.4 MB/s 1+0 records in 1+0 records out 131072 bytes (131 kB, 128 KiB) copied, 0.0150935 s, 8.7 MB/s 1+0 records in 1+0 records out 131072 bytes (131 kB, 128 KiB) copied, 0.0153795 s, 8.5 MB/s 1+0 records in 1+0 records out 131072 bytes (131 kB, 128 KiB) copied, 0.0133007 s, 9.9 MB/s 1+0 records in 1+0 records out 131072 bytes (131 kB, 128 KiB) copied, 0.0541313 s, 2.4 MB/s 1+0 records in 1+0 records out 131072 bytes (131 kB, 128 KiB) copied, 0.012341 s, 10.6 MB/s 1+0 records in 1+0 records out 131072 bytes (131 kB, 128 KiB) copied, 0.0145728 s, 9.0 MB/s 1+0 records in 1+0 records out 131072 bytes (131 kB, 128 KiB) copied, 0.0136344 s, 9.6 MB/s 1+0 records in 1+0 records out 131072 bytes (131 kB, 128 KiB) copied, 0.0145081 s, 9.0 MB/s 1+0 records in 1+0 records out 131072 bytes (131 kB, 128 KiB) copied, 0.0148103 s, 8.9 MB/s 3932160 grants, 0 pages 3932160 grants, 8 pages 3932160 grants, 8 pages 3932160 grants, 8 pages mdc.lustre-MDT0000-mdc-ffff913942dbd000.grant_shrink_interval=5 0 grants, 8 pages 0 grants, 8 pages 0 grants, 8 pages 0 grants, 8 pages 0 grants, 8 pages 0 grants, 8 pages 0 grants, 8 pages 0 grants, 8 pages 0 grants, 8 pages 0 grants, 8 pages 0 grants, 8 pages 0 grants, 8 pages 0 grants, 0 pages mdc.lustre-MDT0000-mdc-ffff913942dbd000.grant_shrink_interval=1200 PASS 64g (54s) == sanity test 64h: grant shrink on read ================= 04:26:25 (1769073985) osc.lustre-OST0000-osc-ffff913942dbd000.grant_shrink=1 osc.lustre-OST0000-osc-ffff913942dbd000.grant_shrink_interval=10 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 3.71659 s, 2.8 MB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0228477 s, 179 kB/s PASS 64h (23s) == sanity test 64i: shrink on reconnect ================== 04:26:48 (1769074008) 64+0 records in 64+0 records out 67108864 bytes (67 MB, 64 MiB) copied, 10.6015 s, 6.3 MB/s fail_loc=0x80000513 fail_val=17 osc.lustre-OST0000-osc-ffff913942dbd000.cur_grant_bytes=64749568B Failing ost1 on oleg440-server Stopping /mnt/lustre-ost1 (opts:) on oleg440-server 04:27:09 (1769074029) shut down facet: ost1 facet_host: oleg440-server facet_failover_host: oleg440-server Failover ost1 to oleg440-server mount facets: ost1 Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg440-server: oleg440-server.virtnet: executing set_default_debug all all pdsh@oleg440-client: oleg440-server: ssh exited with exit code 1 Started lustre-OST0000 04:27:37 (1769074057) targets are mounted 04:27:37 (1769074057) facet_failover done oleg440-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec 8+0 records in 8+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 1.23136 s, 6.8 MB/s PASS 64i (67s) == sanity test 64j: check grants on re-done rpc ========== 04:27:56 (1769074076) fail_loc=0x80000256 checking grant......UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3712 2204928 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 3072 3764224 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 3072 3766272 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 6144 7530496 1% /mnt/lustre pass grant check: client:22544384 server:22544384 PASS 64j (12s) == sanity test 65a: directory with no stripe info ======== 04:28:08 (1769074088) default stripe 1, ost count 2 PASS 65a (8s) == sanity test 65b: directory setstripe -S stripe_size*2 -i 0 -c 1 ========================================================== 04:28:17 (1769074097) dir stripe 1, default stripe 1, ost count 2 PASS 65b (9s) == sanity test 65c: directory setstripe -S stripe_size*4 -i 1 -c 1 ========================================================== 04:28:25 (1769074105) dir stripe 1, default stripe 1, ost count 2 PASS 65c (8s) == sanity test 65d: directory setstripe -S stripe_size -c stripe_count ========================================================== 04:28:33 (1769074113) dir stripe 0, default stripe 1, ost count 2 PASS 65d (8s) == sanity test 65e: directory setstripe defaults ========= 04:28:41 (1769074121) (Default) /mnt/lustre/d65e.sanity default stripe 1, ost count 2 PASS 65e (8s) == sanity test 65f: dir setstripe permission (should return error) ============================================================= 04:28:49 (1769074129) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [setstripe] [/mnt/lustre/d65f.sanityf] lfs setstripe: setstripe error for '/mnt/lustre/d65f.sanityf': Operation not permitted PASS 65f (10s) == sanity test 65g: directory setstripe -d =============== 04:28:59 (1769074139) (Default) /mnt/lustre/d65g.sanity PASS 65g (8s) == sanity test 65h: directory stripe info inherit ============================================================================== 04:29:07 (1769074147) PASS 65h (8s) == sanity test 65i: various tests to set root directory striping ========================================================== 04:29:15 (1769074155) /mnt/lustre stripe_count: -1 stripe_size: 65536 pattern: raid0 stripe_offset: -1 /mnt/lustre/f64j.sanity lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2513 0x9d1 0x240000400 /mnt/lustre/d65a.sanity stripe_count: -1 stripe_size: 65536 pattern: raid0 stripe_offset: -1 /mnt/lustre/f63b.sanity lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2505 0x9c9 0x280000400 /mnt/lustre/d65c.sanity stripe_count: 1 stripe_size: 16777216 pattern: raid0 stripe_offset: 1 /mnt/lustre/d65e.sanity stripe_count: -1 stripe_size: 65536 pattern: raid0 stripe_offset: -1 /mnt/lustre/d65h.sanity stripe_count: 1 stripe_size: 8388608 pattern: raid0 stripe_offset: 0 /mnt/lustre/d65g.sanity stripe_count: -1 stripe_size: 65536 pattern: raid0 stripe_offset: -1 /mnt/lustre/d65d.sanity stripe_count: 1 stripe_size: 4194304 pattern: raid0 stripe_offset: -1 /mnt/lustre/f60b.sanity lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2 0x2 0x240000400 /mnt/lustre/d65f.sanityf stripe_count: -1 stripe_size: 65536 pattern: raid0 stripe_offset: -1 /mnt/lustre/d60f.sanity stripe_count: -1 stripe_size: 65536 pattern: raid0 stripe_offset: -1 /mnt/lustre/f61 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2504 0x9c8 0x240000400 /mnt/lustre/d65b.sanity stripe_count: 1 stripe_size: 8388608 pattern: raid0 stripe_offset: 0 /mnt/lustre/f61b.sanity has no stripe info /mnt/lustre lmm_fid: [0x200000007:0x1:0x0] stripe_count: -1 stripe_size: 65536 pattern: raid0 stripe_offset: -1 /mnt/lustre/f64j.sanity lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000406 lmm_object_id: 0x1c lmm_fid: [0x200000406:0x1c:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2513 0x9d1 0x240000400 /mnt/lustre/d65a.sanity lmm_fid: [0x200000406:0x1d:0x0] stripe_count: -1 stripe_size: 65536 pattern: raid0 stripe_offset: -1 /mnt/lustre/f63b.sanity lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000401 lmm_object_id: 0x1901 lmm_fid: [0x200000401:0x1901:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2505 0x9c9 0x280000400 /mnt/lustre/d65c.sanity lmm_fid: [0x200000406:0x21:0x0] stripe_count: 1 stripe_size: 16777216 pattern: raid0 stripe_offset: 1 /mnt/lustre/d65e.sanity lmm_fid: [0x200000406:0x26:0x0] stripe_count: -1 stripe_size: 65536 pattern: raid0 stripe_offset: -1 /mnt/lustre/d65h.sanity lmm_fid: [0x200000406:0x2c:0x0] stripe_count: 1 stripe_size: 8388608 pattern: raid0 stripe_offset: 0 /mnt/lustre/d65g.sanity lmm_fid: [0x200000406:0x2a:0x0] stripe_count: -1 stripe_size: 65536 pattern: raid0 stripe_offset: -1 /mnt/lustre/d65d.sanity lmm_fid: [0x200000406:0x23:0x0] stripe_count: 1 stripe_size: 4194304 pattern: raid0 stripe_offset: -1 /mnt/lustre/f60b.sanity lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000401 lmm_object_id: 0x3 lmm_fid: [0x200000401:0x3:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2 0x2 0x240000400 /mnt/lustre/d65f.sanityf lmm_fid: [0x200000406:0x29:0x0] stripe_count: -1 stripe_size: 65536 pattern: raid0 stripe_offset: -1 /mnt/lustre/d60f.sanity lmm_fid: [0x200000401:0x138d:0x0] stripe_count: -1 stripe_size: 65536 pattern: raid0 stripe_offset: -1 /mnt/lustre/f61 lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000401 lmm_object_id: 0x18f5 lmm_fid: [0x200000401:0x18f5:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2504 0x9c8 0x240000400 /mnt/lustre/d65b.sanity lmm_fid: [0x200000406:0x1f:0x0] stripe_count: 1 stripe_size: 8388608 pattern: raid0 stripe_offset: 0 /mnt/lustre/f61b.sanity has no stripe info PASS 65i (11s) == sanity test 65j: set default striping on root directory (bug 6367)=========================================================== 04:29:26 (1769074166) PASS 65j (9s) == sanity test 65k: validate manual striping works properly with deactivated OSCs ========================================================== 04:29:35 (1769074175) Check OST status: lustre-OST0000-osc-MDT0000 is active lustre-OST0001-osc-MDT0000 is active - open/close 906 (time 1769074196.01 total 10.00 last 90.58) total: 1000 open/close in 11.39 seconds: 87.80 ops/second Deactivate: lustre-OST0000-osc-MDT0000 /home/green/git/lustre-release/lustre/utils/lfs setstripe -i 0 -c 1 /mnt/lustre/d65k.sanity/0 /home/green/git/lustre-release/lustre/utils/lfs setstripe -i 1 -c 1 /mnt/lustre/d65k.sanity/1 - unlinked 0 (time 1769074208 ; total 0 ; last 0) total: 1000 unlinks in 8 seconds: 125.000000 unlinks/second lustre-OST0000-osc-MDT0000 is Activate oleg440-server: oleg440-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg440-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec - open/close 792 (time 1769074241.38 total 10.00 last 79.19) total: 1000 open/close in 12.75 seconds: 78.42 ops/second Deactivate: lustre-OST0001-osc-MDT0000 /home/green/git/lustre-release/lustre/utils/lfs setstripe -i 0 -c 1 /mnt/lustre/d65k.sanity/0 /home/green/git/lustre-release/lustre/utils/lfs setstripe -i 1 -c 1 /mnt/lustre/d65k.sanity/1 - unlinked 0 (time 1769074253 ; total 0 ; last 0) total: 1000 unlinks in 7 seconds: 142.857147 unlinks/second lustre-OST0001-osc-MDT0000 is Activate oleg440-server: oleg440-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 oleg440-server: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec PASS 65k (101s) == sanity test 65l: lfs find on -1 stripe dir ================================================================================== 04:31:17 (1769074277) PASS 65l (8s) == sanity test 65m: normal user can't set filesystem default stripe ========================================================== 04:31:25 (1769074285) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [setstripe] [-c] [2] [/mnt/lustre] lfs setstripe: setstripe error for '/mnt/lustre': Operation not permitted PASS 65m (7s) == sanity test 65n: don't inherit default layout from root for new subdirectories ========================================================== 04:31:33 (1769074293) Creating new pool test_65n oleg440-server: Pool lustre.test_65n created Adding targets to pool oleg440-server: OST lustre-OST0000_UUID added to pool lustre.test_65n oleg440-server: OST lustre-OST0001_UUID added to pool lustre.test_65n Waiting 90s for 'lustre-OST0000_UUID lustre-OST0001_UUID ' /home/green/git/lustre-release/lustre/utils/lfs getstripe -d /mnt/lustre/d65n.sanity-4 stripe_count: 1 stripe_size: 4194304 pattern: raid0 stripe_offset: -1 pool: test_65n /home/green/git/lustre-release/lustre/utils/lfs getstripe -d /mnt/lustre stripe_count: 1 stripe_size: 4194304 pattern: raid0 stripe_offset: -1 pool: test_65n SKIP: sanity test_65n needs >= 2 MDTs Destroy the created pools: test_65n lustre.test_65n oleg440-server: OST lustre-OST0000_UUID removed from pool lustre.test_65n oleg440-server: OST lustre-OST0001_UUID removed from pool lustre.test_65n oleg440-server: Pool lustre.test_65n destroyed SKIP 65n (28s) == sanity test 65o: pool inheritance for mdt component === 04:32:01 (1769074321) Creating new pool test_65o oleg440-server: Pool lustre.test_65o created Adding targets to pool oleg440-server: OST lustre-OST0000_UUID added to pool lustre.test_65o oleg440-server: OST lustre-OST0001_UUID added to pool lustre.test_65o /mnt/lustre/d65o.sanity lcm_layout_gen: 0 lcm_mirror_count: 1 lcm_entry_count: 2 lcme_id: N/A lcme_mirror_id: N/A lcme_flags: 0 lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 stripe_count: 0 stripe_size: 1048576 pattern: mdt mdt_index: -1 lcme_id: N/A lcme_mirror_id: N/A lcme_flags: 0 lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF stripe_count: 1 stripe_size: 4194304 pattern: raid0 stripe_offset: -1 pool: test_65o /mnt/lustre/d65o.sanity/dir2 lcm_layout_gen: 0 lcm_mirror_count: 1 lcm_entry_count: 2 lcme_id: N/A lcme_mirror_id: N/A lcme_flags: 0 lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 stripe_count: 0 stripe_size: 1048576 pattern: mdt mdt_index: -1 lcme_id: N/A lcme_mirror_id: N/A lcme_flags: 0 lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF stripe_count: 1 stripe_size: 4194304 pattern: raid0 stripe_offset: -1 pool: test_65o lcm_layout_gen: 0 lcm_mirror_count: 1 lcm_entry_count: 1 lcme_id: N/A lcme_mirror_id: N/A lcme_flags: 0 lcme_extent.e_start: 0 lcme_extent.e_end: EOF stripe_count: 1 stripe_size: 4194304 pattern: raid0 stripe_offset: -1 pool: test_65o Destroy the created pools: test_65o lustre.test_65o oleg440-server: OST lustre-OST0000_UUID removed from pool lustre.test_65o oleg440-server: OST lustre-OST0001_UUID removed from pool lustre.test_65o oleg440-server: Pool lustre.test_65o destroyed PASS 65o (29s) == sanity test 65p: setstripe with yaml file and huge number ========================================================== 04:32:30 (1769074350) PASS 65p (9s) == sanity test 65q: setstripe with >=8E offset should fail ========================================================== 04:32:39 (1769074359) lfs setstripe: lfs setstripe: cannot set default composite layout for '/mnt/lustre/d65q.sanity/src_dir': Invalid argument (22) PASS 65q (8s) == sanity test 65r: prevent all-zero offsets ============= 04:32:47 (1769074367) PASS 65r (10s) == sanity test 66: update inode blocks count on client ========================================================================= 04:32:57 (1769074377) 8+0 records in 8+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0259898 s, 315 kB/s PASS 66 (14s) == sanity test 69: verify oa2dentry return -ENOENT doesn't LBUG ================================================================ 04:33:11 (1769074391) directio on /mnt/lustre/f69.sanity.2 for 1x4194304 bytes PASS fail_loc=0x217 directio on /mnt/lustre/f69.sanity for 2x4194304 bytes Write error No such file or directory (rc = -1, len = 8388608) fail_loc=0 directio on /mnt/lustre/f69.sanity for 2x4194304 bytes PASS directio on /mnt/lustre/f69.sanity for 1x4194304 bytes PASS fail_loc=0x217 directio on /mnt/lustre/f69.sanity for 1x4194304 bytes Read error: No such file or directory rc = -1 fail_loc=0 PASS 69 (14s) == sanity test 70a: verify health_check, health_write don't explode (on OST) ========================================================== 04:33:25 (1769074405) enable_health_write=off enable_health_write=0 enable_health_write=on enable_health_write=1 enable_health_write=0 PASS 70a (17s) SKIP: sanity test_71 skipping SLOW test 71 == sanity test 72a: Test that remove suid works properly (bug5695) ============================================================== 04:33:44 (1769074424) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7563] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/f72a.sanity] [bs=512] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.00669209 s, 76.5 kB/s PASS 72a (9s) == sanity test 72b: Test that we keep mode setting if without file data changed (bug 24226) ========================================================== 04:33:53 (1769074433) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7563] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/f72b.sanity-fg] chmod: changing permissions of '/mnt/lustre/f72b.sanity-fg': Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/f72b.sanity-fu] chmod: changing permissions of '/mnt/lustre/f72b.sanity-fu': Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/f72b.sanity-dg] chmod: changing permissions of '/mnt/lustre/f72b.sanity-dg': Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/f72b.sanity-du] chmod: changing permissions of '/mnt/lustre/f72b.sanity-du': Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [2777] [/mnt/lustre/f72b.sanity-fg] chmod: changing permissions of '/mnt/lustre/f72b.sanity-fg': Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [2777] [/mnt/lustre/f72b.sanity-fu] chmod: changing permissions of '/mnt/lustre/f72b.sanity-fu': Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [2777] [/mnt/lustre/f72b.sanity-dg] chmod: changing permissions of '/mnt/lustre/f72b.sanity-dg': Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [2777] [/mnt/lustre/f72b.sanity-du] chmod: changing permissions of '/mnt/lustre/f72b.sanity-du': Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [4777] [/mnt/lustre/f72b.sanity-fg] chmod: changing permissions of '/mnt/lustre/f72b.sanity-fg': Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [4777] [/mnt/lustre/f72b.sanity-fu] chmod: changing permissions of '/mnt/lustre/f72b.sanity-fu': Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [4777] [/mnt/lustre/f72b.sanity-dg] chmod: changing permissions of '/mnt/lustre/f72b.sanity-dg': Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [4777] [/mnt/lustre/f72b.sanity-du] chmod: changing permissions of '/mnt/lustre/f72b.sanity-du': Operation not permitted PASS 72b (10s) == sanity test 73: multiple MDC requests (should not deadlock) ========================================================== 04:34:03 (1769074443) multiop /mnt/lustre/d73-1/f73-1 vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7563 fail_loc=0x80000129 fail_loc=0 /mnt/lustre/d73-1/f73-1 has type file OK /mnt/lustre/d73-1/f73-2 has type file OK /mnt/lustre/d73-2/f73-3 has type file OK PASS 73 (35s) == sanity test 74a: ldlm_enqueue freed-export error path, ls (shouldn't LBUG) ========================================================== 04:34:38 (1769074478) fail_loc=0x8000030e /mnt/lustre/f74a fail_loc=0 PASS 74a (8s) == sanity test 74b: ldlm_enqueue freed-export error path, touch (shouldn't LBUG) ========================================================== 04:34:46 (1769074486) fail_loc=0x8000030e fail_loc=0 PASS 74b (8s) == sanity test 74c: ldlm_lock_create error path, (shouldn't LBUG) ========================================================== 04:34:54 (1769074494) fail_loc=0x319 touch: cannot touch '/mnt/lustre/f74c.sanity': No such file or directory fail_loc=0 PASS 74c (7s) == sanity test 76a: confirm clients recycle inodes properly ============================================================== 04:35:02 (1769074502) before slab objects: 45 created: 512, after slab objects: 45 PASS 76a (80s) == sanity test 76b: confirm clients recycle directory inodes properly ============================================================== 04:36:21 (1769074581) slab objects before: 45, after: 45 PASS 76b (53s) == sanity test 77a: normal checksum read/write operation ========================================================== 04:37:14 (1769074634) 8+0 records in 8+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.3983 s, 21.1 MB/s 8+0 records in 8+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 1.11478 s, 7.5 MB/s PASS 77a (10s) == sanity test 77b: checksum error on client write, read ========================================================== 04:37:24 (1769074644) fail_loc=0x80000409 8+0 records in 8+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 1.36123 s, 6.2 MB/s fail_loc=0 set checksum type to crc32, rc = 0 fail_loc=0x80000408 fail_loc=0 set checksum type to adler, rc = 0 fail_loc=0x80000408 fail_loc=0 set checksum type to crc32c, rc = 0 fail_loc=0x80000408 fail_loc=0 set checksum type to t10ip512, rc = 0 fail_loc=0x80000408 fail_loc=0 set checksum type to t10ip4K, rc = 0 fail_loc=0x80000408 fail_loc=0 set checksum type to t10crc512, rc = 0 fail_loc=0x80000408 fail_loc=0 set checksum type to t10crc4K, rc = 0 fail_loc=0x80000408 fail_loc=0 set checksum type to crc32c, rc = 0 PASS 77b (51s) == sanity test 77c: checksum error on client read with debug ========================================================== 04:38:15 (1769074695) debug_path=/tmp/lustre-log 8+0 records in 8+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 1.30887 s, 6.4 MB/s osc.lustre-OST0000-osc-ffff913942dbd000.checksum_dump=1 osc.lustre-OST0001-osc-ffff913942dbd000.checksum_dump=1 obdfilter.lustre-OST0000.checksum_dump=1 obdfilter.lustre-OST0001.checksum_dump=1 fail_loc=0x80000408 8+0 records in 8+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 3.01116 s, 2.8 MB/s fail_loc=0 osc.lustre-OST0000-osc-ffff913942dbd000.checksum_dump=0 osc.lustre-OST0001-osc-ffff913942dbd000.checksum_dump=0 obdfilter.lustre-OST0000.checksum_dump=0 obdfilter.lustre-OST0001.checksum_dump=0 PASS 77c (47s) == sanity test 77d: checksum error on OST direct write, read ========================================================== 04:39:02 (1769074742) fail_loc=0x80000409 directio on /mnt/lustre/f77d.sanity for 8x1048576 bytes PASS fail_loc=0 fail_loc=0x80000408 directio on /mnt/lustre/f77d.sanity for 8x1048576 bytes PASS fail_loc=0 PASS 77d (12s) == sanity test 77f: repeat checksum error on write (expect error) ========================================================== 04:39:14 (1769074754) set checksum type to crc32, rc = 0 fail_loc=0x409 directio on /mnt/lustre/f77f.sanity for 8x1048576 bytes Write error Input/output error (rc = -1, len = 8388608) fail_loc=0 set checksum type to adler, rc = 0 fail_loc=0x409 directio on /mnt/lustre/f77f.sanity for 8x1048576 bytes Write error Input/output error (rc = -1, len = 8388608) fail_loc=0 set checksum type to crc32c, rc = 0 fail_loc=0x409 directio on /mnt/lustre/f77f.sanity for 8x1048576 bytes Write error Input/output error (rc = -1, len = 8388608) fail_loc=0 set checksum type to t10ip512, rc = 0 fail_loc=0x409