-----============= acceptance-small: ost-pools ============----- Tue Apr 14 14:55:47 EDT 2026 oleg353-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/ost-pools.*ex': No such file or directory excepting tests: oleg353-client.virtnet: executing check_config_client /mnt/lustre oleg353-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg353-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff91c408758000.idle_timeout=debug osc.lustre-OST0001-osc-ffff91c408758000.idle_timeout=debug disable quota as required oleg353-server: oleg353-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 stripe_count: 1 stripe_size: 1048576 pattern: 0 stripe_offset: -1 == ost-pools test 1a: Create a pool with a 1 character pool name ========================================================== 14:56:27 (1776192987) oleg353-server: Pool lustre.p created lustre.p oleg353-server: Pool lustre.p destroyed PASS 1a (19s) == ost-pools test 1b: Create a pool with a 10 char pool name ========================================================== 14:56:46 (1776193006) oleg353-server: Pool lustre.testpool12 created lustre.testpool12 oleg353-server: Pool lustre.testpool12 destroyed PASS 1b (20s) == ost-pools test 1c: Create a pool with a 15 char pool name ========================================================== 14:57:06 (1776193026) oleg353-server: Pool lustre.testpool1234567 created lustre.testpool1234567 oleg353-server: Pool lustre.testpool1234567 destroyed PASS 1c (19s) == ost-pools test 1d: Create a pool with a 16 char pool name; should fail ========================================================== 14:57:25 (1776193045) oleg353-server: error: pool_new: either the pool or file system name is too long (max pool name len is 15 and file system name is 8) oleg353-server: pool_new: File name too long pdsh@oleg353-client: oleg353-server: ssh exited with exit code 36 PASS 1d (8s) == ost-pools test 1e: Create a pool with a 1000 char pool name; should fail ========================================================== 14:57:33 (1776193053) oleg353-server: error: pool_new: either the pool or file system name is too long (max pool name len is 15 and file system name is 8) oleg353-server: pool_new: File name too long pdsh@oleg353-client: oleg353-server: ssh exited with exit code 36 PASS 1e (6s) == ost-pools test 1f: pool_new should fail if fs-name is missing ========================================================== 14:57:40 (1776193060) oleg353-server: fsname is empty oleg353-server: argument .testpool must be . oleg353-server: pool_new: Invalid argument pdsh@oleg353-client: oleg353-server: ssh exited with exit code 22 PASS 1f (7s) == ost-pools test 1g: pool_new should fail if fs-name is missing ========================================================== 14:57:46 (1776193066) oleg353-server: . is missing in testpool oleg353-server: argument testpool must be . oleg353-server: pool_new: Invalid argument pdsh@oleg353-client: oleg353-server: ssh exited with exit code 22 PASS 1g (7s) == ost-pools test 1h: pool_new should fail if poolname is missing ========================================================== 14:57:53 (1776193073) oleg353-server: poolname is empty oleg353-server: argument lustre. must be . oleg353-server: pool_new: Invalid argument pdsh@oleg353-client: oleg353-server: ssh exited with exit code 22 PASS 1h (6s) == ost-pools test 1i: pool_new should fail if poolname and fs-name are missing ========================================================== 14:57:59 (1776193079) oleg353-server: fsname is empty oleg353-server: argument . must be . oleg353-server: pool_new: Invalid argument pdsh@oleg353-client: oleg353-server: ssh exited with exit code 22 PASS 1i (7s) == ost-pools test 1j: pool_new should fail if poolname format is wrong ========================================================== 14:58:06 (1776193086) oleg353-server: . is missing in lustre,testpool oleg353-server: argument lustre,testpool must be . oleg353-server: pool_new: Invalid argument pdsh@oleg353-client: oleg353-server: ssh exited with exit code 22 PASS 1j (6s) == ost-pools test 1k: pool_new should fail if poolname format is wrong ========================================================== 14:58:12 (1776193092) oleg353-server: . is missing in lustre/testpool oleg353-server: argument lustre/testpool must be . oleg353-server: pool_new: Invalid argument pdsh@oleg353-client: oleg353-server: ssh exited with exit code 22 PASS 1k (7s) == ost-pools test 1m: pool_new did not fail even though testpool2 existed ========================================================== 14:58:19 (1776193099) oleg353-server: Pool lustre.testpool2 created oleg353-server: Pool lustre.testpool2 already exists oleg353-server: pool_new: File exists pdsh@oleg353-client: oleg353-server: ssh exited with exit code 17 lustre.testpool2 oleg353-server: Pool lustre.testpool2 destroyed PASS 1m (17s) == ost-pools test 1n: Pool with a 15 char pool name works well ========================================================== 14:58:36 (1776193116) oleg353-server: Pool lustre.testpool1234567 created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool1234567 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 3.20827 s, 32.7 MB/s oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool1234567 lustre.testpool1234567 oleg353-server: Pool lustre.testpool1234567 destroyed PASS 1n (29s) == ost-pools test 2a: pool_add: non-existant pool testpool ========================================================== 14:59:05 (1776193145) PASS 2a (9s) == ost-pools test 2b: pool_add: Invalid pool name ======== 14:59:14 (1776193154) PASS 2b (8s) == ost-pools test 2c: pool_add: OST index combinations === 14:59:22 (1776193162) oleg353-server: Pool lustre.testpool created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool Waiting 90s for '' oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool Waiting 90s for '' oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool Waiting 90s for '' lustre.testpool oleg353-server: Pool lustre.testpool destroyed PASS 2c (46s) == ost-pools test 2d: pool_add: OSTs that don't exist should be rejected ========================================================== 15:00:08 (1776193208) oleg353-server: Pool lustre.testpool created oleg353-server: OST lustre-OST0002_UUID is not part of the 'lustre' fs. oleg353-server: pool_add: No such file or directory pdsh@oleg353-client: oleg353-server: ssh exited with exit code 2 lustre.testpool oleg353-server: Pool lustre.testpool destroyed PASS 2d (16s) == ost-pools test 2e: pool_add: OST already in a pool should be rejected ========================================================== 15:00:24 (1776193224) oleg353-server: Pool lustre.testpool created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool pdsh@oleg353-client: oleg353-server: ssh exited with exit code 17 OST lustre-OST0000_UUID is already in pool lustre.testpool pool_add: File exists lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed PASS 2e (24s) == ost-pools test 3a: pool_remove: non-existant pool ===== 15:00:49 (1776193249) PASS 3a (7s) == ost-pools test 3b: pool_remove: non-existant fsname === 15:00:55 (1776193255) PASS 3b (6s) == ost-pools test 3c: pool_remove: Invalid pool name ===== 15:01:01 (1776193261) PASS 3c (7s) == ost-pools test 3d: pool_remove: OST index combinations ========================================================== 15:01:08 (1776193268) oleg353-server: Pool lustre.testpool created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool lustre.testpool oleg353-server: Pool lustre.testpool destroyed PASS 3d (39s) == ost-pools test 4a: pool_destroy: non-existant pool ==== 15:01:47 (1776193307) PASS 4a (7s) == ost-pools test 4b: pool_destroy: non-existant fs-name ========================================================== 15:01:54 (1776193314) PASS 4b (6s) == ost-pools test 4c: pool_destroy: non-empty pool ======= 15:02:00 (1776193320) oleg353-server: Pool lustre.testpool created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: Pool lustre.testpool not empty, please remove all members oleg353-server: pool_destroy: Directory not empty pdsh@oleg353-client: oleg353-server: ssh exited with exit code 39 lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed PASS 4c (25s) == ost-pools test 5a: lfs pool_list from client ========== 15:02:25 (1776193345) oleg353-server: Pool lustre.testpool created oleg353-server: Pool lustre.testpool2 created Pools from lustre: lustre.testpool lustre.testpool2 oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID added to pool lustre.testpool Pool: lustre.testpool lustre-OST0000_UUID lustre-OST0001_UUID Pool: lustre.nonexistantpool /home/green/git/lustre-release/lustre/utils/lfs pool_list /mnt/lustre Pools from lustre: lustre.testpool lustre.testpool2 Pools from lustre: lustre.testpool lustre.testpool2 lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed lustre.testpool2 oleg353-server: Pool lustre.testpool2 destroyed PASS 5a (40s) == ost-pools test 5b: lctl pool_list from MDS ============ 15:03:05 (1776193385) oleg353-server: Pool lustre.testpool created oleg353-server: Pool lustre.testpool2 created Pools from lustre: lustre.testpool lustre.testpool2 oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID added to pool lustre.testpool Pool: lustre.testpool lustre-OST0000_UUID lustre-OST0001_UUID Pools from nonexistantfs: Pool: lustre.nonexistantpool Pools from /mnt/lustrenonexistant: lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed lustre.testpool2 oleg353-server: Pool lustre.testpool2 destroyed PASS 5b (45s) == ost-pools test 6: getstripe/setstripe ================= 15:03:50 (1776193430) oleg353-server: Pool lustre.testpool created Pools from lustre: lustre.testpool oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID added to pool lustre.testpool oleg353-server: Pool lustre.testpool2 created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool2 Destroy the created pools: testpool,testpool2 lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed lustre.testpool2 oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool2 oleg353-server: Pool lustre.testpool2 destroyed PASS 6 (63s) == ost-pools test 7a: create various pool name =========== 15:04:54 (1776193494) set poolname to 7 Creating new pool oleg353-server: Pool lustre.7 created Adding targets to pool oleg353-server: OST lustre-OST0000_UUID added to pool lustre.7 oleg353-server: OST lustre-OST0001_UUID added to pool lustre.7 oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.7 oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.7 oleg353-server: Pool lustre.7 destroyed set poolname to 713364185 Creating new pool oleg353-server: Pool lustre.713364185 created Adding targets to pool oleg353-server: OST lustre-OST0000_UUID added to pool lustre.713364185 oleg353-server: OST lustre-OST0001_UUID added to pool lustre.713364185 oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.713364185 oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.713364185 oleg353-server: Pool lustre.713364185 destroyed set poolname to 7133283323567 Creating new pool oleg353-server: Pool lustre.7133283323567 created Adding targets to pool oleg353-server: OST lustre-OST0000_UUID added to pool lustre.7133283323567 oleg353-server: OST lustre-OST0001_UUID added to pool lustre.7133283323567 oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.7133283323567 oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.7133283323567 oleg353-server: Pool lustre.7133283323567 destroyed Destroy the created pools: 7,7133283323567,713364185 Destroy the created pools: 7,7133283323567,713364185 Destroy the created pools: 7,7133283323567,713364185 PASS 7a (69s) == ost-pools test 7b: try to create pool name with invalid lengths or names ========================================================== 15:06:03 (1776193563) oleg353-server: . is missing in qwerty oleg353-server: argument qwerty must be . oleg353-server: pool_new: Invalid argument pdsh@oleg353-client: oleg353-server: ssh exited with exit code 22 oleg353-server: poolname is empty oleg353-server: argument lustre. must be . oleg353-server: pool_new: Invalid argument pdsh@oleg353-client: oleg353-server: ssh exited with exit code 22 oleg353-server: error: pool_new can contain only alphanumeric characters, underscores, and dashes besides the required '.' oleg353-server: pool_new: Invalid argument pdsh@oleg353-client: oleg353-server: ssh exited with exit code 22 oleg353-server: error: pool_new: either the pool or file system name is too long (max pool name len is 15 and file system name is 8) oleg353-server: pool_new: File name too long pdsh@oleg353-client: oleg353-server: ssh exited with exit code 36 PASS 7b (8s) == ost-pools test 7c: create a valid pool name and setstripe with a bad one ========================================================== 15:06:11 (1776193571) Creating new pool oleg353-server: Pool lustre.0123456789abcde created Adding targets to pool oleg353-server: OST lustre-OST0000_UUID added to pool lustre.0123456789abcde oleg353-server: OST lustre-OST0001_UUID added to pool lustre.0123456789abcde lfs setstripe: Invalid Poolname '0123456789abcdeX': Invalid argument (22) oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.0123456789abcde oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.0123456789abcde oleg353-server: Pool lustre.0123456789abcde destroyed Destroy the created pools: 0123456789abcde PASS 7c (25s) == ost-pools test 11: OSTs in overlapping/multiple pools ========================================================== 15:06:36 (1776193596) oleg353-server: Pool lustre.testpool created oleg353-server: Pool lustre.testpool2 created oleg353-server: OST lustre-OST0001_UUID added to pool lustre.testpool2 oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool total: 100 open/close in 1.53 seconds: 65.36 ops/second open(/mnt/lustre/d11.ost-pools/dir2/f11.ost-pools0) error: No space left on device total: 0 open/close in 20.48 seconds: 0.00 ops/second ost-pools test_11: @@@@@@ FAIL: createmany /mnt/lustre/d11.ost-pools/dir2/f11.ost-pools failed! Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6470:error() = /home/green/git/lustre-release/lustre/tests/ost-pools.sh:730:test_11() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6828:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6875:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6716:run_test() = /home/green/git/lustre-release/lustre/tests/ost-pools.sh:739:main() Dumping lctl log to /tmp/testlogs//ost-pools.test_11.*.1776193825.log rsync: chown "/tmp/testlogs/.ost-pools.test_11.debug_log.oleg353-server.1776193825.log.AEGSB8" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.ost-pools.test_11.dmesg.oleg353-server.1776193825.log.RlNhZp" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3] Destroy the created pools: testpool,testpool2 lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed lustre.testpool2 oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool2 oleg353-server: Pool lustre.testpool2 destroyed FAIL 11 (249s) == ost-pools test 12: OST Pool Membership ================ 15:10:52 (1776193852) SKIP: ost-pools test_12 needs >=3 OSTs SKIP 12 (2s) == ost-pools test 13: Striping characteristics in a pool ========================================================== 15:10:55 (1776193855) SKIP: ost-pools test_13 needs >= 3 OSTs SKIP 13 (3s) == ost-pools test 14: Round robin and QOS striping within a pool ========================================================== 15:10:58 (1776193858) SKIP: ost-pools test_14 needs >= 3 OSTs SKIP 14 (3s) == ost-pools test 15: One directory per OST/pool ========= 15:11:01 (1776193861) oleg353-server: Pool lustre.testpool0 created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool0 total: 100 open/close in 1.74 seconds: 57.38 ops/second oleg353-server: Pool lustre.testpool1 created oleg353-server: OST lustre-OST0001_UUID added to pool lustre.testpool1 open(/mnt/lustre/d15.ost-pools/dir1/f15.ost-pools0) error: No space left on device total: 0 open/close in 20.27 seconds: 0.00 ops/second ost-pools test_15: @@@@@@ FAIL: createmany /mnt/lustre/d15.ost-pools/dir1/f15.ost-pools failed! Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6470:error() = /home/green/git/lustre-release/lustre/tests/ost-pools.sh:923:test_15() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6828:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6875:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6716:run_test() = /home/green/git/lustre-release/lustre/tests/ost-pools.sh:934:main() Dumping lctl log to /tmp/testlogs//ost-pools.test_15.*.1776193915.log rsync: chown "/tmp/testlogs/.ost-pools.test_15.debug_log.oleg353-server.1776193915.log.wwIdiq" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.ost-pools.test_15.dmesg.oleg353-server.1776193915.log.V2K8l2" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3] Destroy the created pools: testpool0,testpool1 lustre.testpool0 oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool0 oleg353-server: Pool lustre.testpool0 destroyed lustre.testpool1 oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool1 oleg353-server: Pool lustre.testpool1 destroyed FAIL 15 (73s) == ost-pools test 16: Inheritance of pool properties ===== 15:12:20 (1776193940) oleg353-server: Pool lustre.testpool created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool total: 10 open/close in 0.16 seconds: 63.29 ops/second Destroy the created pools: testpool lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed PASS 16 (41s) == ost-pools test 17: Referencing an empty pool ========== 15:13:01 (1776193981) oleg353-server: Pool lustre.testpool created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID added to pool lustre.testpool - open/close 0 (time 1776194010.88 total 20.45 last 0.00) - open/close 1 (time 1776194031.34 total 40.91 last 0.05) - open/close 2 (time 1776194051.82 total 61.40 last 0.05) - open/close 3 (time 1776194072.32 total 81.89 last 0.05) - open/close 4 (time 1776194092.77 total 102.34 last 0.05) - open/close 5 (time 1776194113.28 total 122.85 last 0.05) - open/close 6 (time 1776194133.73 total 143.30 last 0.05) - open/close 7 (time 1776194154.21 total 163.78 last 0.05) - open/close 8 (time 1776194174.71 total 184.28 last 0.05) - open/close 9 (time 1776194195.18 total 204.76 last 0.05) total: 10 open/close in 204.76 seconds: 0.05 ops/second lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed - open/close 0 (time 1776194245.88 total 20.30 last 0.00) - open/close 1 (time 1776194266.35 total 40.78 last 0.05) - open/close 2 (time 1776194286.83 total 61.26 last 0.05) - open/close 3 (time 1776194307.32 total 81.75 last 0.05) - open/close 4 (time 1776194327.78 total 102.21 last 0.05) - open/close 5 (time 1776194348.26 total 122.69 last 0.05) - open/close 6 (time 1776194368.74 total 143.17 last 0.05) - open/close 7 (time 1776194389.23 total 163.66 last 0.05) - open/close 8 (time 1776194409.72 total 184.15 last 0.05) - open/close 9 (time 1776194430.19 total 204.61 last 0.05) total: 10 open/close in 204.61 seconds: 0.05 ops/second PASS 17 (454s) == ost-pools test 18: File create in a directory which references a deleted pool ========================================================== 15:20:35 (1776194435) Create performance, iteration 1, 15 seconds x 3 total: 1047 open/close in 15.02 seconds: 69.71 ops/second iter 1: 1047 creates without pool oleg353-server: Pool lustre.testpool created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID added to pool lustre.testpool total: 1 open/close in 20.29 seconds: 0.05 ops/second iter 1: 1 creates with pool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed total: 1 open/close in 20.17 seconds: 0.05 ops/second iter 1: 1 creates with missing pool Create performance, iteration 2, 15 seconds x 3 total: 1056 open/close in 15.02 seconds: 70.32 ops/second iter 2: 1056 creates without pool oleg353-server: Pool lustre.testpool created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID added to pool lustre.testpool total: 1 open/close in 20.46 seconds: 0.05 ops/second iter 2: 1 creates with pool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed total: 1 open/close in 20.49 seconds: 0.05 ops/second iter 2: 1 creates with missing pool Create performance, iteration 3, 15 seconds x 3 total: 1146 open/close in 15.01 seconds: 76.35 ops/second iter 3: 1146 creates without pool oleg353-server: Pool lustre.testpool created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID added to pool lustre.testpool total: 1 open/close in 20.49 seconds: 0.05 ops/second iter 3: 1 creates with pool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed total: 1 open/close in 20.30 seconds: 0.05 ops/second iter 3: 1 creates with missing pool Avg files created in 15 seconds without pool: 1083 Avg files created in 15 seconds with pool: 1 Avg files created in 15 seconds missing pool: 1 No pool / wide pool: 99 %. ost-pools test_18: @@@@@@ IGNORE (bz23408): Degradation with wide pool is 99% > 30% Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6486:error_ignore() = /home/green/git/lustre-release/lustre/tests/ost-pools.sh:1060:test_18() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6828:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6875:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6716:run_test() = /home/green/git/lustre-release/lustre/tests/ost-pools.sh:1070:main() No pool / missing pool: 99 %. ost-pools test_18: @@@@@@ IGNORE (bz23408): Degradation with wide pool is 99% > 30% Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6486:error_ignore() = /home/green/git/lustre-release/lustre/tests/ost-pools.sh:1066:test_18() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6828:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6875:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6716:run_test() = /home/green/git/lustre-release/lustre/tests/ost-pools.sh:1070:main() PASS 18 (399s) == ost-pools test 19: Pools should not come into play when not specified ========================================================== 15:27:14 (1776194834) oleg353-server: Pool lustre.testpool created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool total: 12 open/close in 0.14 seconds: 84.74 ops/second total: 12 open/close in 0.17 seconds: 70.66 ops/second Destroy the created pools: testpool lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed PASS 19 (46s) == ost-pools test 20: Different pools in a directory hierarchy. ========================================================== 15:28:00 (1776194880) oleg353-server: Pool lustre.testpool created oleg353-server: Pool lustre.testpool2 created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID added to pool lustre.testpool2 lfs setstripe: setstripe error for '/mnt/lustre/d20.ost-pools/dir1/file1': No space left on device ost-pools test_20: @@@@@@ FAIL: /home/green/git/lustre-release/lustre/utils/lfs setstripe -p testpool2 /mnt/lustre/d20.ost-pools/dir1/file1 failed Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6470:error() = /home/green/git/lustre-release/lustre/tests/ost-pools.sh:78:create_file() = /home/green/git/lustre-release/lustre/tests/ost-pools.sh:1125:test_20() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6828:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6875:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6716:run_test() = /home/green/git/lustre-release/lustre/tests/ost-pools.sh:1152:main() Dumping lctl log to /tmp/testlogs//ost-pools.test_20.*.1776194921.log rsync: chown "/tmp/testlogs/.ost-pools.test_20.debug_log.oleg353-server.1776194921.log.VZnTgn" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.ost-pools.test_20.dmesg.oleg353-server.1776194921.log.EmeGbJ" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3] Destroy the created pools: testpool,testpool2 lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed lustre.testpool2 oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool2 oleg353-server: Pool lustre.testpool2 destroyed FAIL 20 (63s) == ost-pools test 21: OST pool with fewer OSTs than stripe count ========================================================== 15:29:10 (1776194950) oleg353-server: Pool lustre.testpool created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool /mnt/lustre/d21.ost-pools/dir/file1 lmm_magic: 0x0BD30BD0 lmm_seq: 0x200000401 lmm_object_id: 0xdff lmm_fid: [0x200000401:0xdff:0x0] lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_pool: testpool obdidx objid objid group 0 3519 0xdbf 0 Destroy the created pools: testpool lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed PASS 21 (24s) == ost-pools test 22: Simultaneous manipulation of a pool ========================================================== 15:29:34 (1776194974) oleg353-server: Pool lustre.testpool created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: Pool lustre.testpool2 created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool2 loop for testpool loop for testpool2 Pool testpool, iteration 1 Pool testpool2, iteration 1 Pool testpool2, iteration 2 Pool testpool2, iteration 3 Pool testpool2, iteration 4 Pool testpool2, iteration 5 Pool testpool2, iteration 6 Pool testpool2, iteration 7 Pool testpool2, iteration 8 Pool testpool2, iteration 9 Pool testpool2, iteration 10 loop for testpool2 complete oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool Pool testpool, iteration 2 - open/close 0 (time 1776195017.96 total 20.54 last 0.00) total: 100 open/close in 22.38 seconds: 4.47 ops/second oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool Pool testpool, iteration 3 oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool Pool testpool, iteration 4 oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool Pool testpool, iteration 5 oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool Pool testpool, iteration 6 oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool Pool testpool, iteration 7 oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool Pool testpool, iteration 8 oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool Pool testpool, iteration 9 oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool Pool testpool, iteration 10 oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool loop for testpool complete Destroy the created pools: testpool,testpool2 lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed lustre.testpool2 oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool2 oleg353-server: Pool lustre.testpool2 destroyed PASS 22 (108s) == ost-pools test 23a: OST pools and quota =============== 15:31:22 (1776195082) running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f7133] oleg353-server: Pool lustre.testpool created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool Disk quotas for usr 500 (uid 500): Filesystem kbytes quota limit grace files quota limit grace /mnt/lustre/d23a.ost-pools/dir 0 3072 3072 - 0 0 0 - lustre-MDT0000_UUID 0 - 0 - 0 - 0 - lustre-OST0000_UUID 0 - 0 - - - - - lustre-OST0001_UUID 0 - 0 - - - - - Total allocated inode limit: 0, total allocated block limit: 0 uid 500 is using default file quota setting -rw-r--r-- 1 sanityusr sanityusr 0 Apr 14 15:31 /mnt/lustre/d23a.ost-pools/dir/f23a.ost-pools-quota first dd failed with EDQUOT. Disk quotas for usr 500 (uid 500): Filesystem kbytes quota limit grace files quota limit grace /mnt/lustre/d23a.ost-pools/dir 2 3072 3072 - 1 0 0 - lustre-MDT0000_UUID 2 - 0 - 1 - 0 - lustre-OST0000_UUID 0 - 2055 - - - - - lustre-OST0001_UUID 0 - 0 - - - - - Total allocated inode limit: 0, total allocated block limit: 2055 uid 500 is using default file quota setting Destroy the created pools: testpool lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed PASS 23a (37s) == ost-pools test 23b: OST pools and OOS ================= 15:31:59 (1776195119) running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f7133] oleg353-server: Pool lustre.testpool created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool OSTCOUNT=2, OSTSIZE=0, AVAIL=3762176 MAXFREE=31457280, SLOW=yes [1 iteration] dd: error writing '/mnt/lustre/d23b.ost-pools/dir/f23b.ost-pools-quota1': No space left on device 3649+0 records in 3648+0 records out 3825205248 bytes (3.8 GB, 3.6 GiB) copied, 71.5083 s, 53.5 MB/s total written: 5242880 stime=1776195135, etime=1776195207, elapsed=72 Filesystem Size Used Avail Use% Mounted on devtmpfs 1.9G 0 1.9G 0% /dev tmpfs 1.9G 0 1.9G 0% /dev/shm tmpfs 1.9G 15M 1.8G 1% /run tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup /dev/nbd0 30G 17G 12G 60% / none 1.9G 24M 1.8G 2% /var/lib/stateless/writable /dev/vda 34M 34M 0 100% /home/green/git/lustre-release 192.168.200.253:/exports/state/oleg353-client.virtnet 138G 117G 15G 90% /var/lib/stateless/state 192.168.200.253:/exports/state/oleg353-client.virtnet/boot 138G 117G 15G 90% /boot 192.168.200.253:/exports/state/oleg353-client.virtnet/etc/kdump.conf 138G 117G 15G 90% /etc/kdump.conf tmpfs 370M 0 370M 0% /run/user/0 192.168.200.253:/exports/testreports/62605/testresults/ost-pools-zfs-rocky8.10_x86_64-rocky8.10_x86_64-retry3 51T 42T 9.5T 82% /tmp/testlogs 192.168.203.153@tcp:/lustre 3.6G 3.4G 181M 96% /mnt/lustre Destroy the created pools: testpool lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed PASS 23b (107s) == ost-pools test 24: Independence of pool from other setstripe parameters ========================================================== 15:33:46 (1776195226) oleg353-server: Pool lustre.testpool created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID added to pool lustre.testpool - open/close 0 (time 1776195255.02 total 20.24 last 0.00) - open/close 1 (time 1776195275.51 total 40.73 last 0.05) - open/close 2 (time 1776195295.98 total 61.20 last 0.05) - open/close 3 (time 1776195316.50 total 81.72 last 0.05) - open/close 4 (time 1776195336.95 total 102.17 last 0.05) - open/close 5 (time 1776195357.42 total 122.63 last 0.05) - open/close 6 (time 1776195377.90 total 143.12 last 0.05) - open/close 7 (time 1776195398.39 total 163.61 last 0.05) - open/close 8 (time 1776195418.86 total 184.08 last 0.05) - open/close 9 (time 1776195439.34 total 204.56 last 0.05) total: 10 open/close in 204.56 seconds: 0.05 ops/second ost-pools test_24: @@@@@@ FAIL: Stripe count 2 not on /mnt/lustre/d24.ost-pools/dir1/f24.ost-pools0:1 Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6470:error() = /home/green/git/lustre-release/lustre/tests/ost-pools.sh:1433:test_24() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6828:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6875:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6716:run_test() = /home/green/git/lustre-release/lustre/tests/ost-pools.sh:1447:main() Dumping lctl log to /tmp/testlogs//ost-pools.test_24.*.1776195443.log Destroy the created pools: testpool rsync: chown "/tmp/testlogs/.ost-pools.test_24.debug_log.oleg353-server.1776195443.log.MTjmeG" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.ost-pools.test_24.dmesg.oleg353-server.1776195443.log.9wbFhR" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3] lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed FAIL 24 (234s) == ost-pools test 25: Create new pool and restart MDS ==== 15:37:46 (1776195466) oleg353-server: Pool lustre.testpool1 created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool1 Failing mds1 on oleg353-server Stopping /mnt/lustre-mds1 (opts:) on oleg353-server reboot facets: mds1 Failover mds1 to oleg353-server mount facets: mds1 Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg353-server: oleg353-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 pdsh@oleg353-client: oleg353-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg353-server: oleg353-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 40 oleg353-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg353-server mds-ost sync done. Creating a file in pool1 Destroy the created pools: testpool1 lustre.testpool1 oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool1 oleg353-server: Pool lustre.testpool1 destroyed PASS 25 (78s) == ost-pools test 26: Choose other OSTs in the pool first in the creation remedy ========================================================== 15:39:04 (1776195544) SKIP: ost-pools test_26 needs >= 3 OSTs SKIP 26 (3s) == ost-pools test 27: Race pool_list and pool_remove ===== 15:39:08 (1776195548) oleg353-server: Pool lustre.testpool created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool fail_loc=0x239 fail_val=10 fail_loc=0 lustre-OST0001_UUID oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool fail_val=0 lustre.testpool oleg353-server: Pool lustre.testpool destroyed PASS 27 (34s) == ost-pools test 28: lfs_migrate with pool name ========= 15:39:42 (1776195582) oleg353-server: Pool lustre.testpool created oleg353-server: Pool lustre.testpool2 created oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID added to pool lustre.testpool oleg353-server: OST lustre-OST0000_UUID added to pool lustre.testpool2 oleg353-server: OST lustre-OST0001_UUID added to pool lustre.testpool2 debug=-1 debug_mb=150 debug=-1 debug_mb=150 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0872637 s, 12.0 MB/s /mnt/lustre/f28.ost-pools lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_pool: testpool obdidx objid objid group 1 2 0x2 0 /mnt/lustre/f28.ost-pools: done /mnt/lustre/f28.ost-pools lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 1 lmm_stripe_offset: 1 lmm_pool: testpool2 obdidx objid objid group 1 3 0x3 0 debug_mb=8 debug_mb=8 Destroy the created pools: testpool,testpool2 lustre.testpool oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool oleg353-server: Pool lustre.testpool destroyed lustre.testpool2 oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.testpool2 oleg353-server: OST lustre-OST0001_UUID removed from pool lustre.testpool2 oleg353-server: Pool lustre.testpool2 destroyed PASS 28 (59s) == ost-pools test 29: check OST pool spilling ============ 15:40:41 (1776195641) SKIP: ost-pools test_29 needs >= 4 OSTs SKIP 29 (4s) == ost-pools test 30: persistent OST pool spilling ======= 15:40:45 (1776195645) SKIP: ost-pools test_30 needs >= 4 OSTs SKIP 30 (3s) == ost-pools test 31: OST pool spilling chained ========== 15:40:48 (1776195648) SKIP: ost-pools test_31 needs >= 4 OSTs SKIP 31 (4s) == ost-pools test 32: force to create a file without pool (no inheritance) ========================================================== 15:40:52 (1776195652) Creating new pool oleg353-server: Pool lustre.test_32 created Adding targets to pool oleg353-server: OST lustre-OST0000_UUID added to pool lustre.test_32 Destroy the created pools: test_32 lustre.test_32 oleg353-server: OST lustre-OST0000_UUID removed from pool lustre.test_32 oleg353-server: Pool lustre.test_32 destroyed PASS 32 (23s) == ost-pools test complete, duration 2726 sec ============ 15:41:15 (1776195675) ost-pools: FAIL: test_11 createmany /mnt/lustre/d11.ost-pools/dir2/f11.ost-pools failed! ost-pools: FAIL: test_15 createmany /mnt/lustre/d15.ost-pools/dir1/f15.ost-pools failed! ost-pools: FAIL: test_20 /home/green/git/lustre-release/lustre/utils/lfs setstripe -p testpool2 /mnt/lustre/d20.ost-pools/dir1/file1 failed ost-pools: FAIL: test_24 Stripe count 2 not on /mnt/lustre/d24.ost-pools/dir1/f24.ost-pools0:1