-----============= acceptance-small: sanityn ============----- Mon Sep 7 19:15:15 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg418-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 40a skipping tests SLOW=no: 33a === sanityn: start setup 19:15:31 (1788822931) === Starting client oleg418-client.virtnet: -o user_xattr,flock 192.168.204.118@tcp:/lustre /mnt/lustre2 Mount client oleg418-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.204.118@tcp:/lustre /mnt/lustre2 Started clients oleg418-client.virtnet: 192.168.204.118@tcp:/lustre on /mnt/lustre2 type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) oleg418-client.virtnet: executing check_config_client /mnt/lustre oleg418-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg418-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff90ecc86f3800.idle_timeout=debug osc.lustre-OST0000-osc-ffff90ecd8972800.idle_timeout=debug osc.lustre-OST0001-osc-ffff90ecc86f3800.idle_timeout=debug osc.lustre-OST0001-osc-ffff90ecd8972800.idle_timeout=debug disable quota as required oleg418-server: oleg418-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 19:16:03 (1788822963) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261782 s, 40.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7646] == sanityn test 77ve: mClock: rule Reserved and Limit rate setting ========================================================== 19:16:07 (1788822967) using SAVE_PROJECT_SUPPORTED=0 ost.OSS.ost_io.nrs_policies=mclock projid reserve=1 limit=1 oleg418-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_mclock_rule=start proj100 projid={100} resvd_rate=100 limit_rate=50: Numerical result out of range oleg418-server: error: set_param: setting 'ost/OSS/ost_io/nrs_mclock_rule'='start proj100 projid={100} resvd_rate=100 limit_rate=50': Numerical result out of range pdsh@oleg418-client: oleg418-server: ssh exited with exit code 34 ost.OSS.ost_io.nrs_mclock_rule=start proj100 projid={100} iops=100 Rule rate: limit_iops 100 ost.OSS.ost_io.nrs_mclock_rule=stop proj100 ost.OSS.ost_io.nrs_mclock_rule=start proj100 projid={100} rate=5MiB/s Rule rate: limit_bps 5 ost.OSS.ost_io.nrs_mclock_rule=stop proj100 ost.OSS.ost_io.nrs_mclock_rule=start proj100 projid={100} limit_rate=300MiB/s Rule rate: limit_bps 300 ost.OSS.ost_io.nrs_mclock_rule=stop proj100 ost.OSS.ost_io.nrs_mclock_rule=start proj100 projid={100} resvd_rate=100 limit_rate=300 Rule rate: iops 100-300 ost.OSS.ost_io.nrs_mclock_rule=stop proj100 ost.OSS.ost_io.nrs_mclock_rule=start proj100 projid={100} resvd_rate=10MiB limit_rate=30MiB Rule rate: bps 10-30 ost.OSS.ost_io.nrs_mclock_rule=stop proj100 ost.OSS.ost_io.nrs_mclock_rule=start proj100 projid={100} resvd_rate=10MiB limit_rate=30MiB resvd_iops=5 limit_iops=25 Rule rate: iops 5-25 bps 10-30 ost.OSS.ost_io.nrs_mclock_rule=stop proj100 ost.OSS.ost_io.nrs_mclock_rule=start proj100 projid={100} bps=5MiB/s Rule rate: limit_bps 5 ost.OSS.ost_io.nrs_mclock_rule=stop proj100 ost.OSS.ost_io.nrs_mclock_rule=start proj100 projid={100} limit_bps=10MiB/s ost.OSS.ost_io.nrs_mclock_rule=change proj100 limit_rate=100 ost.OSS.ost_io.nrs_mclock_rule= regular_requests: CPT 0: proj100 projid={100}, limit_iops 100, seq 9, type=projid, ref 0 default *, limit_iops 10000, seq 1, type=projid, ref 0 high_priority_requests: CPT 0: proj100 projid={100}, limit_iops 100, seq 9, type=projid, ref 0 default *, limit_iops 10000, seq 1, type=projid, ref 0 Rule rate: limit_iops 100 ost.OSS.ost_io.nrs_mclock_rule=stop proj100 ost.OSS.ost_io.nrs_mclock_rule=start proj100 projid={100} resvd_rate=10 limit_rate=300 ost.OSS.ost_io.nrs_mclock_rule=change proj100 limit_rate=200 Rule rate: iops 10-200 ost.OSS.ost_io.nrs_mclock_rule=stop proj100 ost.OSS.ost_io.nrs_mclock_rule=start proj100 projid={100} resvd_rate=2 limit_rate=unlimit ost.OSS.ost_io.nrs_mclock_rule=change proj100 rate=5 ost.OSS.ost_io.nrs_mclock_rule= regular_requests: CPT 0: proj100 projid={100}, iops 2-5, seq 11, type=projid, ref 0 default *, limit_iops 10000, seq 1, type=projid, ref 0 high_priority_requests: CPT 0: proj100 projid={100}, iops 2-5, seq 11, type=projid, ref 0 default *, limit_iops 10000, seq 1, type=projid, ref 0 not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5, bs: 1048576 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8881 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 vs. 5 Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8483 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 vs. 5 cleanup_tbf /mnt/lustre/d77ve.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_mclock_rule=stop proj100 ost.OSS.ost_io.nrs_mclock_rule=start proj100 projid={100} limit_bps=5000B/s Rule rate: limit_bps 5000B/s ost.OSS.ost_io.nrs_mclock_rule=stop proj100 ost.OSS.ost_io.nrs_mclock_rule=start proj100 projid={100} limit_bps=1536KiB/s Rule rate: limit_bps 1572864B/s Cleanup TBF ost.OSS.ost_io.nrs_mclock_rule=stop proj100 ost.OSS.ost_io.nrs_policies=fifo PASS 77ve (106s) == sanityn test 77vq: mClock: a limit and weight policy needs no reservation ========================================================== 19:17:53 (1788823073) using SAVE_PROJECT_SUPPORTED=0 ost.OSS.ost_io.nrs_policies=mclock projid limit=1 weight=1 ost.OSS.ost_io.nrs_mclock_rule=start lw100 projid={100} limit_rate=100 weight=200 ost.OSS.ost_io.nrs_mclock_rule=start wonly projid={300} weight=300 ost.OSS.ost_io.nrs_mclock_rule= regular_requests: CPT 0: wonly projid={300}, rate 0-max, weight 300, seq 3, type=projid, ref 0 lw100 projid={100}, limit_iops 100, weight 200, seq 2, type=projid, ref 0 default *, limit_iops 10000, weight 100, seq 1, type=projid, ref 0 high_priority_requests: CPT 0: wonly projid={300}, rate 0-max, weight 300, seq 3, type=projid, ref 0 lw100 projid={100}, limit_iops 100, weight 200, seq 2, type=projid, ref 0 default *, limit_iops 10000, weight 100, seq 1, type=projid, ref 0 oleg418-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_mclock_rule=start res100 projid={200} resvd_rate=100: Invalid argument oleg418-server: error: set_param: setting 'ost/OSS/ost_io/nrs_mclock_rule'='start res100 projid={200} resvd_rate=100': Invalid argument pdsh@oleg418-client: oleg418-server: ssh exited with exit code 22 ost.OSS.ost_io.nrs_mclock_rule=change lw100 limit_rate=20 not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20, bs: 1048576 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 7.38231 s, 14.2 MB/s Write runtime is 9 s, speed is 11.111111 vs. 20 Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.04396 s, 20.8 MB/s Read runtime is 6 s, speed is 16.666666 vs. 20 cleanup_tbf /mnt/lustre/d77vq.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Cleanup TBF ost.OSS.ost_io.nrs_mclock_rule=stop lw100 ost.OSS.ost_io.nrs_mclock_rule=stop wonly ost.OSS.ost_io.nrs_policies=fifo PASS 77vq (61s) == sanityn test 77vk: Setting unlimit limit rate and zero reserved rate for NRS rule ========================================================== 19:18:54 (1788823134) ost.OSS.ost_io.nrs_policies=mclock projid reserve=1 limit=1 oleg418-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_mclock_rule=start proj100 projid={100} R=100 L=300 W=100: Invalid argument oleg418-server: error: set_param: setting 'ost/OSS/ost_io/nrs_mclock_rule'='start proj100 projid={100} R=100 L=300 W=100': Invalid argument pdsh@oleg418-client: oleg418-server: ssh exited with exit code 22 oleg418-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_mclock_rule=start proj100 projid={100} R=max L=max: Invalid argument oleg418-server: error: set_param: setting 'ost/OSS/ost_io/nrs_mclock_rule'='start proj100 projid={100} R=max L=max': Invalid argument pdsh@oleg418-client: oleg418-server: ssh exited with exit code 22 oleg418-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_mclock_rule=start proj100 projid={100} R=0 L=max: Invalid argument oleg418-server: error: set_param: setting 'ost/OSS/ost_io/nrs_mclock_rule'='start proj100 projid={100} R=0 L=max': Invalid argument pdsh@oleg418-client: oleg418-server: ssh exited with exit code 22 ost.OSS.ost_io.nrs_mclock_rule=stop proj100 ost.OSS.ost_io.nrs_mclock_rule=start proj100 projid={100} R=0 L=100 Rule rate: iops 0-100 ost.OSS.ost_io.nrs_mclock_rule=stop proj100 ost.OSS.ost_io.nrs_mclock_rule=start proj100 projid={100} R=20 L=unlimit Rule rate: iops 20-max ost.OSS.ost_io.nrs_mclock_rule=stop proj100 ost.OSS.ost_io.nrs_mclock_rule=start proj100 projid={100} R=20MiB L=unlimit Rule rate: bps 20-max ost.OSS.ost_io.nrs_policies=mclock projid reserve=1 limit=1 weight=1 ost.OSS.ost_io.nrs_mclock_rule=start proj100 projid={100} R=0 L=max W=100 Rule rate: rate 0-max Cleanup TBF ost.OSS.ost_io.nrs_mclock_rule=stop proj100 ost.OSS.ost_io.nrs_policies=fifo PASS 77vk (28s) == sanityn test 77vl: mClock: Verify R=0 and L=max works as expected ========================================================== 19:19:22 (1788823162) using SAVE_PROJECT_SUPPORTED=0 /usr/bin/fio /usr/bin/which: no jq in (/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:) SKIP: sanityn test_77vl jq is not installed removed '/tmp/tmp.uBs0HnwEl8' removed '/tmp/tmp.QKYn2OyzOo' SKIP 77vl (7s) == sanityn test 77vs: mClock: an unlimited class waits for no other throttle ========================================================== 19:19:29 (1788823169) using SAVE_PROJECT_SUPPORTED=0 ost.OSS.ost_io.nrs_policies=mclock projid limit=1 weight=1 ost.OSS.ost_io.nrs_mclock_rule=start slow projid={100} limit_rate=1 weight=100 ost.OSS.ost_io.nrs_mclock_rule=start fast projid={200} limit_rate=max weight=100 ost.OSS.ost_io.nrs_mclock_rule= regular_requests: CPT 0: fast projid={200}, rate 0-max, weight 100, seq 3, type=projid, ref 0 slow projid={100}, limit_iops 1, weight 100, seq 2, type=projid, ref 0 default *, limit_iops 10000, weight 100, seq 1, type=projid, ref 0 high_priority_requests: CPT 0: fast projid={200}, rate 0-max, weight 100, seq 3, type=projid, ref 0 slow projid={100}, limit_iops 1, weight 100, seq 2, type=projid, ref 0 default *, limit_iops 10000, weight 100, seq 1, type=projid, ref 0 30+0 records in 30+0 records out 31457280 bytes (31 MB, 30 MiB) copied, 2.77845 s, 11.3 MB/s the unlimited class needed 3s for 30 writes Cleanup TBF ost.OSS.ost_io.nrs_mclock_rule=stop slow ost.OSS.ost_io.nrs_mclock_rule=stop fast ost.OSS.ost_io.nrs_policies=fifo PASS 77vs (28s) == sanityn test 77vo: TBF: an unlimited rate is not a valid TBF rate ========================================================== 19:19:57 (1788823197) ost.OSS.ost_io.nrs_policies=tbf nid oleg418-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_tbf_rule=start nidall nid={0@lo} rate=max: Invalid argument oleg418-server: error: set_param: setting 'ost/OSS/ost_io/nrs_tbf_rule'='start nidall nid={0@lo} rate=max': Invalid argument pdsh@oleg418-client: oleg418-server: ssh exited with exit code 22 oleg418-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_tbf_rule=start nidall nid={0@lo} rate=unlimit: Invalid argument oleg418-server: error: set_param: setting 'ost/OSS/ost_io/nrs_tbf_rule'='start nidall nid={0@lo} rate=unlimit': Invalid argument pdsh@oleg418-client: oleg418-server: ssh exited with exit code 22 oleg418-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_tbf_rule=start nidall nid={0@lo} limit_rate=max: Invalid argument oleg418-server: error: set_param: setting 'ost/OSS/ost_io/nrs_tbf_rule'='start nidall nid={0@lo} limit_rate=max': Invalid argument pdsh@oleg418-client: oleg418-server: ssh exited with exit code 22 ost.OSS.ost_io.nrs_tbf_rule=start nidall nid={0@lo} rate=100 ost.OSS.ost_io.nrs_tbf_rule= regular_requests: CPT 0: nidall nid={0@lo} 100, seq 2, type=nid, ref 0 default * 10000, seq 1, type=nid, ref 0 high_priority_requests: CPT 0: nidall nid={0@lo} 100, seq 2, type=nid, ref 0 default * 10000, seq 1, type=nid, ref 0 Cleanup TBF ost.OSS.ost_io.nrs_tbf_rule=stop nidall ost.OSS.ost_io.nrs_policies=fifo PASS 77vo (22s) == sanityn test 77vp: mClock: a zero reservation needs a second dimension ========================================================== 19:20:19 (1788823219) using SAVE_PROJECT_SUPPORTED=0 ost.OSS.ost_io.nrs_policies=mclock projid reserve=1 ost.OSS.ost_io.nrs_mclock_rule=start proj100 projid={100} resvd_rate=100 oleg418-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_mclock_rule=start zero projid={200} resvd_rate=0: Invalid argument oleg418-server: error: set_param: setting 'ost/OSS/ost_io/nrs_mclock_rule'='start zero projid={200} resvd_rate=0': Invalid argument pdsh@oleg418-client: oleg418-server: ssh exited with exit code 22 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.16245 s, 14.4 MB/s 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.404933 s, 41.4 MB/s Cleanup TBF ost.OSS.ost_io.nrs_mclock_rule=stop proj100 ost.OSS.ost_io.nrs_policies=fifo PASS 77vp (21s) cleanup: ====================================================== == sanityn test complete, duration 326 sec =============== 19:20:43 (1788823243) === sanityn: start cleanup 19:20:45 (1788823245) === Stopping clients: oleg418-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg418-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 19:20:51 (1788823251) ===