== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 21:02:04 (1786755724) Stopping clients: oleg252-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg252-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg252-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg252-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg252-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg252-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg252-server oleg252-server: oleg252-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg252-server' oleg252-server: oleg252-server.virtnet: executing load_modules_local oleg252-server: Loading modules from /home/green/git/lustre-release/lustre oleg252-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg252-server: detected 4 online CPUs by sysfs oleg252-server: MODOPTS_LIBCFS= oleg252-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg252-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg252-server: oleg252-server.virtnet: executing set_default_debug -1 all pdsh@oleg252-client: oleg252-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg252-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg252-server: oleg252-server.virtnet: executing set_default_debug -1 all pdsh@oleg252-client: oleg252-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg252-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg252-server: oleg252-server.virtnet: executing set_default_debug -1 all pdsh@oleg252-client: oleg252-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg252-client.virtnet: -o user_xattr,flock 192.168.202.152@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 874 (time 1786755902.20 total 10.00 last 87.36) - open/close 1777 (time 1786755912.20 total 20.00 last 90.30) - open/close 2659 (time 1786755922.21 total 30.02 last 88.08) - open/close 3575 (time 1786755932.23 total 40.04 last 91.40) - open/close 4477 (time 1786755942.24 total 50.05 last 90.13) total: 4500 open/close in 50.29 seconds: 89.49 ops/second - /unlink 1590 (time 1786755956.69 total 10.01 last 158.84) - /unlink 3101 (time 1786755966.69 total 20.01 last 151.09) total: 4500 /unlink in 27.98 seconds: 160.81 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 888 (time 1786755989.56 total 10.00 last 88.79) - open/close 1633 (time 1786755999.56 total 20.01 last 74.46) - open/close 2603 (time 1786756009.57 total 30.02 last 96.91) - open/close 3529 (time 1786756019.57 total 40.02 last 92.58) - open/close 4395 (time 1786756029.59 total 50.03 last 86.49) total: 4500 open/close in 51.17 seconds: 87.95 ops/second - /unlink 1618 (time 1786756044.73 total 10.01 last 161.63) - /unlink 3213 (time 1786756054.73 total 20.01 last 159.47) total: 4500 /unlink in 28.03 seconds: 160.56 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 877 (time 1786756078.34 total 10.01 last 87.62) - open/close 1729 (time 1786756088.35 total 20.02 last 85.10) - open/close 2611 (time 1786756098.35 total 30.02 last 88.17) - open/close 3452 (time 1786756108.36 total 40.03 last 84.05) - open/close 4321 (time 1786756118.36 total 50.03 last 86.87) total: 4500 open/close in 52.62 seconds: 85.52 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1555 (time 1786756134.89 total 10.00 last 155.48) - /unlink 3219 (time 1786756144.89 total 20.00 last 166.37) total: 4500 /unlink in 27.51 seconds: 163.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 966 (time 1786756166.68 total 10.01 last 96.54) - open/close 1993 (time 1786756176.68 total 20.01 last 102.69) - open/close 2890 (time 1786756186.69 total 30.01 last 89.64) - open/close 3768 (time 1786756196.71 total 40.04 last 87.60) total: 4500 open/close in 48.09 seconds: 93.58 ops/second - /unlink 1778 (time 1786756219.29 total 10.00 last 177.76) - /unlink 3353 (time 1786756229.29 total 20.00 last 157.50) total: 4500 /unlink in 26.96 seconds: 166.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 857 (time 1786756250.15 total 10.00 last 85.68) - open/close 1722 (time 1786756260.15 total 20.00 last 86.49) - open/close 2565 (time 1786756270.16 total 30.01 last 84.26) - open/close 3232 (time 1786756280.17 total 40.02 last 66.62) - open/close 4030 (time 1786756290.18 total 50.03 last 79.73) total: 4500 open/close in 55.98 seconds: 80.38 ops/second - /unlink 1817 (time 1786756309.40 total 10.00 last 181.66) - /unlink 3571 (time 1786756319.40 total 20.01 last 175.34) total: 4500 /unlink in 25.53 seconds: 176.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 819 (time 1786756339.56 total 10.00 last 81.88) - open/close 1688 (time 1786756349.58 total 20.03 last 86.65) - open/close 2647 (time 1786756359.59 total 30.04 last 95.84) - open/close 3509 (time 1786756369.60 total 40.04 last 86.15) - open/close 4341 (time 1786756379.60 total 50.04 last 83.19) total: 4500 open/close in 52.40 seconds: 85.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1678 (time 1786756396.66 total 10.00 last 167.73) - /unlink 3087 (time 1786756406.66 total 20.00 last 140.90) total: 4500 /unlink in 28.30 seconds: 159.03 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 841 (time 1786756429.72 total 10.00 last 84.07) - open/close 1736 (time 1786756439.73 total 20.01 last 89.44) - open/close 2445 (time 1786756449.73 total 30.01 last 70.90) - open/close 3474 (time 1786756459.74 total 40.01 last 102.85) - open/close 4412 (time 1786756469.74 total 50.02 last 93.80) total: 4500 open/close in 50.90 seconds: 88.41 ops/second - /unlink 1899 (time 1786756484.91 total 10.00 last 189.90) - /unlink 3353 (time 1786756494.91 total 20.00 last 145.35) total: 4500 /unlink in 26.17 seconds: 171.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 1006 (time 1786756515.08 total 10.01 last 100.49) - open/close 1950 (time 1786756525.10 total 20.03 last 94.25) - open/close 2788 (time 1786756535.10 total 30.03 last 83.79) - open/close 3773 (time 1786756545.10 total 40.03 last 98.49) total: 4500 open/close in 47.75 seconds: 94.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1663 (time 1786756567.03 total 10.01 last 166.21) - /unlink 3405 (time 1786756577.03 total 20.01 last 174.18) total: 4500 /unlink in 27.07 seconds: 166.24 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 880 (time 1786756598.52 total 10.01 last 87.87) - open/close 1788 (time 1786756608.53 total 20.02 last 90.72) - open/close 2725 (time 1786756618.54 total 30.03 last 93.65) - open/close 3561 (time 1786756628.54 total 40.03 last 83.57) - open/close 4364 (time 1786756638.55 total 50.04 last 80.25) total: 4500 open/close in 51.36 seconds: 87.61 ops/second - /unlink 1212 (time 1786756654.31 total 10.00 last 121.14) - /unlink 2718 (time 1786756664.32 total 20.02 last 150.44) - /unlink 4498 (time 1786756674.32 total 30.02 last 177.98) total: 4500 /unlink in 30.02 seconds: 149.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 934 (time 1786756688.24 total 10.01 last 93.31) - open/close 1853 (time 1786756698.25 total 20.02 last 91.82) - open/close 2840 (time 1786756708.26 total 30.03 last 98.61) - open/close 3775 (time 1786756718.27 total 40.04 last 93.40) total: 4500 open/close in 48.16 seconds: 93.44 ops/second - /unlink 1845 (time 1786756739.96 total 10.00 last 184.50) - /unlink 3446 (time 1786756749.97 total 20.01 last 159.89) total: 4500 /unlink in 26.50 seconds: 169.78 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 693 (time 1786756770.13 total 10.02 last 69.18) - open/close 1594 (time 1786756780.13 total 20.02 last 90.09) - open/close 2494 (time 1786756790.13 total 30.02 last 89.98) - open/close 3342 (time 1786756800.15 total 40.03 last 84.69) - open/close 4301 (time 1786756810.17 total 50.06 last 95.69) total: 4500 open/close in 51.79 seconds: 86.89 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1547 (time 1786756825.52 total 10.01 last 154.62) - /unlink 3029 (time 1786756835.53 total 20.01 last 148.16) - /unlink 4140 (time 1786756845.53 total 30.01 last 111.05) total: 4500 /unlink in 34.03 seconds: 132.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 735 (time 1786756865.05 total 10.00 last 73.50) - open/close 1614 (time 1786756875.07 total 20.01 last 87.78) - open/close 2406 (time 1786756885.08 total 30.02 last 79.13) - open/close 3163 (time 1786756895.08 total 40.03 last 75.67) - open/close 3582 (time 1786756905.09 total 50.04 last 41.85) - open/close 4294 (time 1786756915.11 total 60.05 last 71.11) total: 4500 open/close in 62.87 seconds: 71.58 ops/second - /unlink 1577 (time 1786756933.01 total 10.01 last 157.57) - /unlink 3167 (time 1786756943.01 total 20.01 last 158.92) total: 4500 /unlink in 27.57 seconds: 163.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 926 (time 1786756965.17 total 10.01 last 92.51) - open/close 1781 (time 1786756975.17 total 20.01 last 85.47) - open/close 2806 (time 1786756985.20 total 30.04 last 102.23) - open/close 3810 (time 1786756995.20 total 40.04 last 100.35) total: 4500 open/close in 47.20 seconds: 95.33 ops/second - /unlink 1628 (time 1786757016.76 total 10.01 last 162.71) - /unlink 3253 (time 1786757026.76 total 20.01 last 162.43) total: 4500 /unlink in 27.52 seconds: 163.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 958 (time 1786757049.05 total 10.00 last 95.77) - open/close 1866 (time 1786757059.05 total 20.01 last 90.76) - open/close 2641 (time 1786757069.05 total 30.01 last 77.47) - open/close 3512 (time 1786757079.06 total 40.01 last 87.08) - open/close 4435 (time 1786757089.06 total 50.01 last 92.29) total: 4500 open/close in 50.75 seconds: 88.67 ops/second - /unlink 1547 (time 1786757105.38 total 10.00 last 154.64) - /unlink 3042 (time 1786757115.38 total 20.01 last 149.47) - /unlink 4407 (time 1786757125.39 total 30.01 last 136.45) total: 4500 /unlink in 31.04 seconds: 144.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 802 (time 1786757141.04 total 10.00 last 80.17) - open/close 1444 (time 1786757151.06 total 20.02 last 64.07) - open/close 2229 (time 1786757161.07 total 30.03 last 78.45) - open/close 2894 (time 1786757171.08 total 40.04 last 66.42) - open/close 3419 (time 1786757181.09 total 50.05 last 52.44) - open/close 4238 (time 1786757191.10 total 60.06 last 81.81) total: 4500 open/close in 63.83 seconds: 70.50 ops/second - /unlink 1562 (time 1786757210.49 total 10.00 last 156.18) - /unlink 3028 (time 1786757220.49 total 20.00 last 146.58) total: 4500 /unlink in 29.57 seconds: 152.16 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 827 (time 1786757244.39 total 10.01 last 82.66) - open/close 1758 (time 1786757254.39 total 20.01 last 93.09) - open/close 2583 (time 1786757264.40 total 30.01 last 82.46) - open/close 3573 (time 1786757274.41 total 40.03 last 98.85) total: 4500 open/close in 49.70 seconds: 90.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1117 (time 1786757299.11 total 10.04 last 111.27) - /unlink 2072 (time 1786757309.12 total 20.04 last 95.47) - /unlink 3107 (time 1786757319.12 total 30.04 last 103.49) - /unlink 4357 (time 1786757329.13 total 40.05 last 124.90) total: 4500 /unlink in 41.61 seconds: 108.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 846 (time 1786757345.86 total 10.01 last 84.54) - open/close 1656 (time 1786757355.87 total 20.01 last 80.95) - open/close 2432 (time 1786757365.87 total 30.02 last 77.56) - open/close 3264 (time 1786757375.88 total 40.03 last 83.10) - open/close 4157 (time 1786757385.89 total 50.04 last 89.21) total: 4500 open/close in 54.54 seconds: 82.51 ops/second - /unlink 1633 (time 1786757405.78 total 10.00 last 163.26) - /unlink 2709 (time 1786757415.79 total 20.01 last 107.57) - /unlink 4149 (time 1786757425.80 total 30.02 last 143.77) total: 4500 /unlink in 32.83 seconds: 137.08 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 868 (time 1786757443.38 total 10.07 last 86.19) - open/close 1803 (time 1786757453.38 total 20.07 last 93.50) - open/close 2622 (time 1786757463.39 total 30.08 last 81.83) - open/close 3561 (time 1786757473.39 total 40.08 last 93.88) - open/close 4341 (time 1786757483.43 total 50.11 last 77.76) total: 4500 open/close in 53.02 seconds: 84.88 ops/second - /unlink 1445 (time 1786757501.42 total 10.00 last 144.47) - /unlink 3125 (time 1786757511.42 total 20.00 last 167.99) total: 4500 /unlink in 28.87 seconds: 155.86 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11323: 95204 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg252-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg252-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg252-server stop mds service on oleg252-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg252-server stop mds service on oleg252-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg252-server unloading modules via unload_modules_local on: 'oleg252-server' oleg252-server: oleg252-server.virtnet: executing unload_modules_local oleg252-server: modules unloaded.