== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 18:30:33 (1786314633) Stopping clients: oleg211-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg211-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg211-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg211-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg211-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg211-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg211-server oleg211-server: oleg211-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg211-server' oleg211-server: oleg211-server.virtnet: executing load_modules_local oleg211-server: Loading modules from /home/green/git/lustre-release/lustre oleg211-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg211-server: detected 4 online CPUs by sysfs oleg211-server: MODOPTS_LIBCFS= oleg211-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg211-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg211-server: oleg211-server.virtnet: executing set_default_debug -1 all pdsh@oleg211-client: oleg211-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg211-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg211-server: oleg211-server.virtnet: executing set_default_debug -1 all pdsh@oleg211-client: oleg211-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg211-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg211-server: oleg211-server.virtnet: executing set_default_debug -1 all pdsh@oleg211-client: oleg211-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg211-client.virtnet: -o user_xattr,flock 192.168.202.111@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 797 (time 1786314849.26 total 10.00 last 79.69) - open/close 1789 (time 1786314859.26 total 20.01 last 99.14) - open/close 2783 (time 1786314869.27 total 30.01 last 99.34) - open/close 3558 (time 1786314879.29 total 40.04 last 77.34) - open/close 4173 (time 1786314889.29 total 50.04 last 61.47) total: 4500 open/close in 53.86 seconds: 83.55 ops/second - /unlink 1619 (time 1786314906.74 total 10.00 last 161.87) - /unlink 3122 (time 1786314916.75 total 20.01 last 150.24) total: 4500 /unlink in 29.21 seconds: 154.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 627 (time 1786314941.21 total 10.02 last 62.60) - open/close 1336 (time 1786314951.21 total 20.02 last 70.87) - open/close 1862 (time 1786314961.23 total 30.04 last 52.52) - open/close 2434 (time 1786314971.24 total 40.05 last 57.12) - open/close 3211 (time 1786314981.25 total 50.06 last 77.66) - open/close 4018 (time 1786314991.25 total 60.06 last 80.65) total: 4500 open/close in 65.51 seconds: 68.69 ops/second - /unlink 1692 (time 1786315011.02 total 10.00 last 169.17) - /unlink 3229 (time 1786315021.02 total 20.00 last 153.66) total: 4500 /unlink in 27.85 seconds: 161.59 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 827 (time 1786315043.95 total 10.00 last 82.69) - open/close 1646 (time 1786315053.97 total 20.01 last 81.79) - open/close 2385 (time 1786315063.97 total 30.02 last 73.88) - open/close 3363 (time 1786315073.98 total 40.02 last 97.73) - open/close 4227 (time 1786315083.98 total 50.03 last 86.39) total: 4500 open/close in 53.13 seconds: 84.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1697 (time 1786315102.18 total 10.00 last 169.68) - /unlink 3060 (time 1786315112.18 total 20.00 last 136.30) - /unlink 4238 (time 1786315122.19 total 30.02 last 117.61) total: 4500 /unlink in 32.30 seconds: 139.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 920 (time 1786315139.56 total 10.00 last 91.97) - open/close 1776 (time 1786315149.56 total 20.00 last 85.60) - open/close 2699 (time 1786315159.56 total 30.01 last 92.22) - open/close 3483 (time 1786315169.57 total 40.01 last 78.39) - open/close 4272 (time 1786315179.57 total 50.02 last 78.86) total: 4500 open/close in 52.32 seconds: 86.01 ops/second - /unlink 1299 (time 1786315196.11 total 10.00 last 129.85) - /unlink 2654 (time 1786315206.11 total 20.01 last 135.48) - /unlink 4117 (time 1786315216.12 total 30.01 last 146.30) total: 4500 /unlink in 32.75 seconds: 137.39 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 683 (time 1786315234.50 total 10.01 last 68.20) - open/close 1343 (time 1786315244.52 total 20.04 last 65.86) - open/close 2037 (time 1786315254.53 total 30.05 last 69.30) - open/close 2785 (time 1786315264.54 total 40.06 last 74.74) - open/close 3516 (time 1786315274.54 total 50.06 last 73.09) - open/close 4313 (time 1786315284.55 total 60.07 last 79.62) total: 4500 open/close in 62.65 seconds: 71.83 ops/second - /unlink 1353 (time 1786315302.45 total 10.00 last 135.25) - /unlink 2803 (time 1786315312.45 total 20.00 last 144.98) - /unlink 4106 (time 1786315322.45 total 30.01 last 130.29) total: 4500 /unlink in 33.32 seconds: 135.07 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 711 (time 1786315342.21 total 10.00 last 71.06) - open/close 1528 (time 1786315352.21 total 20.01 last 81.66) - open/close 2297 (time 1786315362.22 total 30.02 last 76.85) - open/close 3045 (time 1786315372.22 total 40.02 last 74.78) - open/close 3809 (time 1786315382.23 total 50.03 last 76.30) - open/close 4493 (time 1786315392.24 total 60.04 last 68.35) total: 4500 open/close in 60.14 seconds: 74.83 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1501 (time 1786315407.69 total 10.00 last 150.08) - /unlink 3052 (time 1786315417.69 total 20.01 last 155.01) - /unlink 4163 (time 1786315427.71 total 30.02 last 110.95) total: 4500 /unlink in 33.33 seconds: 135.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 586 (time 1786315447.50 total 10.00 last 58.58) - open/close 1118 (time 1786315457.50 total 20.00 last 53.19) - open/close 1745 (time 1786315467.51 total 30.01 last 62.65) - open/close 2359 (time 1786315477.53 total 40.03 last 61.30) - open/close 3065 (time 1786315487.53 total 50.03 last 70.56) - open/close 3885 (time 1786315497.54 total 60.04 last 81.95) total: 4500 open/close in 67.71 seconds: 66.46 ops/second - /unlink 1486 (time 1786315520.75 total 10.00 last 148.59) - /unlink 2902 (time 1786315530.76 total 20.00 last 141.57) - /unlink 4319 (time 1786315540.76 total 30.01 last 141.62) total: 4500 /unlink in 31.37 seconds: 143.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 729 (time 1786315558.17 total 10.00 last 72.86) - open/close 1385 (time 1786315568.20 total 20.03 last 65.42) - open/close 2125 (time 1786315578.20 total 30.04 last 73.96) - open/close 2869 (time 1786315588.21 total 40.05 last 74.34) - open/close 3580 (time 1786315598.21 total 50.05 last 71.09) - open/close 4309 (time 1786315608.22 total 60.06 last 72.83) total: 4500 open/close in 62.89 seconds: 71.56 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1626 (time 1786315625.76 total 10.00 last 162.53) - /unlink 2971 (time 1786315635.77 total 20.01 last 134.43) - /unlink 4340 (time 1786315645.77 total 30.01 last 136.85) total: 4500 /unlink in 31.18 seconds: 144.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 717 (time 1786315661.38 total 10.00 last 71.68) - open/close 1586 (time 1786315671.39 total 20.01 last 86.81) - open/close 2409 (time 1786315681.40 total 30.02 last 82.24) - open/close 3179 (time 1786315691.40 total 40.03 last 76.97) - open/close 3850 (time 1786315701.42 total 50.04 last 67.00) total: 4500 open/close in 58.25 seconds: 77.25 ops/second - /unlink 1441 (time 1786315724.92 total 10.00 last 144.09) - /unlink 2866 (time 1786315734.92 total 20.00 last 142.48) total: 4500 /unlink in 29.27 seconds: 153.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 737 (time 1786315759.28 total 10.01 last 73.63) - open/close 1455 (time 1786315769.29 total 20.02 last 71.72) - open/close 2138 (time 1786315779.30 total 30.03 last 68.24) - open/close 3035 (time 1786315789.33 total 40.06 last 89.45) - open/close 3715 (time 1786315799.33 total 50.06 last 67.97) total: 4500 open/close in 60.02 seconds: 74.98 ops/second - /unlink 1349 (time 1786315824.80 total 10.01 last 134.83) - /unlink 2901 (time 1786315834.80 total 20.01 last 155.14) - /unlink 4280 (time 1786315844.81 total 30.02 last 137.72) total: 4500 /unlink in 31.55 seconds: 142.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 760 (time 1786315861.75 total 10.01 last 75.94) - open/close 1594 (time 1786315871.76 total 20.01 last 83.36) - open/close 2313 (time 1786315881.76 total 30.02 last 71.87) - open/close 3215 (time 1786315891.77 total 40.02 last 90.15) - open/close 4032 (time 1786315901.77 total 50.03 last 81.64) total: 4500 open/close in 56.51 seconds: 79.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1566 (time 1786315923.20 total 10.00 last 156.60) - /unlink 2957 (time 1786315933.20 total 20.00 last 139.06) - /unlink 4489 (time 1786315943.20 total 30.00 last 153.18) total: 4500 /unlink in 30.15 seconds: 149.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 628 (time 1786315958.87 total 10.01 last 62.76) - open/close 1508 (time 1786315968.89 total 20.02 last 87.88) - open/close 2316 (time 1786315978.89 total 30.02 last 80.77) - open/close 3118 (time 1786315988.89 total 40.03 last 80.17) - open/close 3943 (time 1786315998.90 total 50.03 last 82.48) total: 4500 open/close in 56.95 seconds: 79.02 ops/second - /unlink 1396 (time 1786316019.84 total 10.01 last 139.53) - /unlink 2690 (time 1786316029.84 total 20.01 last 129.35) - /unlink 4267 (time 1786316039.85 total 30.01 last 157.67) total: 4500 /unlink in 31.62 seconds: 142.30 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 929 (time 1786316056.82 total 10.02 last 92.73) - open/close 1875 (time 1786316066.83 total 20.03 last 94.46) - open/close 2789 (time 1786316076.83 total 30.03 last 91.40) - open/close 3598 (time 1786316086.84 total 40.04 last 80.83) - open/close 4353 (time 1786316096.85 total 50.05 last 75.45) total: 4500 open/close in 51.95 seconds: 86.63 ops/second - /unlink 1343 (time 1786316113.51 total 10.00 last 134.27) - /unlink 2709 (time 1786316123.52 total 20.01 last 136.54) - /unlink 4278 (time 1786316133.52 total 30.01 last 156.88) total: 4500 /unlink in 31.48 seconds: 142.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 835 (time 1786316150.81 total 10.01 last 83.42) - open/close 1677 (time 1786316160.81 total 20.01 last 84.20) - open/close 2429 (time 1786316170.81 total 30.01 last 75.19) - open/close 3104 (time 1786316180.82 total 40.02 last 67.45) - open/close 3984 (time 1786316190.82 total 50.02 last 87.97) total: 4500 open/close in 56.06 seconds: 80.27 ops/second - /unlink 1655 (time 1786316210.74 total 10.00 last 165.47) - /unlink 3325 (time 1786316220.75 total 20.01 last 166.80) total: 4500 /unlink in 27.81 seconds: 161.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 881 (time 1786316243.17 total 10.01 last 88.04) - open/close 1580 (time 1786316253.18 total 20.02 last 69.82) - open/close 2340 (time 1786316263.18 total 30.02 last 75.98) - open/close 3064 (time 1786316273.19 total 40.02 last 72.38) - open/close 3879 (time 1786316283.19 total 50.03 last 81.46) total: 4500 open/close in 58.65 seconds: 76.73 ops/second - /unlink 1543 (time 1786316306.21 total 10.00 last 154.28) - /unlink 3113 (time 1786316316.22 total 20.01 last 156.92) total: 4500 /unlink in 29.29 seconds: 153.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 785 (time 1786316340.60 total 10.00 last 78.48) - open/close 1616 (time 1786316350.60 total 20.00 last 83.09) - open/close 2521 (time 1786316360.60 total 30.00 last 90.49) - open/close 3448 (time 1786316370.61 total 40.01 last 92.64) - open/close 4376 (time 1786316380.62 total 50.02 last 92.69) total: 4500 open/close in 51.36 seconds: 87.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1481 (time 1786316396.08 total 10.00 last 148.05) - /unlink 2806 (time 1786316406.10 total 20.02 last 132.32) - /unlink 4291 (time 1786316416.10 total 30.02 last 148.43) total: 4500 /unlink in 31.57 seconds: 142.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 853 (time 1786316432.67 total 10.01 last 85.23) - open/close 1713 (time 1786316442.71 total 20.04 last 85.70) - open/close 2416 (time 1786316452.72 total 30.05 last 70.25) - open/close 3323 (time 1786316462.72 total 40.05 last 90.66) - open/close 4217 (time 1786316472.72 total 50.06 last 89.36) total: 4500 open/close in 52.83 seconds: 85.18 ops/second - /unlink 1745 (time 1786316490.51 total 10.00 last 174.43) - /unlink 3335 (time 1786316500.52 total 20.01 last 158.83) total: 4500 /unlink in 27.78 seconds: 161.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 722 (time 1786316523.01 total 10.01 last 72.13) - open/close 1546 (time 1786316533.02 total 20.02 last 82.33) - open/close 2261 (time 1786316543.03 total 30.03 last 71.42) - open/close 3015 (time 1786316553.04 total 40.03 last 75.35) - open/close 3801 (time 1786316563.04 total 50.04 last 78.58) total: 4500 open/close in 59.28 seconds: 75.91 ops/second - /unlink 1679 (time 1786316587.84 total 10.00 last 167.84) - /unlink 3204 (time 1786316597.84 total 20.00 last 152.47) total: 4500 /unlink in 28.54 seconds: 157.68 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11316: 95308 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg211-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg211-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg211-server stop mds service on oleg211-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg211-server stop mds service on oleg211-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg211-server unloading modules via unload_modules_local on: 'oleg211-server' oleg211-server: oleg211-server.virtnet: executing unload_modules_local oleg211-server: modules unloaded.