== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 23:38:32 (1786419512) Stopping clients: oleg442-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg442-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg442-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg442-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg442-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg442-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg442-server oleg442-server: oleg442-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg442-server' oleg442-server: oleg442-server.virtnet: executing load_modules_local oleg442-server: Loading modules from /home/green/git/lustre-release/lustre oleg442-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg442-server: detected 4 online CPUs by sysfs oleg442-server: MODOPTS_LIBCFS= oleg442-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg442-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg442-server: oleg442-server.virtnet: executing set_default_debug -1 all pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg442-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg442-server: oleg442-server.virtnet: executing set_default_debug -1 all pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg442-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg442-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg442-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg442-server: oleg442-server.virtnet: executing set_default_debug -1 all pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg442-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg442-client.virtnet: -o user_xattr,flock 192.168.204.142@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 815 (time 1786419731.15 total 10.01 last 81.39) - open/close 1581 (time 1786419741.16 total 20.02 last 76.55) - open/close 2363 (time 1786419751.16 total 30.02 last 78.17) - open/close 3214 (time 1786419761.17 total 40.03 last 85.06) - open/close 4044 (time 1786419771.18 total 50.04 last 82.93) total: 4500 open/close in 55.93 seconds: 80.46 ops/second - /unlink 1474 (time 1786419792.17 total 10.02 last 147.16) - /unlink 2746 (time 1786419802.18 total 20.03 last 127.08) - /unlink 4061 (time 1786419812.19 total 30.03 last 131.42) total: 4500 /unlink in 33.55 seconds: 134.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 750 (time 1786419832.55 total 10.00 last 74.98) - open/close 1473 (time 1786419842.56 total 20.01 last 72.26) - open/close 2232 (time 1786419852.56 total 30.01 last 75.90) - open/close 2961 (time 1786419862.56 total 40.01 last 72.88) - open/close 3691 (time 1786419872.57 total 50.02 last 72.93) - open/close 4401 (time 1786419882.58 total 60.03 last 70.95) total: 4500 open/close in 61.28 seconds: 73.43 ops/second - /unlink 1315 (time 1786419898.50 total 10.01 last 131.39) - /unlink 2573 (time 1786419908.50 total 20.01 last 125.75) - /unlink 3908 (time 1786419918.50 total 30.02 last 133.46) total: 4500 /unlink in 34.15 seconds: 131.76 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 768 (time 1786419938.69 total 10.01 last 76.75) - open/close 1451 (time 1786419948.70 total 20.02 last 68.19) - open/close 2154 (time 1786419958.71 total 30.03 last 70.24) - open/close 2893 (time 1786419968.71 total 40.03 last 73.89) - open/close 3737 (time 1786419978.72 total 50.04 last 84.35) - open/close 4405 (time 1786419988.72 total 60.05 last 66.75) total: 4500 open/close in 61.34 seconds: 73.36 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1264 (time 1786420005.77 total 10.00 last 126.39) - /unlink 2557 (time 1786420015.78 total 20.01 last 129.12) - /unlink 3882 (time 1786420025.79 total 30.02 last 132.42) total: 4500 /unlink in 34.50 seconds: 130.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 803 (time 1786420045.45 total 10.01 last 80.24) - open/close 1608 (time 1786420055.46 total 20.02 last 80.44) - open/close 2361 (time 1786420065.46 total 30.02 last 75.26) - open/close 3131 (time 1786420075.46 total 40.02 last 77.00) - open/close 3883 (time 1786420085.46 total 50.02 last 75.18) total: 4500 open/close in 58.31 seconds: 77.17 ops/second - /unlink 1295 (time 1786420108.18 total 10.01 last 129.38) - /unlink 2599 (time 1786420118.18 total 20.01 last 130.40) - /unlink 3941 (time 1786420128.20 total 30.02 last 133.99) total: 4500 /unlink in 34.25 seconds: 131.38 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 781 (time 1786420147.97 total 10.01 last 78.03) - open/close 1546 (time 1786420157.97 total 20.02 last 76.44) - open/close 2387 (time 1786420167.98 total 30.02 last 84.04) - open/close 3194 (time 1786420177.98 total 40.03 last 80.68) - open/close 4003 (time 1786420187.99 total 50.03 last 80.87) total: 4500 open/close in 56.41 seconds: 79.77 ops/second - /unlink 1395 (time 1786420209.11 total 10.00 last 139.50) - /unlink 2694 (time 1786420219.11 total 20.00 last 129.90) - /unlink 3932 (time 1786420229.11 total 30.00 last 123.80) total: 4500 /unlink in 34.21 seconds: 131.52 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 737 (time 1786420248.76 total 10.00 last 73.69) - open/close 1398 (time 1786420258.76 total 20.00 last 66.09) - open/close 2122 (time 1786420268.77 total 30.01 last 72.33) - open/close 2831 (time 1786420278.79 total 40.02 last 70.82) - open/close 3593 (time 1786420288.79 total 50.03 last 76.16) - open/close 4341 (time 1786420298.80 total 60.04 last 74.74) total: 4500 open/close in 62.22 seconds: 72.33 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1372 (time 1786420315.76 total 10.00 last 137.17) - /unlink 2640 (time 1786420325.76 total 20.00 last 126.77) - /unlink 3914 (time 1786420335.76 total 30.01 last 127.34) total: 4500 /unlink in 33.99 seconds: 132.38 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 741 (time 1786420355.68 total 10.01 last 74.04) - open/close 1447 (time 1786420365.69 total 20.02 last 70.53) - open/close 2241 (time 1786420375.69 total 30.02 last 79.36) - open/close 3023 (time 1786420385.70 total 40.03 last 78.17) - open/close 3778 (time 1786420395.70 total 50.03 last 75.49) total: 4500 open/close in 58.79 seconds: 76.55 ops/second - /unlink 1400 (time 1786420419.28 total 10.00 last 139.98) - /unlink 2652 (time 1786420429.29 total 20.01 last 125.13) - /unlink 3956 (time 1786420439.29 total 30.01 last 130.38) total: 4500 /unlink in 34.29 seconds: 131.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 771 (time 1786420459.07 total 10.00 last 77.08) - open/close 1586 (time 1786420469.08 total 20.01 last 81.41) - open/close 2389 (time 1786420479.08 total 30.01 last 80.29) - open/close 3067 (time 1786420489.09 total 40.02 last 67.74) - open/close 3715 (time 1786420499.10 total 50.03 last 64.74) - open/close 4391 (time 1786420509.11 total 60.04 last 67.56) total: 4500 open/close in 61.86 seconds: 72.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1261 (time 1786420527.10 total 10.00 last 126.05) - /unlink 2503 (time 1786420537.10 total 20.00 last 124.19) - /unlink 3806 (time 1786420547.10 total 30.01 last 130.27) total: 4500 /unlink in 35.46 seconds: 126.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 773 (time 1786420567.98 total 10.01 last 77.24) - open/close 1548 (time 1786420577.98 total 20.01 last 77.47) - open/close 2278 (time 1786420587.99 total 30.02 last 72.95) - open/close 3036 (time 1786420598.00 total 40.02 last 75.76) - open/close 3823 (time 1786420608.00 total 50.03 last 78.64) total: 4500 open/close in 58.92 seconds: 76.37 ops/second - /unlink 1454 (time 1786420632.41 total 10.01 last 145.33) - /unlink 2911 (time 1786420642.41 total 20.01 last 145.67) - /unlink 4124 (time 1786420652.42 total 30.01 last 121.22) total: 4500 /unlink in 33.06 seconds: 136.12 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 734 (time 1786420670.91 total 10.01 last 73.36) - open/close 1522 (time 1786420680.92 total 20.01 last 78.73) - open/close 2272 (time 1786420690.93 total 30.03 last 74.89) - open/close 2993 (time 1786420700.93 total 40.03 last 72.07) - open/close 3681 (time 1786420710.95 total 50.05 last 68.66) - open/close 4388 (time 1786420720.97 total 60.07 last 70.60) total: 4500 open/close in 61.51 seconds: 73.16 ops/second - /unlink 1220 (time 1786420737.27 total 10.00 last 121.97) - /unlink 2439 (time 1786420747.28 total 20.01 last 121.80) - /unlink 3619 (time 1786420757.28 total 30.01 last 117.98) total: 4500 /unlink in 36.82 seconds: 122.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 726 (time 1786420780.44 total 10.02 last 72.45) - open/close 1473 (time 1786420790.44 total 20.02 last 74.69) - open/close 2123 (time 1786420800.46 total 30.04 last 64.91) - open/close 2751 (time 1786420810.46 total 40.04 last 62.77) - open/close 3502 (time 1786420820.47 total 50.05 last 75.04) - open/close 4269 (time 1786420830.48 total 60.06 last 76.59) total: 4500 open/close in 62.91 seconds: 71.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1325 (time 1786420849.47 total 10.00 last 132.48) - /unlink 2663 (time 1786420859.47 total 20.00 last 133.79) - /unlink 3844 (time 1786420869.48 total 30.00 last 118.08) total: 4500 /unlink in 35.98 seconds: 125.08 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 799 (time 1786420890.29 total 10.00 last 79.90) - open/close 1591 (time 1786420900.30 total 20.01 last 79.14) - open/close 2337 (time 1786420910.32 total 30.02 last 74.49) - open/close 3093 (time 1786420920.32 total 40.02 last 75.60) - open/close 3775 (time 1786420930.32 total 50.03 last 68.19) - open/close 4492 (time 1786420940.32 total 60.03 last 71.70) total: 4500 open/close in 60.11 seconds: 74.87 ops/second - /unlink 1398 (time 1786420955.65 total 10.00 last 139.77) - /unlink 2645 (time 1786420965.65 total 20.00 last 124.68) - /unlink 3812 (time 1786420975.67 total 30.02 last 116.55) total: 4500 /unlink in 35.43 seconds: 127.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 713 (time 1786420996.46 total 10.01 last 71.25) - open/close 1537 (time 1786421006.46 total 20.01 last 82.38) - open/close 2300 (time 1786421016.48 total 30.02 last 76.20) - open/close 3012 (time 1786421026.49 total 40.03 last 71.11) - open/close 3786 (time 1786421036.49 total 50.04 last 77.37) total: 4500 open/close in 59.41 seconds: 75.75 ops/second - /unlink 1267 (time 1786421061.49 total 10.00 last 126.68) - /unlink 2377 (time 1786421071.50 total 20.00 last 110.97) - /unlink 3589 (time 1786421081.50 total 30.01 last 121.17) total: 4500 /unlink in 37.56 seconds: 119.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 766 (time 1786421104.66 total 10.01 last 76.52) - open/close 1505 (time 1786421114.66 total 20.01 last 73.88) - open/close 2222 (time 1786421124.67 total 30.02 last 71.63) - open/close 2974 (time 1786421134.67 total 40.02 last 75.19) - open/close 3770 (time 1786421144.71 total 50.07 last 79.26) total: 4500 open/close in 59.62 seconds: 75.48 ops/second - /unlink 1323 (time 1786421168.89 total 10.00 last 132.26) - /unlink 2442 (time 1786421178.90 total 20.01 last 111.87) - /unlink 3760 (time 1786421188.91 total 30.01 last 131.69) total: 4500 /unlink in 36.04 seconds: 124.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 787 (time 1786421210.76 total 10.01 last 78.62) - open/close 1489 (time 1786421220.76 total 20.01 last 70.19) - open/close 2239 (time 1786421230.77 total 30.02 last 74.96) - open/close 3027 (time 1786421240.78 total 40.03 last 78.72) - open/close 3782 (time 1786421250.78 total 50.03 last 75.47) total: 4500 open/close in 58.78 seconds: 76.55 ops/second - /unlink 1352 (time 1786421274.20 total 10.00 last 135.20) - /unlink 2595 (time 1786421284.20 total 20.00 last 124.26) - /unlink 3906 (time 1786421294.20 total 30.01 last 131.03) total: 4500 /unlink in 34.67 seconds: 129.81 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 778 (time 1786421314.69 total 10.00 last 77.79) - open/close 1496 (time 1786421324.69 total 20.00 last 71.79) - open/close 2308 (time 1786421334.70 total 30.01 last 81.15) - open/close 3143 (time 1786421344.70 total 40.01 last 83.49) - open/close 4006 (time 1786421354.70 total 50.01 last 86.26) total: 4500 open/close in 56.27 seconds: 79.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1401 (time 1786421375.71 total 10.00 last 140.06) - /unlink 2654 (time 1786421385.72 total 20.01 last 125.19) - /unlink 3976 (time 1786421395.72 total 30.01 last 132.20) total: 4500 /unlink in 34.23 seconds: 131.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 753 (time 1786421414.98 total 10.00 last 75.28) - open/close 1462 (time 1786421424.99 total 20.01 last 70.85) - open/close 2253 (time 1786421435.00 total 30.02 last 79.03) - open/close 3041 (time 1786421445.00 total 40.02 last 78.79) - open/close 3772 (time 1786421455.02 total 50.04 last 72.98) - open/close 4459 (time 1786421465.03 total 60.05 last 68.58) total: 4500 open/close in 60.59 seconds: 74.27 ops/second - /unlink 1273 (time 1786421482.46 total 10.00 last 127.29) - /unlink 2471 (time 1786421492.46 total 20.00 last 119.78) - /unlink 3817 (time 1786421502.46 total 30.01 last 134.56) total: 4500 /unlink in 35.12 seconds: 128.14 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 796 (time 1786421522.37 total 10.01 last 79.53) - open/close 1598 (time 1786421532.38 total 20.01 last 80.16) - open/close 2422 (time 1786421542.39 total 30.02 last 82.34) - open/close 3206 (time 1786421552.39 total 40.03 last 78.36) - open/close 3975 (time 1786421562.40 total 50.03 last 76.86) total: 4500 open/close in 56.81 seconds: 79.21 ops/second - /unlink 1300 (time 1786421585.01 total 10.00 last 129.96) - /unlink 2524 (time 1786421595.02 total 20.01 last 122.34) - /unlink 3786 (time 1786421605.02 total 30.01 last 126.20) total: 4500 /unlink in 35.33 seconds: 127.37 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11316: 95434 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg442-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg442-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg442-server stop mds service on oleg442-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg442-server stop mds service on oleg442-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg442-server unloading modules via unload_modules_local on: 'oleg442-server' oleg442-server: oleg442-server.virtnet: executing unload_modules_local oleg442-server: modules unloaded.