== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 13:46:24 (1786815984) Stopping clients: oleg120-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg120-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg120-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg120-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg120-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg120-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg120-server oleg120-server: oleg120-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg120-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg120-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg120-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg120-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg120-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg120-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 787 (time 1786816191.56 total 10.03 last 78.48) - open/close 1693 (time 1786816201.57 total 20.04 last 90.53) - open/close 2707 (time 1786816211.57 total 30.04 last 101.35) - open/close 3763 (time 1786816221.58 total 40.05 last 105.50) total: 4500 open/close in 48.25 seconds: 93.26 ops/second - /unlink 1630 (time 1786816243.80 total 10.00 last 162.94) - /unlink 3240 (time 1786816253.81 total 20.01 last 160.89) total: 4500 /unlink in 28.42 seconds: 158.36 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 816 (time 1786816276.35 total 10.02 last 81.45) - open/close 1596 (time 1786816286.36 total 20.02 last 77.96) - open/close 2537 (time 1786816296.36 total 30.02 last 94.10) - open/close 3377 (time 1786816306.36 total 40.02 last 84.00) - open/close 4253 (time 1786816316.37 total 50.04 last 87.50) total: 4500 open/close in 52.48 seconds: 85.74 ops/second - /unlink 1680 (time 1786816332.61 total 10.00 last 167.95) - /unlink 3185 (time 1786816342.61 total 20.00 last 150.48) total: 4500 /unlink in 27.36 seconds: 164.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 955 (time 1786816363.69 total 10.01 last 95.37) - open/close 1756 (time 1786816373.72 total 20.05 last 79.85) - open/close 2634 (time 1786816383.73 total 30.06 last 87.70) - open/close 3518 (time 1786816393.74 total 40.06 last 88.33) - open/close 4364 (time 1786816403.74 total 50.07 last 84.55) total: 4500 open/close in 51.76 seconds: 86.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1721 (time 1786816419.79 total 10.00 last 172.08) - /unlink 3170 (time 1786816429.79 total 20.00 last 144.88) total: 4500 /unlink in 27.52 seconds: 163.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 786 (time 1786816452.93 total 10.01 last 78.50) - open/close 1655 (time 1786816462.94 total 20.02 last 86.84) - open/close 2474 (time 1786816472.95 total 30.03 last 81.83) - open/close 3352 (time 1786816482.95 total 40.03 last 87.76) - open/close 4206 (time 1786816492.95 total 50.04 last 85.39) total: 4500 open/close in 53.36 seconds: 84.33 ops/second - /unlink 1704 (time 1786816510.68 total 10.00 last 170.33) - /unlink 3205 (time 1786816520.69 total 20.01 last 149.95) total: 4500 /unlink in 28.37 seconds: 158.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 920 (time 1786816543.58 total 10.02 last 91.83) - open/close 1917 (time 1786816553.58 total 20.02 last 99.67) - open/close 2872 (time 1786816563.58 total 30.02 last 95.48) - open/close 3668 (time 1786816573.59 total 40.03 last 79.57) - open/close 4360 (time 1786816583.59 total 50.03 last 69.20) total: 4500 open/close in 51.72 seconds: 87.01 ops/second - /unlink 1569 (time 1786816599.64 total 10.00 last 156.84) - /unlink 3304 (time 1786816609.64 total 20.00 last 173.49) total: 4500 /unlink in 27.92 seconds: 161.16 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 774 (time 1786816632.06 total 10.01 last 77.34) - open/close 1641 (time 1786816642.06 total 20.01 last 86.67) - open/close 2494 (time 1786816652.06 total 30.01 last 85.27) - open/close 3381 (time 1786816662.07 total 40.02 last 88.63) - open/close 4374 (time 1786816672.08 total 50.03 last 99.21) total: 4500 open/close in 51.40 seconds: 87.56 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1601 (time 1786816687.83 total 10.00 last 160.05) - /unlink 2825 (time 1786816697.84 total 20.02 last 122.23) total: 4500 /unlink in 29.97 seconds: 150.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 954 (time 1786816722.35 total 10.00 last 95.39) - open/close 1871 (time 1786816732.36 total 20.00 last 91.67) - open/close 2786 (time 1786816742.37 total 30.01 last 91.41) - open/close 3685 (time 1786816752.37 total 40.02 last 89.88) total: 4500 open/close in 49.57 seconds: 90.79 ops/second - /unlink 1539 (time 1786816776.08 total 10.00 last 153.88) - /unlink 3197 (time 1786816786.09 total 20.00 last 165.75) total: 4500 /unlink in 29.31 seconds: 153.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 800 (time 1786816811.00 total 10.02 last 79.82) - open/close 1671 (time 1786816821.02 total 20.03 last 86.99) - open/close 2529 (time 1786816831.02 total 30.04 last 85.75) - open/close 3492 (time 1786816841.03 total 40.05 last 96.25) - open/close 4384 (time 1786816851.04 total 50.06 last 89.10) total: 4500 open/close in 51.40 seconds: 87.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1381 (time 1786816867.21 total 10.01 last 138.01) - /unlink 2685 (time 1786816877.22 total 20.01 last 130.36) - /unlink 3647 (time 1786816887.23 total 30.02 last 96.06) total: 4500 /unlink in 37.03 seconds: 121.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 835 (time 1786816909.09 total 10.00 last 83.46) - open/close 1609 (time 1786816919.10 total 20.02 last 77.29) - open/close 2378 (time 1786816929.11 total 30.03 last 76.84) - open/close 3149 (time 1786816939.12 total 40.03 last 77.04) - open/close 4114 (time 1786816949.12 total 50.04 last 96.46) total: 4500 open/close in 54.33 seconds: 82.83 ops/second - /unlink 1546 (time 1786816968.25 total 10.01 last 154.43) - /unlink 3087 (time 1786816978.25 total 20.01 last 154.09) - /unlink 4387 (time 1786816988.25 total 30.02 last 129.95) total: 4500 /unlink in 31.14 seconds: 144.52 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 710 (time 1786817005.42 total 10.00 last 70.97) - open/close 1367 (time 1786817015.43 total 20.01 last 65.68) - open/close 2192 (time 1786817025.44 total 30.01 last 82.44) - open/close 3062 (time 1786817035.45 total 40.02 last 86.91) - open/close 3928 (time 1786817045.47 total 50.05 last 86.39) total: 4500 open/close in 56.54 seconds: 79.60 ops/second - /unlink 1673 (time 1786817066.13 total 10.00 last 167.27) - /unlink 3169 (time 1786817076.13 total 20.00 last 149.58) total: 4500 /unlink in 29.39 seconds: 153.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 797 (time 1786817100.46 total 10.01 last 79.60) - open/close 1492 (time 1786817110.46 total 20.02 last 69.48) - open/close 2343 (time 1786817120.47 total 30.02 last 85.03) - open/close 3079 (time 1786817130.48 total 40.03 last 73.53) - open/close 3785 (time 1786817140.49 total 50.04 last 70.51) - open/close 4496 (time 1786817150.50 total 60.05 last 71.06) total: 4500 open/close in 60.12 seconds: 74.86 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1750 (time 1786817166.15 total 10.00 last 174.95) - /unlink 3283 (time 1786817176.15 total 20.01 last 153.25) total: 4500 /unlink in 27.70 seconds: 162.44 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 768 (time 1786817198.74 total 10.00 last 76.77) - open/close 1624 (time 1786817208.76 total 20.01 last 85.51) - open/close 2493 (time 1786817218.76 total 30.02 last 86.89) - open/close 3344 (time 1786817228.76 total 40.02 last 85.06) - open/close 4262 (time 1786817238.77 total 50.03 last 91.71) total: 4500 open/close in 52.17 seconds: 86.26 ops/second - /unlink 1624 (time 1786817255.37 total 10.00 last 162.34) - /unlink 3063 (time 1786817265.37 total 20.00 last 143.89) - /unlink 4464 (time 1786817275.37 total 30.01 last 140.05) total: 4500 /unlink in 30.38 seconds: 148.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 952 (time 1786817291.11 total 10.00 last 95.17) - open/close 1941 (time 1786817301.11 total 20.01 last 98.85) - open/close 2876 (time 1786817311.11 total 30.01 last 93.47) - open/close 3728 (time 1786817321.12 total 40.01 last 85.20) total: 4500 open/close in 48.80 seconds: 92.21 ops/second - /unlink 1475 (time 1786817344.94 total 10.03 last 147.08) - /unlink 3218 (time 1786817354.94 total 20.03 last 174.26) total: 4500 /unlink in 29.61 seconds: 151.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 820 (time 1786817380.35 total 10.00 last 82.00) - open/close 1773 (time 1786817390.36 total 20.00 last 95.26) - open/close 2658 (time 1786817400.36 total 30.01 last 88.49) - open/close 3500 (time 1786817410.37 total 40.01 last 84.13) - open/close 4362 (time 1786817420.38 total 50.03 last 86.07) total: 4500 open/close in 51.94 seconds: 86.64 ops/second - /unlink 1606 (time 1786817436.62 total 10.00 last 160.60) - /unlink 3154 (time 1786817446.62 total 20.00 last 154.78) total: 4500 /unlink in 28.88 seconds: 155.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 617 (time 1786817470.85 total 10.01 last 61.63) - open/close 1262 (time 1786817480.87 total 20.03 last 64.38) - open/close 2056 (time 1786817490.88 total 30.05 last 79.27) - open/close 2811 (time 1786817500.89 total 40.05 last 75.44) - open/close 3635 (time 1786817510.90 total 50.06 last 82.35) - open/close 4443 (time 1786817520.90 total 60.07 last 80.76) total: 4500 open/close in 60.64 seconds: 74.21 ops/second - /unlink 1558 (time 1786817535.78 total 10.01 last 155.69) - /unlink 2990 (time 1786817545.78 total 20.01 last 143.16) - /unlink 4459 (time 1786817555.78 total 30.01 last 146.88) total: 4500 /unlink in 30.37 seconds: 148.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 832 (time 1786817570.18 total 10.02 last 83.04) - open/close 1663 (time 1786817580.19 total 20.03 last 83.04) - open/close 2581 (time 1786817590.19 total 30.03 last 91.79) - open/close 3341 (time 1786817600.19 total 40.03 last 75.95) - open/close 4073 (time 1786817610.20 total 50.04 last 73.17) total: 4500 open/close in 56.01 seconds: 80.34 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1687 (time 1786817631.06 total 10.01 last 168.51) - /unlink 3371 (time 1786817641.07 total 20.01 last 168.34) total: 4500 /unlink in 27.23 seconds: 165.27 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 861 (time 1786817663.02 total 10.01 last 86.01) - open/close 1724 (time 1786817673.02 total 20.01 last 86.29) - open/close 2457 (time 1786817683.03 total 30.02 last 73.26) - open/close 3336 (time 1786817693.04 total 40.03 last 87.82) - open/close 4293 (time 1786817703.04 total 50.03 last 95.65) total: 4500 open/close in 52.30 seconds: 86.04 ops/second - /unlink 1451 (time 1786817720.80 total 10.00 last 145.07) - /unlink 2647 (time 1786817730.80 total 20.00 last 119.60) - /unlink 4201 (time 1786817740.80 total 30.00 last 155.38) total: 4500 /unlink in 31.96 seconds: 140.81 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 880 (time 1786817759.14 total 10.01 last 87.94) - open/close 1606 (time 1786817769.15 total 20.01 last 72.59) - open/close 2525 (time 1786817779.15 total 30.01 last 91.86) - open/close 3446 (time 1786817789.16 total 40.02 last 92.02) - open/close 4330 (time 1786817799.17 total 50.03 last 88.33) total: 4500 open/close in 52.04 seconds: 86.48 ops/second - /unlink 1767 (time 1786817816.49 total 10.01 last 176.58) - /unlink 3099 (time 1786817826.50 total 20.02 last 133.05) - /unlink 4388 (time 1786817836.50 total 30.02 last 128.90) total: 4500 /unlink in 30.81 seconds: 146.07 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11316: 95282 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg120-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg120-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg120-server stop mds service on oleg120-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg120-server stop mds service on oleg120-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg120-server unloading modules via unload_modules_local on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing unload_modules_local oleg120-server: modules unloaded.