== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 14:07:18 (1787854038) Stopping clients: oleg158-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg158-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg158-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg158-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg158-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg158-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg158-server oleg158-server: oleg158-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1049: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg158-server' oleg158-server: oleg158-server.virtnet: executing load_modules_local oleg158-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1049: echo: write error: Device or resource busy oleg158-server: Loading modules from /home/green/git/lustre-release/lustre oleg158-server: detected 4 online CPUs by sysfs oleg158-server: MODOPTS_LIBCFS= oleg158-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg158-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg158-server: oleg158-server.virtnet: executing set_default_debug -1 all pdsh@oleg158-client: oleg158-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg158-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg158-server: oleg158-server.virtnet: executing set_default_debug -1 all pdsh@oleg158-client: oleg158-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg158-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg158-server: oleg158-server.virtnet: executing set_default_debug -1 all pdsh@oleg158-client: oleg158-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg158-client.virtnet: -o user_xattr,flock 192.168.201.158@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 654 (time 1787854238.61 total 10.01 last 65.34) - open/close 1281 (time 1787854248.61 total 20.01 last 62.68) - open/close 2121 (time 1787854258.62 total 30.02 last 83.94) - open/close 2989 (time 1787854268.62 total 40.02 last 86.78) - open/close 3859 (time 1787854278.62 total 50.02 last 86.97) total: 4500 open/close in 58.64 seconds: 76.74 ops/second - /unlink 1280 (time 1787854302.60 total 10.02 last 127.81) - /unlink 2544 (time 1787854312.60 total 20.02 last 126.39) - /unlink 4008 (time 1787854322.61 total 30.02 last 146.34) total: 4500 /unlink in 33.28 seconds: 135.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 867 (time 1787854340.93 total 10.01 last 86.58) - open/close 1568 (time 1787854350.94 total 20.02 last 70.05) - open/close 2321 (time 1787854360.94 total 30.02 last 75.27) - open/close 3007 (time 1787854370.97 total 40.05 last 68.43) - open/close 3851 (time 1787854380.97 total 50.06 last 84.34) total: 4500 open/close in 56.84 seconds: 79.17 ops/second - /unlink 1566 (time 1787854401.63 total 10.00 last 156.57) - /unlink 3285 (time 1787854411.63 total 20.01 last 171.76) total: 4500 /unlink in 27.29 seconds: 164.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 763 (time 1787854434.34 total 10.00 last 76.28) - open/close 1684 (time 1787854444.34 total 20.01 last 92.07) - open/close 2639 (time 1787854454.35 total 30.01 last 95.45) - open/close 3512 (time 1787854464.35 total 40.02 last 87.24) - open/close 4462 (time 1787854474.36 total 50.02 last 94.96) total: 4500 open/close in 50.41 seconds: 89.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1206 (time 1787854489.22 total 10.00 last 120.59) - /unlink 2693 (time 1787854499.23 total 20.01 last 148.61) total: 4500 /unlink in 29.98 seconds: 150.10 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 893 (time 1787854523.39 total 10.01 last 89.21) - open/close 1773 (time 1787854533.41 total 20.03 last 87.83) - open/close 2501 (time 1787854543.42 total 30.04 last 72.73) - open/close 3452 (time 1787854553.42 total 40.04 last 95.07) - open/close 4358 (time 1787854563.43 total 50.05 last 90.56) total: 4500 open/close in 51.44 seconds: 87.48 ops/second - /unlink 1703 (time 1787854579.53 total 10.00 last 170.24) - /unlink 3234 (time 1787854589.54 total 20.01 last 152.99) total: 4500 /unlink in 28.16 seconds: 159.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 836 (time 1787854613.36 total 10.00 last 83.60) - open/close 1728 (time 1787854623.37 total 20.01 last 89.11) - open/close 2711 (time 1787854633.37 total 30.02 last 98.23) - open/close 3666 (time 1787854643.38 total 40.02 last 95.44) total: 4500 open/close in 48.30 seconds: 93.17 ops/second - /unlink 1565 (time 1787854667.55 total 10.00 last 156.49) - /unlink 3159 (time 1787854677.55 total 20.00 last 159.39) total: 4500 /unlink in 26.90 seconds: 167.26 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 906 (time 1787854698.79 total 10.00 last 90.59) - open/close 1764 (time 1787854708.79 total 20.01 last 85.76) - open/close 2663 (time 1787854718.80 total 30.01 last 89.85) - open/close 3597 (time 1787854728.80 total 40.02 last 93.35) total: 4500 open/close in 48.67 seconds: 92.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1521 (time 1787854751.95 total 10.00 last 152.05) - /unlink 3219 (time 1787854761.95 total 20.01 last 169.73) total: 4500 /unlink in 28.82 seconds: 156.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 949 (time 1787854785.16 total 10.00 last 94.86) - open/close 1897 (time 1787854795.17 total 20.01 last 94.71) - open/close 2913 (time 1787854805.17 total 30.02 last 101.56) - open/close 3764 (time 1787854815.17 total 40.02 last 85.09) total: 4500 open/close in 48.63 seconds: 92.54 ops/second - /unlink 1350 (time 1787854838.05 total 10.03 last 134.66) - /unlink 2405 (time 1787854848.05 total 20.03 last 105.50) - /unlink 4100 (time 1787854858.05 total 30.03 last 169.44) total: 4500 /unlink in 32.26 seconds: 139.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 865 (time 1787854874.57 total 10.00 last 86.48) - open/close 1652 (time 1787854884.58 total 20.02 last 78.60) - open/close 2136 (time 1787854894.59 total 30.02 last 48.39) - open/close 2678 (time 1787854904.59 total 40.02 last 54.18) - open/close 3301 (time 1787854914.60 total 50.03 last 62.22) - open/close 3968 (time 1787854924.61 total 60.04 last 66.64) total: 4500 open/close in 67.92 seconds: 66.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1559 (time 1787854947.02 total 10.00 last 155.90) - /unlink 3217 (time 1787854957.02 total 20.00 last 165.75) total: 4500 /unlink in 28.05 seconds: 160.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 1004 (time 1787854979.80 total 10.00 last 100.38) - open/close 1892 (time 1787854989.80 total 20.01 last 88.76) - open/close 2802 (time 1787854999.80 total 30.01 last 91.00) - open/close 3614 (time 1787855009.81 total 40.01 last 81.17) total: 4500 open/close in 49.73 seconds: 90.49 ops/second - /unlink 2054 (time 1787855033.50 total 10.00 last 205.33) - /unlink 3801 (time 1787855043.51 total 20.01 last 174.53) total: 4500 /unlink in 24.33 seconds: 184.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 952 (time 1787855062.46 total 10.00 last 95.20) - open/close 1923 (time 1787855072.47 total 20.00 last 97.06) - open/close 2838 (time 1787855082.47 total 30.01 last 91.42) - open/close 3773 (time 1787855092.48 total 40.02 last 93.47) total: 4500 open/close in 47.46 seconds: 94.82 ops/second - /unlink 1609 (time 1787855114.25 total 10.01 last 160.75) - /unlink 3334 (time 1787855124.25 total 20.01 last 172.44) total: 4500 /unlink in 27.46 seconds: 163.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 882 (time 1787855146.01 total 10.01 last 88.12) - open/close 1911 (time 1787855156.02 total 20.01 last 102.87) - open/close 2819 (time 1787855166.02 total 30.01 last 90.80) - open/close 3753 (time 1787855176.02 total 40.02 last 93.35) - open/close 4416 (time 1787855186.04 total 50.03 last 66.20) total: 4500 open/close in 51.89 seconds: 86.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1359 (time 1787855204.55 total 10.01 last 135.82) - /unlink 2634 (time 1787855214.56 total 20.01 last 127.41) - /unlink 3603 (time 1787855224.56 total 30.02 last 96.87) total: 4500 /unlink in 34.96 seconds: 128.72 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 805 (time 1787855244.06 total 10.00 last 80.46) - open/close 1565 (time 1787855254.06 total 20.01 last 75.99) - open/close 2257 (time 1787855264.06 total 30.01 last 69.19) - open/close 2736 (time 1787855274.08 total 40.02 last 47.83) - open/close 3587 (time 1787855284.09 total 50.03 last 85.03) total: 4500 open/close in 59.69 seconds: 75.39 ops/second - /unlink 1637 (time 1787855307.88 total 10.00 last 163.62) - /unlink 3425 (time 1787855317.89 total 20.02 last 178.58) total: 4500 /unlink in 25.95 seconds: 173.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 936 (time 1787855337.78 total 10.00 last 93.57) - open/close 1813 (time 1787855347.78 total 20.01 last 87.66) - open/close 2732 (time 1787855357.79 total 30.01 last 91.85) - open/close 3613 (time 1787855367.80 total 40.03 last 87.95) - open/close 4466 (time 1787855377.80 total 50.03 last 85.30) total: 4500 open/close in 50.54 seconds: 89.04 ops/second - /unlink 1581 (time 1787855391.96 total 10.00 last 158.08) - /unlink 3164 (time 1787855401.96 total 20.00 last 158.28) total: 4500 /unlink in 29.12 seconds: 154.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 946 (time 1787855425.85 total 10.01 last 94.55) - open/close 1535 (time 1787855435.86 total 20.01 last 58.87) - open/close 2088 (time 1787855445.87 total 30.02 last 55.26) - open/close 2762 (time 1787855455.87 total 40.02 last 67.36) - open/close 3629 (time 1787855465.87 total 50.03 last 86.68) total: 4500 open/close in 59.78 seconds: 75.28 ops/second - /unlink 1525 (time 1787855490.31 total 10.01 last 152.40) - /unlink 3130 (time 1787855500.31 total 20.01 last 160.47) total: 4500 /unlink in 29.94 seconds: 150.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 845 (time 1787855525.82 total 10.02 last 84.32) - open/close 1710 (time 1787855535.83 total 20.04 last 86.37) - open/close 2579 (time 1787855545.84 total 30.04 last 86.84) - open/close 3394 (time 1787855555.85 total 40.05 last 81.43) - open/close 4278 (time 1787855565.85 total 50.06 last 88.37) total: 4500 open/close in 52.26 seconds: 86.11 ops/second - /unlink 1466 (time 1787855582.28 total 10.00 last 146.59) - /unlink 2979 (time 1787855592.29 total 20.00 last 151.25) - /unlink 4256 (time 1787855602.30 total 30.01 last 127.57) total: 4500 /unlink in 32.41 seconds: 138.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 702 (time 1787855620.19 total 10.00 last 70.17) - open/close 1406 (time 1787855630.19 total 20.01 last 70.38) - open/close 2065 (time 1787855640.20 total 30.01 last 65.88) - open/close 2819 (time 1787855650.20 total 40.01 last 75.38) - open/close 3663 (time 1787855660.20 total 50.02 last 84.37) total: 4500 open/close in 58.68 seconds: 76.69 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1663 (time 1787855683.31 total 10.00 last 166.23) - /unlink 3361 (time 1787855693.31 total 20.01 last 169.76) total: 4500 /unlink in 28.44 seconds: 158.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 812 (time 1787855717.10 total 10.00 last 81.18) - open/close 1717 (time 1787855727.12 total 20.02 last 90.35) - open/close 2533 (time 1787855737.12 total 30.02 last 81.58) - open/close 3222 (time 1787855747.14 total 40.04 last 68.79) - open/close 4059 (time 1787855757.14 total 50.04 last 83.64) total: 4500 open/close in 54.91 seconds: 81.95 ops/second - /unlink 1337 (time 1787855777.34 total 10.00 last 133.68) - /unlink 3050 (time 1787855787.35 total 20.01 last 171.22) - /unlink 4380 (time 1787855797.35 total 30.01 last 132.97) total: 4500 /unlink in 31.11 seconds: 144.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 931 (time 1787855813.96 total 10.00 last 93.08) - open/close 1754 (time 1787855823.96 total 20.00 last 82.30) - open/close 2600 (time 1787855833.97 total 30.01 last 84.54) - open/close 3382 (time 1787855843.97 total 40.02 last 78.15) - open/close 4131 (time 1787855853.98 total 50.02 last 74.88) total: 4500 open/close in 57.17 seconds: 78.71 ops/second - /unlink 1441 (time 1787855876.59 total 10.00 last 144.07) - /unlink 2875 (time 1787855886.59 total 20.00 last 143.40) - /unlink 4428 (time 1787855896.59 total 30.01 last 155.24) total: 4500 /unlink in 30.94 seconds: 145.44 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11375: 95364 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg158-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg158-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg158-server stop mds service on oleg158-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg158-server stop mds service on oleg158-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg158-server unloading modules via unload_modules_local on: 'oleg158-server' oleg158-server: oleg158-server.virtnet: executing unload_modules_local oleg158-server: modules unloaded.