== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 00:47:33 (1786942053) Stopping clients: oleg110-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg110-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg110-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg110-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg110-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg110-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg110-server oleg110-server: oleg110-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg110-server' oleg110-server: oleg110-server.virtnet: executing load_modules_local oleg110-server: Loading modules from /home/green/git/lustre-release/lustre oleg110-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg110-server: detected 4 online CPUs by sysfs oleg110-server: MODOPTS_LIBCFS= oleg110-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg110-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg110-server: oleg110-server.virtnet: executing set_default_debug -1 all pdsh@oleg110-client: oleg110-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg110-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg110-server: oleg110-server.virtnet: executing set_default_debug -1 all pdsh@oleg110-client: oleg110-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg110-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg110-server: oleg110-server.virtnet: executing set_default_debug -1 all pdsh@oleg110-client: oleg110-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 735 (time 1786942272.78 total 10.01 last 73.46) - open/close 1564 (time 1786942282.78 total 20.01 last 82.83) - open/close 2404 (time 1786942292.78 total 30.01 last 84.00) - open/close 3239 (time 1786942302.79 total 40.02 last 83.47) - open/close 4024 (time 1786942312.81 total 50.04 last 78.31) total: 4500 open/close in 56.44 seconds: 79.73 ops/second - /unlink 1376 (time 1786942333.50 total 10.00 last 137.53) - /unlink 2816 (time 1786942343.50 total 20.01 last 143.95) - /unlink 4351 (time 1786942353.50 total 30.01 last 153.48) total: 4500 /unlink in 31.25 seconds: 143.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 741 (time 1786942369.60 total 10.02 last 73.96) - open/close 1550 (time 1786942379.61 total 20.03 last 80.84) - open/close 2436 (time 1786942389.61 total 30.03 last 88.56) - open/close 3175 (time 1786942399.61 total 40.03 last 73.87) - open/close 4022 (time 1786942409.63 total 50.05 last 84.60) total: 4500 open/close in 55.78 seconds: 80.67 ops/second - /unlink 1486 (time 1786942429.08 total 10.00 last 148.58) - /unlink 3113 (time 1786942439.08 total 20.00 last 162.67) total: 4500 /unlink in 29.79 seconds: 151.05 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 935 (time 1786942463.35 total 10.01 last 93.45) - open/close 1796 (time 1786942473.35 total 20.01 last 86.05) - open/close 2607 (time 1786942483.36 total 30.02 last 81.00) - open/close 3494 (time 1786942493.37 total 40.03 last 88.64) - open/close 4456 (time 1786942503.38 total 50.04 last 96.14) total: 4500 open/close in 50.48 seconds: 89.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1453 (time 1786942518.19 total 10.00 last 145.30) - /unlink 2867 (time 1786942528.19 total 20.00 last 141.34) - /unlink 4356 (time 1786942538.20 total 30.01 last 148.85) total: 4500 /unlink in 31.19 seconds: 144.27 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 707 (time 1786942554.01 total 10.00 last 70.69) - open/close 1460 (time 1786942564.02 total 20.02 last 75.20) - open/close 2251 (time 1786942574.03 total 30.02 last 79.05) - open/close 3149 (time 1786942584.03 total 40.02 last 89.79) - open/close 3982 (time 1786942594.04 total 50.03 last 83.24) total: 4500 open/close in 56.54 seconds: 79.59 ops/second - /unlink 1591 (time 1786942614.92 total 10.01 last 158.99) - /unlink 3128 (time 1786942624.93 total 20.02 last 153.57) total: 4500 /unlink in 29.78 seconds: 151.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 860 (time 1786942648.49 total 10.01 last 85.88) - open/close 1612 (time 1786942658.51 total 20.03 last 75.09) - open/close 2344 (time 1786942668.52 total 30.04 last 73.13) - open/close 3153 (time 1786942678.52 total 40.05 last 80.83) - open/close 3952 (time 1786942688.53 total 50.05 last 79.88) total: 4500 open/close in 56.77 seconds: 79.26 ops/second - /unlink 1770 (time 1786942708.36 total 10.01 last 176.91) - /unlink 3533 (time 1786942718.36 total 20.01 last 176.24) total: 4500 /unlink in 26.04 seconds: 172.82 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 896 (time 1786942737.95 total 10.01 last 89.52) - open/close 1875 (time 1786942747.95 total 20.02 last 97.84) - open/close 2797 (time 1786942757.96 total 30.03 last 92.11) - open/close 3722 (time 1786942767.96 total 40.03 last 92.50) total: 4500 open/close in 47.60 seconds: 94.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1703 (time 1786942788.24 total 10.01 last 170.19) - /unlink 3312 (time 1786942798.25 total 20.02 last 160.67) total: 4500 /unlink in 27.53 seconds: 163.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 900 (time 1786942818.77 total 10.00 last 89.96) - open/close 1841 (time 1786942828.78 total 20.01 last 94.02) - open/close 2719 (time 1786942838.78 total 30.02 last 87.76) - open/close 3800 (time 1786942848.78 total 40.02 last 108.05) total: 4500 open/close in 46.91 seconds: 95.92 ops/second - /unlink 1402 (time 1786942869.28 total 10.00 last 140.17) - /unlink 3007 (time 1786942879.29 total 20.01 last 160.45) - /unlink 4346 (time 1786942889.29 total 30.01 last 133.82) total: 4500 /unlink in 31.66 seconds: 142.12 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 560 (time 1786942908.51 total 10.01 last 55.96) - open/close 1028 (time 1786942918.53 total 20.02 last 46.73) - open/close 1617 (time 1786942928.55 total 30.04 last 58.78) - open/close 2193 (time 1786942938.56 total 40.05 last 57.54) - open/close 2730 (time 1786942948.58 total 50.08 last 53.58) - open/close 3436 (time 1786942958.59 total 60.09 last 70.49) - open/close 4180 (time 1786942968.59 total 70.09 last 74.39) total: 4500 open/close in 74.22 seconds: 60.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1364 (time 1786942987.36 total 10.01 last 136.33) - /unlink 2862 (time 1786942997.36 total 20.01 last 149.78) - /unlink 4280 (time 1786943007.36 total 30.01 last 141.75) total: 4500 /unlink in 31.71 seconds: 141.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 786 (time 1786943024.11 total 10.02 last 78.42) - open/close 1473 (time 1786943034.13 total 20.04 last 68.58) - open/close 2244 (time 1786943044.15 total 30.06 last 76.94) - open/close 3036 (time 1786943054.16 total 40.07 last 79.16) - open/close 3874 (time 1786943064.16 total 50.07 last 83.76) total: 4500 open/close in 58.31 seconds: 77.18 ops/second - /unlink 1358 (time 1786943087.12 total 10.01 last 135.72) - /unlink 2764 (time 1786943097.13 total 20.01 last 140.50) - /unlink 4189 (time 1786943107.13 total 30.02 last 142.39) total: 4500 /unlink in 32.39 seconds: 138.95 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 774 (time 1786943123.93 total 10.02 last 77.23) - open/close 1559 (time 1786943133.94 total 20.03 last 78.43) - open/close 2406 (time 1786943143.94 total 30.03 last 84.69) - open/close 3209 (time 1786943153.94 total 40.04 last 80.25) - open/close 4105 (time 1786943163.95 total 50.05 last 89.52) total: 4500 open/close in 54.64 seconds: 82.36 ops/second - /unlink 1426 (time 1786943182.94 total 10.00 last 142.58) - /unlink 2698 (time 1786943192.94 total 20.00 last 127.19) - /unlink 4058 (time 1786943202.94 total 30.00 last 135.97) total: 4500 /unlink in 33.05 seconds: 136.17 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 828 (time 1786943221.08 total 10.01 last 82.74) - open/close 1652 (time 1786943231.08 total 20.01 last 82.34) - open/close 2604 (time 1786943241.09 total 30.02 last 95.12) - open/close 3369 (time 1786943251.10 total 40.03 last 76.42) - open/close 4110 (time 1786943261.11 total 50.04 last 74.03) total: 4500 open/close in 54.90 seconds: 81.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1567 (time 1786943280.05 total 10.00 last 156.67) - /unlink 3002 (time 1786943290.05 total 20.00 last 143.46) total: 4500 /unlink in 29.07 seconds: 154.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 713 (time 1786943313.74 total 10.00 last 71.27) - open/close 1444 (time 1786943323.74 total 20.01 last 73.06) - open/close 2274 (time 1786943333.74 total 30.01 last 82.98) - open/close 3076 (time 1786943343.75 total 40.01 last 80.18) - open/close 3888 (time 1786943353.75 total 50.01 last 81.20) total: 4500 open/close in 57.94 seconds: 77.66 ops/second - /unlink 1396 (time 1786943376.09 total 10.00 last 139.55) - /unlink 3060 (time 1786943386.09 total 20.00 last 166.38) total: 4500 /unlink in 29.58 seconds: 152.14 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 905 (time 1786943410.10 total 10.00 last 90.46) - open/close 1821 (time 1786943420.11 total 20.01 last 91.55) - open/close 2618 (time 1786943430.12 total 30.01 last 79.67) - open/close 3448 (time 1786943440.12 total 40.02 last 82.94) - open/close 4280 (time 1786943450.14 total 50.03 last 83.10) total: 4500 open/close in 53.18 seconds: 84.62 ops/second - /unlink 1631 (time 1786943467.16 total 10.00 last 163.07) - /unlink 3115 (time 1786943477.17 total 20.01 last 148.28) total: 4500 /unlink in 29.14 seconds: 154.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 782 (time 1786943500.96 total 10.01 last 78.12) - open/close 1651 (time 1786943510.96 total 20.01 last 86.89) - open/close 2474 (time 1786943520.97 total 30.02 last 82.26) - open/close 3402 (time 1786943530.98 total 40.02 last 92.74) - open/close 4181 (time 1786943540.98 total 50.02 last 77.88) total: 4500 open/close in 54.76 seconds: 82.17 ops/second - /unlink 1361 (time 1786943560.50 total 10.01 last 136.02) - /unlink 2739 (time 1786943570.50 total 20.01 last 137.75) - /unlink 4058 (time 1786943580.51 total 30.02 last 131.81) total: 4500 /unlink in 33.44 seconds: 134.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 741 (time 1786943598.61 total 10.00 last 74.09) - open/close 1505 (time 1786943608.61 total 20.00 last 76.38) - open/close 2190 (time 1786943618.63 total 30.02 last 68.37) - open/close 2971 (time 1786943628.63 total 40.03 last 78.07) - open/close 3609 (time 1786943638.66 total 50.05 last 63.66) - open/close 4400 (time 1786943648.66 total 60.05 last 79.07) total: 4500 open/close in 61.21 seconds: 73.52 ops/second - /unlink 1484 (time 1786943664.37 total 10.00 last 148.38) - /unlink 3167 (time 1786943674.37 total 20.00 last 168.24) total: 4500 /unlink in 29.22 seconds: 153.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 880 (time 1786943697.39 total 10.01 last 87.90) - open/close 1784 (time 1786943707.39 total 20.01 last 90.39) - open/close 2771 (time 1786943717.39 total 30.01 last 98.70) - open/close 3676 (time 1786943727.40 total 40.03 last 90.38) - open/close 4433 (time 1786943737.41 total 50.03 last 75.65) total: 4500 open/close in 50.73 seconds: 88.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1729 (time 1786943751.27 total 10.00 last 172.83) - /unlink 3380 (time 1786943761.27 total 20.01 last 165.08) total: 4500 /unlink in 26.71 seconds: 168.48 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 887 (time 1786943781.44 total 10.00 last 88.67) - open/close 1815 (time 1786943791.45 total 20.01 last 92.75) - open/close 2677 (time 1786943801.45 total 30.01 last 86.19) - open/close 3495 (time 1786943811.45 total 40.01 last 81.77) - open/close 4336 (time 1786943821.46 total 50.02 last 84.06) total: 4500 open/close in 52.06 seconds: 86.44 ops/second - /unlink 1677 (time 1786943836.87 total 10.01 last 167.50) - /unlink 3361 (time 1786943846.88 total 20.01 last 168.39) total: 4500 /unlink in 28.54 seconds: 157.68 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 814 (time 1786943869.50 total 10.01 last 81.33) - open/close 1666 (time 1786943879.52 total 20.03 last 85.04) - open/close 2534 (time 1786943889.53 total 30.04 last 86.68) - open/close 3532 (time 1786943899.53 total 40.04 last 99.77) - open/close 4439 (time 1786943909.53 total 50.05 last 90.68) total: 4500 open/close in 50.82 seconds: 88.55 ops/second - /unlink 1666 (time 1786943923.61 total 10.00 last 166.54) - /unlink 3292 (time 1786943933.62 total 20.01 last 162.50) total: 4500 /unlink in 27.22 seconds: 165.31 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11316: 94819 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg110-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg110-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg110-server stop mds service on oleg110-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg110-server stop mds service on oleg110-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg110-server unloading modules via unload_modules_local on: 'oleg110-server' oleg110-server: oleg110-server.virtnet: executing unload_modules_local oleg110-server: modules unloaded.