== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 16:20:47 (1786652447) Stopping clients: oleg139-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg139-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg139-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg139-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg139-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg139-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg139-server oleg139-server: oleg139-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg139-server' oleg139-server: oleg139-server.virtnet: executing load_modules_local oleg139-server: Loading modules from /home/green/git/lustre-release/lustre oleg139-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg139-server: detected 4 online CPUs by sysfs oleg139-server: MODOPTS_LIBCFS= oleg139-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg139-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg139-server: oleg139-server.virtnet: executing set_default_debug -1 all pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg139-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg139-server: oleg139-server.virtnet: executing set_default_debug -1 all pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg139-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg139-server: oleg139-server.virtnet: executing set_default_debug -1 all pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg139-client.virtnet: -o user_xattr,flock 192.168.201.139@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 661 (time 1786652653.75 total 10.01 last 66.06) - open/close 1460 (time 1786652663.75 total 20.01 last 79.86) - open/close 2318 (time 1786652673.76 total 30.02 last 85.75) - open/close 3007 (time 1786652683.77 total 40.02 last 68.86) - open/close 3797 (time 1786652693.77 total 50.03 last 78.95) total: 4500 open/close in 58.61 seconds: 76.77 ops/second - /unlink 1459 (time 1786652716.78 total 10.00 last 145.87) - /unlink 3119 (time 1786652726.78 total 20.00 last 165.97) total: 4500 /unlink in 28.97 seconds: 155.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 846 (time 1786652750.56 total 10.00 last 84.60) - open/close 1788 (time 1786652760.57 total 20.01 last 94.15) - open/close 2774 (time 1786652770.57 total 30.01 last 98.54) - open/close 3704 (time 1786652780.60 total 40.04 last 92.72) total: 4500 open/close in 47.82 seconds: 94.11 ops/second - /unlink 1410 (time 1786652803.21 total 10.00 last 141.00) - /unlink 2795 (time 1786652813.22 total 20.01 last 138.38) - /unlink 4264 (time 1786652823.22 total 30.01 last 146.87) total: 4500 /unlink in 31.48 seconds: 142.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 935 (time 1786652838.85 total 10.00 last 93.47) - open/close 1701 (time 1786652848.85 total 20.00 last 76.60) - open/close 2542 (time 1786652858.85 total 30.01 last 84.07) - open/close 3396 (time 1786652868.86 total 40.02 last 85.28) - open/close 4243 (time 1786652878.87 total 50.03 last 84.62) total: 4500 open/close in 53.70 seconds: 83.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1521 (time 1786652896.78 total 10.00 last 152.09) - /unlink 3072 (time 1786652906.78 total 20.00 last 155.09) - /unlink 4459 (time 1786652916.79 total 30.01 last 138.54) total: 4500 /unlink in 30.55 seconds: 147.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 825 (time 1786652931.60 total 10.01 last 82.45) - open/close 1736 (time 1786652941.61 total 20.02 last 90.99) - open/close 2519 (time 1786652951.63 total 30.04 last 78.13) - open/close 3356 (time 1786652961.64 total 40.05 last 83.65) - open/close 4263 (time 1786652971.64 total 50.05 last 90.64) total: 4500 open/close in 53.88 seconds: 83.52 ops/second - /unlink 1684 (time 1786652990.15 total 10.00 last 168.33) - /unlink 3402 (time 1786653000.15 total 20.01 last 171.77) total: 4500 /unlink in 28.50 seconds: 157.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 809 (time 1786653023.13 total 10.01 last 80.79) - open/close 1626 (time 1786653033.14 total 20.03 last 81.59) - open/close 2305 (time 1786653043.14 total 30.03 last 67.89) - open/close 3074 (time 1786653053.15 total 40.03 last 76.88) - open/close 3879 (time 1786653063.15 total 50.04 last 80.44) total: 4500 open/close in 57.99 seconds: 77.60 ops/second - /unlink 1773 (time 1786653085.34 total 10.00 last 177.26) - /unlink 3531 (time 1786653095.34 total 20.00 last 175.79) total: 4500 /unlink in 27.45 seconds: 163.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 883 (time 1786653117.74 total 10.01 last 88.21) - open/close 1657 (time 1786653127.75 total 20.02 last 77.35) - open/close 2542 (time 1786653137.76 total 30.03 last 88.39) - open/close 3435 (time 1786653147.76 total 40.03 last 89.30) - open/close 4237 (time 1786653157.77 total 50.04 last 80.13) total: 4500 open/close in 52.87 seconds: 85.12 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1376 (time 1786653175.14 total 10.00 last 137.56) - /unlink 2987 (time 1786653185.15 total 20.01 last 160.92) - /unlink 4470 (time 1786653195.15 total 30.02 last 148.26) total: 4500 /unlink in 30.23 seconds: 148.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 908 (time 1786653209.77 total 10.01 last 90.69) - open/close 1761 (time 1786653219.78 total 20.02 last 85.22) - open/close 2611 (time 1786653229.78 total 30.02 last 84.98) - open/close 3471 (time 1786653239.80 total 40.05 last 85.80) - open/close 4267 (time 1786653249.80 total 50.05 last 79.59) total: 4500 open/close in 52.30 seconds: 86.04 ops/second - /unlink 1654 (time 1786653265.61 total 10.00 last 165.32) - /unlink 3176 (time 1786653275.61 total 20.01 last 152.12) total: 4500 /unlink in 28.01 seconds: 160.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 926 (time 1786653297.81 total 10.00 last 92.57) - open/close 1951 (time 1786653307.81 total 20.01 last 102.43) - open/close 2892 (time 1786653317.82 total 30.01 last 94.05) - open/close 3697 (time 1786653327.82 total 40.02 last 80.46) total: 4500 open/close in 49.45 seconds: 90.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1447 (time 1786653351.59 total 10.00 last 144.66) - /unlink 3088 (time 1786653361.60 total 20.01 last 164.04) total: 4500 /unlink in 28.70 seconds: 156.78 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 904 (time 1786653385.12 total 10.01 last 90.35) - open/close 1684 (time 1786653395.14 total 20.02 last 77.87) - open/close 2455 (time 1786653405.15 total 30.03 last 77.00) - open/close 3309 (time 1786653415.16 total 40.05 last 85.31) - open/close 4151 (time 1786653425.17 total 50.06 last 84.10) total: 4500 open/close in 53.71 seconds: 83.79 ops/second - /unlink 1612 (time 1786653442.57 total 10.01 last 161.11) - /unlink 2994 (time 1786653452.57 total 20.01 last 138.17) total: 4500 /unlink in 29.32 seconds: 153.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 857 (time 1786653477.52 total 10.00 last 85.67) - open/close 1693 (time 1786653487.53 total 20.01 last 83.53) - open/close 2527 (time 1786653497.53 total 30.02 last 83.36) - open/close 3388 (time 1786653507.53 total 40.02 last 86.09) - open/close 4256 (time 1786653517.54 total 50.02 last 86.75) total: 4500 open/close in 53.51 seconds: 84.10 ops/second - /unlink 1661 (time 1786653535.77 total 10.00 last 166.05) - /unlink 2909 (time 1786653545.78 total 20.01 last 124.68) - /unlink 4161 (time 1786653555.78 total 30.02 last 125.17) total: 4500 /unlink in 32.50 seconds: 138.44 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 790 (time 1786653572.52 total 10.01 last 78.90) - open/close 1596 (time 1786653582.53 total 20.03 last 80.49) - open/close 2445 (time 1786653592.54 total 30.03 last 84.86) - open/close 3230 (time 1786653602.55 total 40.04 last 78.39) - open/close 3880 (time 1786653612.55 total 50.05 last 64.99) total: 4500 open/close in 57.06 seconds: 78.86 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1621 (time 1786653634.13 total 10.00 last 162.09) - /unlink 3235 (time 1786653644.14 total 20.00 last 161.37) total: 4500 /unlink in 27.90 seconds: 161.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 809 (time 1786653667.86 total 10.01 last 80.86) - open/close 1572 (time 1786653677.88 total 20.03 last 76.09) - open/close 2295 (time 1786653687.89 total 30.04 last 72.24) - open/close 3136 (time 1786653697.90 total 40.05 last 84.01) - open/close 4014 (time 1786653707.91 total 50.06 last 87.75) total: 4500 open/close in 56.10 seconds: 80.22 ops/second - /unlink 1716 (time 1786653728.15 total 10.00 last 171.58) - /unlink 3346 (time 1786653738.16 total 20.01 last 162.85) total: 4500 /unlink in 26.90 seconds: 167.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 798 (time 1786653759.46 total 10.00 last 79.78) - open/close 1748 (time 1786653769.47 total 20.01 last 94.95) - open/close 2696 (time 1786653779.48 total 30.02 last 94.67) - open/close 3569 (time 1786653789.48 total 40.02 last 87.28) - open/close 4415 (time 1786653799.49 total 50.03 last 84.55) total: 4500 open/close in 51.26 seconds: 87.79 ops/second - /unlink 1399 (time 1786653815.29 total 10.00 last 139.89) - /unlink 3249 (time 1786653825.29 total 20.00 last 185.00) - /unlink 4437 (time 1786653835.29 total 30.00 last 118.80) total: 4500 /unlink in 30.53 seconds: 147.38 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 535 (time 1786653850.79 total 10.04 last 53.29) - open/close 1096 (time 1786653860.81 total 20.06 last 56.01) - open/close 1843 (time 1786653870.84 total 30.09 last 74.47) - open/close 2696 (time 1786653880.84 total 40.09 last 85.25) - open/close 3624 (time 1786653890.85 total 50.10 last 92.72) total: 4500 open/close in 59.33 seconds: 75.84 ops/second - /unlink 1594 (time 1786653913.93 total 10.00 last 159.37) - /unlink 3130 (time 1786653923.93 total 20.01 last 153.53) total: 4500 /unlink in 28.01 seconds: 160.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 675 (time 1786653946.53 total 10.01 last 67.40) - open/close 1301 (time 1786653956.53 total 20.01 last 62.60) - open/close 2088 (time 1786653966.54 total 30.02 last 78.68) - open/close 2931 (time 1786653976.54 total 40.02 last 84.26) - open/close 3737 (time 1786653986.55 total 50.03 last 80.53) - open/close 4411 (time 1786653996.58 total 60.06 last 67.19) total: 4500 open/close in 60.81 seconds: 74.00 ops/second - /unlink 1738 (time 1786654011.55 total 10.00 last 173.78) - /unlink 3334 (time 1786654021.56 total 20.01 last 159.51) total: 4500 /unlink in 27.62 seconds: 162.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 850 (time 1786654043.26 total 10.00 last 84.96) - open/close 1789 (time 1786654053.26 total 20.01 last 93.89) - open/close 2696 (time 1786654063.27 total 30.01 last 90.64) - open/close 3563 (time 1786654073.27 total 40.02 last 86.65) - open/close 4357 (time 1786654083.28 total 50.02 last 79.37) total: 4500 open/close in 52.04 seconds: 86.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1530 (time 1786654099.34 total 10.00 last 152.98) - /unlink 3136 (time 1786654109.34 total 20.00 last 160.60) - /unlink 4460 (time 1786654119.34 total 30.00 last 132.36) total: 4500 /unlink in 30.29 seconds: 148.58 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 945 (time 1786654133.64 total 10.01 last 94.43) - open/close 1515 (time 1786654143.66 total 20.02 last 56.92) - open/close 2241 (time 1786654153.67 total 30.03 last 72.52) - open/close 2845 (time 1786654163.67 total 40.04 last 60.35) - open/close 3539 (time 1786654173.69 total 50.06 last 69.27) - open/close 4289 (time 1786654183.70 total 60.07 last 74.95) total: 4500 open/close in 62.45 seconds: 72.06 ops/second - /unlink 1546 (time 1786654200.60 total 10.00 last 154.55) - /unlink 3071 (time 1786654210.61 total 20.00 last 152.48) - /unlink 4388 (time 1786654220.61 total 30.01 last 131.68) total: 4500 /unlink in 30.93 seconds: 145.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 892 (time 1786654236.17 total 10.01 last 89.13) - open/close 1549 (time 1786654246.18 total 20.01 last 65.66) - open/close 2341 (time 1786654256.19 total 30.03 last 79.08) - open/close 3182 (time 1786654266.20 total 40.04 last 84.03) - open/close 4131 (time 1786654276.21 total 50.04 last 94.87) total: 4500 open/close in 53.72 seconds: 83.77 ops/second - /unlink 1712 (time 1786654293.87 total 10.00 last 171.14) - /unlink 3384 (time 1786654303.87 total 20.01 last 167.14) total: 4500 /unlink in 28.35 seconds: 158.75 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11370: 95377 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg139-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg139-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg139-server stop mds service on oleg139-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg139-server stop mds service on oleg139-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg139-server unloading modules via unload_modules_local on: 'oleg139-server' oleg139-server: oleg139-server.virtnet: executing unload_modules_local oleg139-server: modules unloaded.