== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 15:30:02 (1787859002) Stopping clients: oleg132-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg132-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg132-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg132-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg132-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg132-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg132-server oleg132-server: oleg132-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1049: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg132-server' oleg132-server: oleg132-server.virtnet: executing load_modules_local oleg132-server: Loading modules from /home/green/git/lustre-release/lustre oleg132-server: detected 4 online CPUs by sysfs oleg132-server: MODOPTS_LIBCFS= oleg132-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1049: echo: write error: Device or resource busy oleg132-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg132-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg132-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg132-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg132-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg132-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg132-server: oleg132-server.virtnet: executing set_default_debug -1 all pdsh@oleg132-client: oleg132-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg132-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg132-client.virtnet: -o user_xattr,flock 192.168.201.132@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 630 (time 1787859204.53 total 10.01 last 62.96) - open/close 1648 (time 1787859214.53 total 20.01 last 101.79) - open/close 2614 (time 1787859224.53 total 30.01 last 96.55) - open/close 3581 (time 1787859234.54 total 40.01 last 96.67) total: 4500 open/close in 49.13 seconds: 91.60 ops/second - /unlink 1711 (time 1787859257.12 total 10.00 last 171.05) - /unlink 3462 (time 1787859267.12 total 20.01 last 175.05) total: 4500 /unlink in 25.80 seconds: 174.39 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 981 (time 1787859286.56 total 10.01 last 97.98) - open/close 1869 (time 1787859296.56 total 20.02 last 88.76) - open/close 2769 (time 1787859306.56 total 30.02 last 89.99) - open/close 3493 (time 1787859316.57 total 40.02 last 72.37) - open/close 4384 (time 1787859326.57 total 50.03 last 89.04) total: 4500 open/close in 51.32 seconds: 87.68 ops/second - /unlink 1759 (time 1787859343.20 total 10.00 last 175.82) - /unlink 3398 (time 1787859353.21 total 20.01 last 163.85) total: 4500 /unlink in 26.58 seconds: 169.27 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 835 (time 1787859374.04 total 10.01 last 83.43) - open/close 1837 (time 1787859384.04 total 20.01 last 100.16) - open/close 2708 (time 1787859394.05 total 30.02 last 87.01) - open/close 3542 (time 1787859404.08 total 40.05 last 83.19) - open/close 4310 (time 1787859414.08 total 50.05 last 76.76) total: 4500 open/close in 52.57 seconds: 85.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1654 (time 1787859430.77 total 10.01 last 165.30) - /unlink 3356 (time 1787859440.77 total 20.01 last 170.14) total: 4500 /unlink in 27.57 seconds: 163.23 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 909 (time 1787859463.18 total 10.00 last 90.87) - open/close 1927 (time 1787859473.20 total 20.02 last 101.67) - open/close 2966 (time 1787859483.20 total 30.02 last 103.87) - open/close 3840 (time 1787859493.21 total 40.03 last 87.33) total: 4500 open/close in 46.82 seconds: 96.11 ops/second - /unlink 2025 (time 1787859513.62 total 10.00 last 202.42) - /unlink 3680 (time 1787859523.64 total 20.03 last 165.14) total: 4500 /unlink in 24.40 seconds: 184.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 863 (time 1787859542.92 total 10.00 last 86.29) - open/close 1737 (time 1787859552.93 total 20.01 last 87.36) - open/close 2566 (time 1787859562.97 total 30.04 last 82.58) - open/close 3479 (time 1787859572.98 total 40.05 last 91.21) - open/close 4367 (time 1787859582.99 total 50.07 last 88.65) total: 4500 open/close in 52.15 seconds: 86.29 ops/second - /unlink 1799 (time 1787859599.05 total 10.00 last 179.87) - /unlink 3465 (time 1787859609.05 total 20.00 last 166.59) total: 4500 /unlink in 26.00 seconds: 173.06 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 994 (time 1787859629.06 total 10.01 last 99.34) - open/close 1848 (time 1787859639.06 total 20.01 last 85.39) - open/close 2625 (time 1787859649.07 total 30.01 last 77.68) - open/close 3454 (time 1787859659.07 total 40.01 last 82.86) total: 4500 open/close in 49.65 seconds: 90.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1597 (time 1787859683.24 total 10.01 last 159.59) - /unlink 3169 (time 1787859693.24 total 20.01 last 157.19) total: 4500 /unlink in 29.26 seconds: 153.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 805 (time 1787859717.63 total 10.00 last 80.47) - open/close 1562 (time 1787859727.64 total 20.01 last 75.63) - open/close 2365 (time 1787859737.65 total 30.02 last 80.25) - open/close 3303 (time 1787859747.65 total 40.02 last 93.75) - open/close 4241 (time 1787859757.66 total 50.04 last 93.67) total: 4500 open/close in 52.72 seconds: 85.35 ops/second - /unlink 1637 (time 1787859774.58 total 10.00 last 163.69) - /unlink 3181 (time 1787859784.58 total 20.00 last 154.39) total: 4500 /unlink in 27.37 seconds: 164.40 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 972 (time 1787859806.27 total 10.03 last 96.92) - open/close 1825 (time 1787859816.27 total 20.03 last 85.26) - open/close 2759 (time 1787859826.28 total 30.04 last 93.33) - open/close 3679 (time 1787859836.29 total 40.05 last 91.94) total: 4500 open/close in 49.10 seconds: 91.66 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1784 (time 1787859859.43 total 10.01 last 178.26) - /unlink 3486 (time 1787859869.45 total 20.02 last 170.00) total: 4500 /unlink in 26.45 seconds: 170.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 868 (time 1787859891.05 total 10.01 last 86.75) - open/close 1617 (time 1787859901.09 total 20.04 last 74.64) - open/close 2490 (time 1787859911.09 total 30.05 last 87.24) - open/close 3377 (time 1787859921.10 total 40.05 last 88.66) - open/close 4263 (time 1787859931.10 total 50.06 last 88.58) total: 4500 open/close in 52.58 seconds: 85.58 ops/second - /unlink 1610 (time 1787859948.09 total 10.00 last 160.92) - /unlink 3131 (time 1787859958.09 total 20.01 last 152.07) total: 4500 /unlink in 28.86 seconds: 155.92 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 824 (time 1787859981.89 total 10.01 last 82.35) - open/close 1736 (time 1787859991.90 total 20.02 last 91.08) - open/close 2573 (time 1787860001.92 total 30.04 last 83.55) - open/close 3354 (time 1787860011.93 total 40.04 last 78.05) - open/close 4152 (time 1787860021.93 total 50.05 last 79.74) total: 4500 open/close in 53.54 seconds: 84.05 ops/second - /unlink 1686 (time 1787860039.63 total 10.00 last 168.53) - /unlink 2940 (time 1787860049.64 total 20.01 last 125.30) - /unlink 4246 (time 1787860059.64 total 30.01 last 130.60) total: 4500 /unlink in 31.84 seconds: 141.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 860 (time 1787860075.71 total 10.01 last 85.95) - open/close 1825 (time 1787860085.71 total 20.01 last 96.44) - open/close 2829 (time 1787860095.72 total 30.02 last 100.34) - open/close 3527 (time 1787860105.73 total 40.02 last 69.75) - open/close 4216 (time 1787860115.73 total 50.03 last 68.89) total: 4500 open/close in 52.92 seconds: 85.03 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1519 (time 1787860132.88 total 10.01 last 151.75) - /unlink 2885 (time 1787860142.89 total 20.02 last 136.53) - /unlink 4420 (time 1787860152.89 total 30.02 last 153.48) total: 4500 /unlink in 31.05 seconds: 144.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 967 (time 1787860168.39 total 10.01 last 96.59) - open/close 1883 (time 1787860178.39 total 20.01 last 91.60) - open/close 2766 (time 1787860188.39 total 30.01 last 88.28) - open/close 3578 (time 1787860198.40 total 40.02 last 81.16) - open/close 4469 (time 1787860208.41 total 50.03 last 89.01) total: 4500 open/close in 50.39 seconds: 89.31 ops/second - /unlink 1891 (time 1787860222.83 total 10.02 last 188.79) - /unlink 3556 (time 1787860232.84 total 20.02 last 166.47) total: 4500 /unlink in 25.72 seconds: 174.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 760 (time 1787860254.05 total 10.01 last 75.96) - open/close 1399 (time 1787860264.05 total 20.01 last 63.89) - open/close 2135 (time 1787860274.06 total 30.01 last 73.59) - open/close 2796 (time 1787860284.11 total 40.06 last 65.77) - open/close 3352 (time 1787860294.11 total 50.06 last 55.58) - open/close 4011 (time 1787860304.12 total 60.07 last 65.85) total: 4500 open/close in 65.34 seconds: 68.87 ops/second - /unlink 1612 (time 1787860324.17 total 10.00 last 161.18) - /unlink 3173 (time 1787860334.17 total 20.00 last 156.05) total: 4500 /unlink in 28.40 seconds: 158.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 636 (time 1787860357.51 total 10.01 last 63.57) - open/close 1512 (time 1787860367.52 total 20.01 last 87.54) - open/close 2304 (time 1787860377.52 total 30.02 last 79.18) - open/close 3066 (time 1787860387.52 total 40.02 last 76.20) - open/close 3700 (time 1787860397.53 total 50.03 last 63.31) - open/close 4250 (time 1787860407.54 total 60.03 last 54.97) total: 4500 open/close in 63.86 seconds: 70.47 ops/second - /unlink 1769 (time 1787860426.30 total 10.01 last 176.69) - /unlink 3320 (time 1787860436.30 total 20.01 last 155.10) total: 4500 /unlink in 27.45 seconds: 163.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 758 (time 1787860459.15 total 10.00 last 75.79) - open/close 1525 (time 1787860469.16 total 20.00 last 76.69) - open/close 2479 (time 1787860479.17 total 30.01 last 95.30) - open/close 3216 (time 1787860489.17 total 40.02 last 73.65) - open/close 4096 (time 1787860499.18 total 50.02 last 87.96) total: 4500 open/close in 54.62 seconds: 82.39 ops/second - /unlink 1466 (time 1787860518.07 total 10.00 last 146.54) - /unlink 3165 (time 1787860528.08 total 20.01 last 169.81) total: 4500 /unlink in 28.22 seconds: 159.48 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 815 (time 1787860552.91 total 10.00 last 81.48) - open/close 1673 (time 1787860562.91 total 20.01 last 85.76) - open/close 2731 (time 1787860572.92 total 30.01 last 105.72) - open/close 3545 (time 1787860582.92 total 40.02 last 81.38) - open/close 4262 (time 1787860592.93 total 50.03 last 71.61) total: 4500 open/close in 52.30 seconds: 86.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1338 (time 1787860609.66 total 10.00 last 133.79) - /unlink 2749 (time 1787860619.66 total 20.01 last 141.03) - /unlink 4274 (time 1787860629.66 total 30.01 last 152.48) total: 4500 /unlink in 31.81 seconds: 141.48 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 795 (time 1787860646.67 total 10.00 last 79.50) - open/close 1765 (time 1787860656.68 total 20.01 last 96.93) - open/close 2648 (time 1787860666.68 total 30.01 last 88.25) - open/close 3446 (time 1787860676.69 total 40.02 last 79.75) - open/close 4350 (time 1787860686.72 total 50.05 last 90.13) total: 4500 open/close in 51.72 seconds: 87.00 ops/second - /unlink 1605 (time 1787860702.56 total 10.00 last 160.49) - /unlink 3286 (time 1787860712.56 total 20.00 last 168.09) total: 4500 /unlink in 27.66 seconds: 162.72 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 949 (time 1787860735.60 total 10.00 last 94.89) - open/close 1889 (time 1787860745.61 total 20.01 last 93.95) - open/close 2836 (time 1787860755.62 total 30.02 last 94.57) - open/close 3536 (time 1787860765.63 total 40.03 last 69.95) - open/close 4233 (time 1787860775.63 total 50.03 last 69.67) total: 4500 open/close in 53.51 seconds: 84.09 ops/second - /unlink 1649 (time 1787860793.58 total 10.00 last 164.89) - /unlink 3287 (time 1787860803.58 total 20.00 last 163.75) total: 4500 /unlink in 27.59 seconds: 163.09 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11375: 95163 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg132-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg132-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg132-server stop mds service on oleg132-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg132-server stop mds service on oleg132-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg132-server unloading modules via unload_modules_local on: 'oleg132-server' oleg132-server: oleg132-server.virtnet: executing unload_modules_local oleg132-server: modules unloaded.