== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 09:20:05 (1787145605) Stopping clients: oleg313-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg313-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg313-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg313-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg313-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg313-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg313-server oleg313-server: oleg313-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg313-server' oleg313-server: oleg313-server.virtnet: executing load_modules_local oleg313-server: Loading modules from /home/green/git/lustre-release/lustre oleg313-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg313-server: detected 4 online CPUs by sysfs oleg313-server: MODOPTS_LIBCFS= oleg313-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg313-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg313-server: oleg313-server.virtnet: executing set_default_debug -1 all pdsh@oleg313-client: oleg313-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg313-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg313-server: oleg313-server.virtnet: executing set_default_debug -1 all pdsh@oleg313-client: oleg313-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg313-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg313-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg313-server: oleg313-server.virtnet: executing set_default_debug -1 all pdsh@oleg313-client: oleg313-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg313-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg313-client.virtnet: -o user_xattr,flock 192.168.203.113@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 792 (time 1787145807.93 total 10.01 last 79.15) - open/close 1626 (time 1787145817.93 total 20.01 last 83.38) - open/close 2360 (time 1787145827.94 total 30.02 last 73.31) - open/close 3158 (time 1787145837.95 total 40.03 last 79.74) - open/close 3963 (time 1787145847.95 total 50.03 last 80.49) total: 4500 open/close in 56.47 seconds: 79.69 ops/second - /unlink 1663 (time 1787145867.99 total 10.00 last 166.23) - /unlink 3210 (time 1787145877.99 total 20.01 last 154.65) total: 4500 /unlink in 28.69 seconds: 156.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 844 (time 1787145900.69 total 10.00 last 84.38) - open/close 1700 (time 1787145910.71 total 20.02 last 85.47) - open/close 2498 (time 1787145920.71 total 30.02 last 79.77) - open/close 3298 (time 1787145930.72 total 40.03 last 79.94) - open/close 4146 (time 1787145940.72 total 50.03 last 84.75) total: 4500 open/close in 54.68 seconds: 82.30 ops/second - /unlink 1553 (time 1787145959.38 total 10.00 last 155.29) - /unlink 2881 (time 1787145969.38 total 20.01 last 132.74) - /unlink 4408 (time 1787145979.39 total 30.01 last 152.60) total: 4500 /unlink in 30.88 seconds: 145.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 794 (time 1787145994.04 total 10.01 last 79.35) - open/close 1512 (time 1787146004.06 total 20.02 last 71.67) - open/close 2252 (time 1787146014.07 total 30.03 last 73.98) - open/close 3162 (time 1787146024.07 total 40.03 last 90.96) - open/close 4000 (time 1787146034.09 total 50.06 last 83.60) total: 4500 open/close in 56.48 seconds: 79.67 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1303 (time 1787146054.11 total 10.00 last 130.26) - /unlink 2554 (time 1787146064.12 total 20.01 last 125.01) - /unlink 3951 (time 1787146074.12 total 30.02 last 139.63) total: 4500 /unlink in 33.66 seconds: 133.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 742 (time 1787146092.15 total 10.01 last 74.14) - open/close 1575 (time 1787146102.15 total 20.01 last 83.28) - open/close 2365 (time 1787146112.17 total 30.03 last 78.85) - open/close 3167 (time 1787146122.18 total 40.04 last 80.08) - open/close 3942 (time 1787146132.19 total 50.05 last 77.46) total: 4500 open/close in 57.88 seconds: 77.74 ops/second - /unlink 1351 (time 1787146155.64 total 10.00 last 135.05) - /unlink 2930 (time 1787146165.65 total 20.01 last 157.79) - /unlink 4422 (time 1787146175.65 total 30.01 last 149.18) total: 4500 /unlink in 30.76 seconds: 146.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 723 (time 1787146190.95 total 10.00 last 72.28) - open/close 1549 (time 1787146200.95 total 20.00 last 82.60) - open/close 2395 (time 1787146210.96 total 30.01 last 84.54) - open/close 3246 (time 1787146220.97 total 40.02 last 85.06) - open/close 4028 (time 1787146230.97 total 50.02 last 78.18) total: 4500 open/close in 55.97 seconds: 80.40 ops/second - /unlink 1580 (time 1787146251.10 total 10.01 last 157.87) - /unlink 3085 (time 1787146261.11 total 20.02 last 150.36) total: 4500 /unlink in 28.83 seconds: 156.10 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 775 (time 1787146284.25 total 10.00 last 77.48) - open/close 1576 (time 1787146294.26 total 20.01 last 80.04) - open/close 2455 (time 1787146304.26 total 30.01 last 87.88) - open/close 3291 (time 1787146314.27 total 40.02 last 83.53) - open/close 4138 (time 1787146324.27 total 50.02 last 84.70) total: 4500 open/close in 54.62 seconds: 82.39 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1578 (time 1787146342.93 total 10.01 last 157.68) - /unlink 3040 (time 1787146352.93 total 20.01 last 146.15) - /unlink 4433 (time 1787146362.93 total 30.01 last 139.28) total: 4500 /unlink in 30.54 seconds: 147.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 820 (time 1787146377.87 total 10.01 last 81.88) - open/close 1637 (time 1787146387.87 total 20.02 last 81.65) - open/close 2391 (time 1787146397.88 total 30.03 last 75.35) - open/close 3053 (time 1787146407.88 total 40.03 last 66.20) - open/close 3683 (time 1787146417.89 total 50.04 last 62.93) - open/close 4380 (time 1787146427.91 total 60.06 last 69.58) total: 4500 open/close in 62.29 seconds: 72.25 ops/second - /unlink 1484 (time 1787146444.13 total 10.00 last 148.35) - /unlink 2923 (time 1787146454.13 total 20.01 last 143.82) - /unlink 4412 (time 1787146464.14 total 30.01 last 148.84) total: 4500 /unlink in 30.85 seconds: 145.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 822 (time 1787146479.10 total 10.00 last 82.18) - open/close 1603 (time 1787146489.11 total 20.01 last 78.05) - open/close 2420 (time 1787146499.12 total 30.02 last 81.60) - open/close 3238 (time 1787146509.13 total 40.03 last 81.73) - open/close 4059 (time 1787146519.13 total 50.03 last 82.10) total: 4500 open/close in 56.24 seconds: 80.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1489 (time 1787146539.92 total 10.00 last 148.87) - /unlink 2990 (time 1787146549.92 total 20.00 last 150.06) - /unlink 4427 (time 1787146559.93 total 30.01 last 143.61) total: 4500 /unlink in 30.76 seconds: 146.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 842 (time 1787146575.13 total 10.00 last 84.17) - open/close 1682 (time 1787146585.13 total 20.00 last 83.99) - open/close 2473 (time 1787146595.13 total 30.00 last 79.10) - open/close 3314 (time 1787146605.14 total 40.01 last 84.02) - open/close 4083 (time 1787146615.14 total 50.02 last 76.88) total: 4500 open/close in 55.59 seconds: 80.95 ops/second - /unlink 1422 (time 1787146634.64 total 10.00 last 142.17) - /unlink 2940 (time 1787146644.64 total 20.00 last 151.80) total: 4500 /unlink in 29.94 seconds: 150.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 877 (time 1787146669.15 total 10.00 last 87.69) - open/close 1660 (time 1787146679.16 total 20.01 last 78.21) - open/close 2476 (time 1787146689.17 total 30.02 last 81.55) - open/close 3373 (time 1787146699.17 total 40.02 last 89.64) - open/close 4298 (time 1787146709.18 total 50.03 last 92.42) total: 4500 open/close in 52.33 seconds: 85.99 ops/second - /unlink 1513 (time 1787146725.46 total 10.00 last 151.29) - /unlink 2999 (time 1787146735.47 total 20.02 last 148.39) total: 4500 /unlink in 29.54 seconds: 152.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 807 (time 1787146759.89 total 10.01 last 80.65) - open/close 1596 (time 1787146769.92 total 20.03 last 78.69) - open/close 2268 (time 1787146779.92 total 30.04 last 67.18) - open/close 3064 (time 1787146789.93 total 40.04 last 79.58) - open/close 3829 (time 1787146799.94 total 50.05 last 76.42) total: 4500 open/close in 58.23 seconds: 77.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1478 (time 1787146822.28 total 10.00 last 147.74) - /unlink 2876 (time 1787146832.28 total 20.01 last 139.73) - /unlink 4341 (time 1787146842.28 total 30.01 last 146.48) total: 4500 /unlink in 31.11 seconds: 144.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 841 (time 1787146857.51 total 10.00 last 84.08) - open/close 1687 (time 1787146867.53 total 20.02 last 84.47) - open/close 2524 (time 1787146877.53 total 30.02 last 83.69) - open/close 3299 (time 1787146887.54 total 40.03 last 77.41) - open/close 4087 (time 1787146897.55 total 50.04 last 78.72) total: 4500 open/close in 54.99 seconds: 81.84 ops/second - /unlink 1483 (time 1787146916.20 total 10.00 last 148.26) - /unlink 2977 (time 1787146926.20 total 20.01 last 149.34) - /unlink 4490 (time 1787146936.21 total 30.01 last 151.22) total: 4500 /unlink in 30.14 seconds: 149.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 814 (time 1787146950.47 total 10.01 last 81.32) - open/close 1568 (time 1787146960.47 total 20.02 last 75.34) - open/close 2340 (time 1787146970.48 total 30.02 last 77.17) - open/close 3196 (time 1787146980.49 total 40.03 last 85.48) - open/close 4032 (time 1787146990.50 total 50.04 last 83.52) total: 4500 open/close in 56.41 seconds: 79.77 ops/second - /unlink 1471 (time 1787147011.07 total 10.00 last 147.07) - /unlink 2893 (time 1787147021.08 total 20.00 last 142.17) - /unlink 4233 (time 1787147031.08 total 30.01 last 133.97) total: 4500 /unlink in 31.88 seconds: 141.16 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 835 (time 1787147046.11 total 10.00 last 83.47) - open/close 1609 (time 1787147056.11 total 20.01 last 77.39) - open/close 2447 (time 1787147066.11 total 30.01 last 83.77) - open/close 3222 (time 1787147076.12 total 40.01 last 77.46) - open/close 4044 (time 1787147086.12 total 50.01 last 82.19) total: 4500 open/close in 57.87 seconds: 77.76 ops/second - /unlink 1469 (time 1787147108.19 total 10.01 last 146.81) - /unlink 3013 (time 1787147118.20 total 20.01 last 154.31) total: 4500 /unlink in 29.40 seconds: 153.07 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 853 (time 1787147141.47 total 10.00 last 85.29) - open/close 1686 (time 1787147151.47 total 20.00 last 83.28) - open/close 2427 (time 1787147161.48 total 30.02 last 74.01) - open/close 3259 (time 1787147171.49 total 40.03 last 83.11) - open/close 4098 (time 1787147181.50 total 50.03 last 83.85) total: 4500 open/close in 54.95 seconds: 81.90 ops/second - /unlink 1557 (time 1787147201.61 total 10.00 last 155.62) - /unlink 2956 (time 1787147211.61 total 20.01 last 139.88) - /unlink 4461 (time 1787147221.61 total 30.01 last 150.46) total: 4500 /unlink in 30.33 seconds: 148.37 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 837 (time 1787147235.77 total 10.00 last 83.69) - open/close 1715 (time 1787147245.78 total 20.01 last 87.74) - open/close 2452 (time 1787147255.78 total 30.01 last 73.69) - open/close 3285 (time 1787147265.78 total 40.01 last 83.26) - open/close 4131 (time 1787147275.79 total 50.02 last 84.54) total: 4500 open/close in 54.45 seconds: 82.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1578 (time 1787147293.96 total 10.00 last 157.73) - /unlink 2699 (time 1787147303.97 total 20.01 last 112.09) - /unlink 3788 (time 1787147313.97 total 30.01 last 108.86) total: 4500 /unlink in 35.70 seconds: 126.03 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 614 (time 1787147334.21 total 10.01 last 61.35) - open/close 1413 (time 1787147344.21 total 20.02 last 79.83) - open/close 2298 (time 1787147354.23 total 30.03 last 88.36) - open/close 3054 (time 1787147364.24 total 40.04 last 75.55) - open/close 3889 (time 1787147374.24 total 50.04 last 83.50) total: 4500 open/close in 57.39 seconds: 78.41 ops/second - /unlink 1380 (time 1787147395.33 total 10.00 last 137.96) - /unlink 2878 (time 1787147405.33 total 20.01 last 149.76) - /unlink 4403 (time 1787147415.34 total 30.01 last 152.46) total: 4500 /unlink in 30.80 seconds: 146.10 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 864 (time 1787147430.07 total 10.00 last 86.40) - open/close 1669 (time 1787147440.08 total 20.01 last 80.42) - open/close 2298 (time 1787147450.09 total 30.02 last 62.84) - open/close 3154 (time 1787147460.09 total 40.03 last 85.55) - open/close 3979 (time 1787147470.10 total 50.03 last 82.44) total: 4500 open/close in 55.99 seconds: 80.37 ops/second - /unlink 1587 (time 1787147489.80 total 10.00 last 158.68) - /unlink 3190 (time 1787147499.80 total 20.01 last 160.24) total: 4500 /unlink in 27.98 seconds: 160.84 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11316: 95232 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg313-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg313-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg313-server stop mds service on oleg313-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg313-server stop mds service on oleg313-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg313-server unloading modules via unload_modules_local on: 'oleg313-server' oleg313-server: oleg313-server.virtnet: executing unload_modules_local oleg313-server: modules unloaded.