== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 20:56:45 (1789606605) Stopping clients: oleg213-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg213-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg213-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg213-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg213-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg213-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg213-server oleg213-server: oleg213-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg213-server' oleg213-server: oleg213-server.virtnet: executing load_modules_local oleg213-server: kmemleak disabled oleg213-server: Loading modules from /home/green/git/lustre-release/lustre oleg213-server: detected 4 online CPUs by sysfs oleg213-server: MODOPTS_LIBCFS= oleg213-server: Force libcfs to create 2 CPU partitions oleg213-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg213-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg213-server: oleg213-server.virtnet: executing set_default_debug -1 all pdsh@oleg213-client: oleg213-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg213-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg213-server: oleg213-server.virtnet: executing set_default_debug -1 all pdsh@oleg213-client: oleg213-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg213-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg213-server: oleg213-server.virtnet: executing set_default_debug -1 all pdsh@oleg213-client: oleg213-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg213-client.virtnet: -o user_xattr,flock 192.168.202.113@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 764 (time 1789606817.00 total 10.00 last 76.39) - open/close 1529 (time 1789606827.00 total 20.00 last 76.49) - open/close 2264 (time 1789606837.00 total 30.00 last 73.49) - open/close 3018 (time 1789606847.01 total 40.01 last 75.33) - open/close 3722 (time 1789606857.01 total 50.01 last 70.39) - open/close 4369 (time 1789606867.02 total 60.02 last 64.65) total: 4500 open/close in 61.96 seconds: 72.63 ops/second - /unlink 1472 (time 1789606884.62 total 10.00 last 147.15) - /unlink 2776 (time 1789606894.62 total 20.00 last 130.40) - /unlink 4101 (time 1789606904.62 total 30.01 last 132.45) total: 4500 /unlink in 32.94 seconds: 136.59 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 759 (time 1789606922.88 total 10.00 last 75.87) - open/close 1409 (time 1789606932.88 total 20.01 last 64.97) - open/close 2162 (time 1789606942.89 total 30.01 last 75.26) - open/close 2892 (time 1789606952.89 total 40.02 last 72.96) - open/close 3638 (time 1789606962.90 total 50.02 last 74.57) - open/close 4470 (time 1789606972.90 total 60.02 last 83.18) total: 4500 open/close in 60.38 seconds: 74.53 ops/second - /unlink 1361 (time 1789606987.58 total 10.00 last 136.09) - /unlink 2553 (time 1789606997.59 total 20.01 last 119.13) - /unlink 3915 (time 1789607007.59 total 30.01 last 136.20) total: 4500 /unlink in 35.64 seconds: 126.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 527 (time 1789607029.25 total 10.00 last 52.68) - open/close 1174 (time 1789607039.25 total 20.00 last 64.70) - open/close 1802 (time 1789607049.27 total 30.02 last 62.68) - open/close 2495 (time 1789607059.27 total 40.03 last 69.28) - open/close 3214 (time 1789607069.28 total 50.03 last 71.85) - open/close 3932 (time 1789607079.28 total 60.04 last 71.78) total: 4500 open/close in 66.95 seconds: 67.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1354 (time 1789607100.68 total 10.01 last 135.28) - /unlink 2542 (time 1789607110.68 total 20.01 last 118.77) - /unlink 3788 (time 1789607120.68 total 30.01 last 124.59) total: 4500 /unlink in 35.87 seconds: 125.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 569 (time 1789607141.95 total 10.01 last 56.84) - open/close 1046 (time 1789607151.95 total 20.02 last 47.67) - open/close 1580 (time 1789607161.96 total 30.02 last 53.36) - open/close 2130 (time 1789607171.97 total 40.03 last 54.96) - open/close 2665 (time 1789607181.97 total 50.03 last 53.48) - open/close 3374 (time 1789607191.98 total 60.05 last 70.81) - open/close 4074 (time 1789607201.98 total 70.05 last 69.99) total: 4500 open/close in 77.13 seconds: 58.34 ops/second - /unlink 1484 (time 1789607223.97 total 10.01 last 148.19) - /unlink 2791 (time 1789607233.97 total 20.02 last 130.67) - /unlink 4234 (time 1789607243.97 total 30.02 last 144.29) total: 4500 /unlink in 32.20 seconds: 139.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 719 (time 1789607261.32 total 10.00 last 71.88) - open/close 1442 (time 1789607271.33 total 20.01 last 72.23) - open/close 2026 (time 1789607281.37 total 30.06 last 58.13) - open/close 2719 (time 1789607291.38 total 40.07 last 69.24) - open/close 3319 (time 1789607301.39 total 50.07 last 59.96) - open/close 3947 (time 1789607311.39 total 60.08 last 62.76) total: 4500 open/close in 68.43 seconds: 65.76 ops/second - /unlink 1494 (time 1789607336.62 total 10.01 last 149.21) - /unlink 2729 (time 1789607346.63 total 20.02 last 123.44) - /unlink 4069 (time 1789607356.64 total 30.03 last 133.88) total: 4500 /unlink in 32.82 seconds: 137.12 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 752 (time 1789607374.14 total 10.02 last 75.06) - open/close 1446 (time 1789607384.14 total 20.02 last 69.38) - open/close 2291 (time 1789607394.14 total 30.02 last 84.49) - open/close 3065 (time 1789607404.15 total 40.03 last 77.36) - open/close 3920 (time 1789607414.16 total 50.04 last 85.35) total: 4500 open/close in 57.48 seconds: 78.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1170 (time 1789607437.64 total 10.01 last 116.88) - /unlink 2314 (time 1789607447.64 total 20.02 last 114.31) - /unlink 3535 (time 1789607457.64 total 30.02 last 122.08) total: 4500 /unlink in 38.30 seconds: 117.49 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 647 (time 1789607482.24 total 10.01 last 64.62) - open/close 1368 (time 1789607492.26 total 20.03 last 71.98) - open/close 2049 (time 1789607502.26 total 30.03 last 68.10) - open/close 2867 (time 1789607512.27 total 40.04 last 81.75) - open/close 3602 (time 1789607522.28 total 50.05 last 73.43) - open/close 4281 (time 1789607532.29 total 60.06 last 67.80) total: 4500 open/close in 62.59 seconds: 71.90 ops/second - /unlink 1375 (time 1789607550.29 total 10.00 last 137.47) - /unlink 2425 (time 1789607560.29 total 20.01 last 104.93) - /unlink 3629 (time 1789607570.29 total 30.01 last 120.39) total: 4500 /unlink in 36.98 seconds: 121.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 626 (time 1789607592.88 total 10.00 last 62.58) - open/close 1340 (time 1789607602.89 total 20.02 last 71.31) - open/close 1975 (time 1789607612.89 total 30.02 last 63.49) - open/close 2657 (time 1789607622.91 total 40.03 last 68.09) - open/close 3427 (time 1789607632.91 total 50.04 last 76.97) - open/close 4259 (time 1789607642.92 total 60.05 last 83.11) total: 4500 open/close in 62.98 seconds: 71.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1305 (time 1789607661.22 total 10.01 last 130.36) - /unlink 2506 (time 1789607671.23 total 20.02 last 119.99) - /unlink 3564 (time 1789607681.24 total 30.04 last 105.62) total: 4500 /unlink in 39.79 seconds: 113.10 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 594 (time 1789607706.00 total 10.01 last 59.34) - open/close 1269 (time 1789607716.01 total 20.02 last 67.44) - open/close 2001 (time 1789607726.02 total 30.03 last 73.09) - open/close 2660 (time 1789607736.04 total 40.05 last 65.80) - open/close 3298 (time 1789607746.04 total 50.05 last 63.80) - open/close 4037 (time 1789607756.05 total 60.06 last 73.80) total: 4500 open/close in 66.51 seconds: 67.66 ops/second - /unlink 1365 (time 1789607777.98 total 10.00 last 136.44) - /unlink 2658 (time 1789607787.98 total 20.01 last 129.25) - /unlink 3934 (time 1789607797.98 total 30.01 last 127.59) total: 4500 /unlink in 34.21 seconds: 131.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 726 (time 1789607817.83 total 10.00 last 72.58) - open/close 1506 (time 1789607827.84 total 20.01 last 77.94) - open/close 2281 (time 1789607837.86 total 30.03 last 77.32) - open/close 2980 (time 1789607847.87 total 40.04 last 69.87) - open/close 3723 (time 1789607857.88 total 50.05 last 74.22) - open/close 4446 (time 1789607867.88 total 60.05 last 72.27) total: 4500 open/close in 60.97 seconds: 73.81 ops/second - /unlink 1381 (time 1789607883.46 total 10.00 last 138.09) - /unlink 2622 (time 1789607893.46 total 20.00 last 124.07) - /unlink 3987 (time 1789607903.47 total 30.01 last 136.43) total: 4500 /unlink in 33.63 seconds: 133.83 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 727 (time 1789607922.31 total 10.01 last 72.60) - open/close 1480 (time 1789607932.32 total 20.03 last 75.19) - open/close 2246 (time 1789607942.34 total 30.05 last 76.44) - open/close 2998 (time 1789607952.35 total 40.06 last 75.16) - open/close 3725 (time 1789607962.36 total 50.07 last 72.60) - open/close 4495 (time 1789607972.37 total 60.08 last 76.93) total: 4500 open/close in 60.12 seconds: 74.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1213 (time 1789607988.36 total 10.00 last 121.29) - /unlink 2283 (time 1789607998.36 total 20.00 last 106.98) - /unlink 3657 (time 1789608008.37 total 30.00 last 137.37) total: 4500 /unlink in 36.68 seconds: 122.67 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 701 (time 1789608030.01 total 10.00 last 70.10) - open/close 1377 (time 1789608040.03 total 20.02 last 67.49) - open/close 2176 (time 1789608050.04 total 30.02 last 79.84) - open/close 2797 (time 1789608060.04 total 40.03 last 62.07) - open/close 3462 (time 1789608070.05 total 50.04 last 66.44) - open/close 4060 (time 1789608080.06 total 60.05 last 59.74) total: 4500 open/close in 67.16 seconds: 67.00 ops/second - /unlink 1279 (time 1789608102.47 total 10.00 last 127.85) - /unlink 2626 (time 1789608112.48 total 20.01 last 134.58) - /unlink 3773 (time 1789608122.48 total 30.02 last 114.61) total: 4500 /unlink in 36.28 seconds: 124.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 640 (time 1789608144.93 total 10.01 last 63.92) - open/close 1202 (time 1789608154.94 total 20.02 last 56.19) - open/close 1799 (time 1789608164.96 total 30.04 last 59.54) - open/close 2368 (time 1789608174.97 total 40.05 last 56.88) - open/close 2892 (time 1789608184.97 total 50.05 last 52.39) - open/close 3449 (time 1789608194.99 total 60.06 last 55.61) - open/close 4018 (time 1789608205.00 total 70.08 last 56.79) total: 4500 open/close in 75.98 seconds: 59.22 ops/second - /unlink 1281 (time 1789608225.65 total 10.00 last 128.04) - /unlink 2583 (time 1789608235.65 total 20.01 last 130.15) - /unlink 3978 (time 1789608245.65 total 30.01 last 139.48) total: 4500 /unlink in 33.26 seconds: 135.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 680 (time 1789608264.31 total 10.00 last 67.98) - open/close 1486 (time 1789608274.31 total 20.01 last 80.57) - open/close 2156 (time 1789608284.33 total 30.03 last 66.88) - open/close 2776 (time 1789608294.34 total 40.03 last 61.97) - open/close 3453 (time 1789608304.34 total 50.04 last 67.67) - open/close 4037 (time 1789608314.35 total 60.04 last 58.38) total: 4500 open/close in 68.09 seconds: 66.09 ops/second - /unlink 1293 (time 1789608339.73 total 10.00 last 129.25) - /unlink 2616 (time 1789608349.73 total 20.01 last 132.23) - /unlink 3968 (time 1789608359.74 total 30.02 last 135.06) total: 4500 /unlink in 33.97 seconds: 132.48 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 673 (time 1789608378.61 total 10.00 last 67.27) - open/close 1319 (time 1789608388.63 total 20.02 last 64.51) - open/close 2063 (time 1789608398.63 total 30.03 last 74.35) - open/close 2872 (time 1789608408.64 total 40.03 last 80.88) - open/close 3586 (time 1789608418.64 total 50.04 last 71.35) - open/close 4308 (time 1789608428.66 total 60.05 last 72.11) total: 4500 open/close in 62.50 seconds: 72.00 ops/second - /unlink 1163 (time 1789608446.65 total 10.01 last 116.20) - /unlink 2366 (time 1789608456.67 total 20.03 last 120.02) - /unlink 3672 (time 1789608466.67 total 30.04 last 130.55) total: 4500 /unlink in 36.71 seconds: 122.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 685 (time 1789608488.68 total 10.01 last 68.41) - open/close 1327 (time 1789608498.70 total 20.03 last 64.11) - open/close 2022 (time 1789608508.72 total 30.05 last 69.34) - open/close 2696 (time 1789608518.74 total 40.07 last 67.28) - open/close 3342 (time 1789608528.75 total 50.08 last 64.53) - open/close 4087 (time 1789608538.76 total 60.09 last 74.44) total: 4500 open/close in 66.36 seconds: 67.81 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1437 (time 1789608560.05 total 10.00 last 143.68) - /unlink 2762 (time 1789608570.05 total 20.00 last 132.49) - /unlink 4165 (time 1789608580.05 total 30.00 last 140.30) total: 4500 /unlink in 32.50 seconds: 138.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 764 (time 1789608597.96 total 10.00 last 76.38) - open/close 1523 (time 1789608607.96 total 20.01 last 75.88) - open/close 2270 (time 1789608617.97 total 30.01 last 74.64) - open/close 3007 (time 1789608627.97 total 40.02 last 73.65) - open/close 3740 (time 1789608637.98 total 50.02 last 73.26) - open/close 4411 (time 1789608647.98 total 60.03 last 67.07) total: 4500 open/close in 60.93 seconds: 73.85 ops/second - /unlink 1143 (time 1789608663.37 total 10.01 last 114.21) - /unlink 2226 (time 1789608673.39 total 20.02 last 108.14) - /unlink 3403 (time 1789608683.39 total 30.03 last 117.65) - /unlink 4430 (time 1789608693.39 total 40.03 last 102.69) total: 4500 /unlink in 41.07 seconds: 109.58 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 834 (time 1789608712.18 total 10.01 last 83.32) - open/close 1588 (time 1789608722.21 total 20.04 last 75.17) - open/close 2391 (time 1789608732.23 total 30.05 last 80.18) - open/close 3141 (time 1789608742.23 total 40.06 last 74.96) - open/close 3791 (time 1789608752.26 total 50.09 last 64.84) - open/close 4430 (time 1789608762.27 total 60.10 last 63.79) total: 4500 open/close in 60.86 seconds: 73.94 ops/second - /unlink 1564 (time 1789608777.71 total 10.00 last 156.33) - /unlink 2976 (time 1789608787.71 total 20.01 last 141.19) - /unlink 4146 (time 1789608797.72 total 30.02 last 116.88) total: 4500 /unlink in 32.81 seconds: 137.16 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11350: 95343 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg213-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg213-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg213-server stop mds service on oleg213-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg213-server stop mds service on oleg213-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg213-server unloading modules via unload_modules_local on: 'oleg213-server' oleg213-server: oleg213-server.virtnet: executing unload_modules_local oleg213-server: kmemleak disabled modules unloaded.