== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 05:18:37 (1788427117) Stopping clients: oleg233-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg233-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg233-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg233-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg233-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg233-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg233-server oleg233-server: oleg233-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg233-server' oleg233-server: oleg233-server.virtnet: executing load_modules_local oleg233-server: kmemleak disabled oleg233-server: Loading modules from /home/green/git/lustre-release/lustre oleg233-server: detected 4 online CPUs by sysfs oleg233-server: MODOPTS_LIBCFS= oleg233-server: Force libcfs to create 2 CPU partitions oleg233-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg233-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg233-server: oleg233-server.virtnet: executing set_default_debug -1 all pdsh@oleg233-client: oleg233-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg233-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg233-server: oleg233-server.virtnet: executing set_default_debug -1 all pdsh@oleg233-client: oleg233-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg233-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg233-server: oleg233-server.virtnet: executing set_default_debug -1 all pdsh@oleg233-client: oleg233-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg233-client.virtnet: -o user_xattr,flock 192.168.202.133@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 707 (time 1788427362.17 total 10.01 last 70.60) - open/close 1395 (time 1788427372.17 total 20.02 last 68.78) - open/close 2118 (time 1788427382.19 total 30.03 last 72.18) - open/close 2762 (time 1788427392.21 total 40.05 last 64.26) - open/close 3473 (time 1788427402.22 total 50.06 last 71.04) - open/close 4016 (time 1788427412.23 total 60.08 last 54.21) total: 4500 open/close in 69.22 seconds: 65.01 ops/second - /unlink 1608 (time 1788427436.29 total 10.00 last 160.76) - /unlink 2598 (time 1788427446.29 total 20.01 last 98.95) - /unlink 3858 (time 1788427456.30 total 30.01 last 125.97) total: 4500 /unlink in 35.46 seconds: 126.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 652 (time 1788427477.98 total 10.01 last 65.16) - open/close 1281 (time 1788427487.98 total 20.01 last 62.88) - open/close 2048 (time 1788427497.99 total 30.02 last 76.61) - open/close 2672 (time 1788427508.00 total 40.03 last 62.36) - open/close 3278 (time 1788427518.00 total 50.03 last 60.59) - open/close 3964 (time 1788427528.00 total 60.03 last 68.59) total: 4500 open/close in 68.12 seconds: 66.06 ops/second - /unlink 1227 (time 1788427551.92 total 10.00 last 122.68) - /unlink 2496 (time 1788427561.92 total 20.01 last 126.85) - /unlink 3697 (time 1788427571.92 total 30.01 last 120.09) total: 4500 /unlink in 36.49 seconds: 123.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 741 (time 1788427593.98 total 10.01 last 74.04) - open/close 1398 (time 1788427603.99 total 20.02 last 65.64) - open/close 2034 (time 1788427614.00 total 30.02 last 63.56) - open/close 2717 (time 1788427624.04 total 40.06 last 68.04) - open/close 3450 (time 1788427634.05 total 50.07 last 73.22) - open/close 4112 (time 1788427644.06 total 60.08 last 66.16) total: 4500 open/close in 65.81 seconds: 68.38 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1256 (time 1788427665.06 total 10.01 last 125.52) - /unlink 2429 (time 1788427675.08 total 20.02 last 117.10) - /unlink 3722 (time 1788427685.08 total 30.03 last 129.25) total: 4500 /unlink in 38.03 seconds: 118.34 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 421 (time 1788427710.72 total 10.00 last 42.10) - open/close 1063 (time 1788427720.73 total 20.01 last 64.15) - open/close 1585 (time 1788427730.74 total 30.02 last 52.16) - open/close 2150 (time 1788427740.75 total 40.03 last 56.45) - open/close 2802 (time 1788427750.75 total 50.03 last 65.17) - open/close 3312 (time 1788427760.76 total 60.04 last 50.96) - open/close 3898 (time 1788427770.76 total 70.04 last 58.59) total: 4500 open/close in 78.43 seconds: 57.38 ops/second - /unlink 1270 (time 1788427794.35 total 10.00 last 126.96) - /unlink 2484 (time 1788427804.36 total 20.01 last 121.36) - /unlink 3858 (time 1788427814.36 total 30.01 last 137.36) total: 4500 /unlink in 34.79 seconds: 129.33 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 594 (time 1788427834.45 total 10.02 last 59.26) - open/close 1204 (time 1788427844.46 total 20.03 last 60.95) - open/close 1788 (time 1788427854.46 total 30.04 last 58.37) - open/close 2477 (time 1788427864.47 total 40.05 last 68.84) - open/close 3131 (time 1788427874.47 total 50.05 last 65.38) - open/close 3736 (time 1788427884.48 total 60.06 last 60.46) - open/close 4291 (time 1788427894.49 total 70.07 last 55.44) total: 4500 open/close in 74.03 seconds: 60.79 ops/second - /unlink 1056 (time 1788427914.58 total 10.00 last 105.55) - /unlink 2055 (time 1788427924.59 total 20.01 last 99.81) - /unlink 3305 (time 1788427934.60 total 30.02 last 124.97) - /unlink 4457 (time 1788427944.60 total 40.02 last 115.19) total: 4500 /unlink in 40.46 seconds: 111.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 726 (time 1788427961.63 total 10.00 last 72.58) - open/close 1386 (time 1788427971.65 total 20.01 last 65.93) - open/close 1992 (time 1788427981.65 total 30.01 last 60.59) - open/close 2658 (time 1788427991.65 total 40.02 last 66.55) - open/close 3401 (time 1788428001.66 total 50.03 last 74.28) - open/close 4107 (time 1788428011.67 total 60.03 last 70.54) total: 4500 open/close in 65.67 seconds: 68.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1281 (time 1788428032.28 total 10.01 last 127.99) - /unlink 2346 (time 1788428042.28 total 20.01 last 106.49) - /unlink 3595 (time 1788428052.28 total 30.01 last 124.90) total: 4500 /unlink in 37.30 seconds: 120.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 587 (time 1788428075.48 total 10.01 last 58.64) - open/close 1258 (time 1788428085.49 total 20.01 last 67.07) - open/close 1977 (time 1788428095.49 total 30.02 last 71.85) - open/close 2601 (time 1788428105.50 total 40.02 last 62.38) - open/close 3238 (time 1788428115.50 total 50.03 last 63.66) - open/close 4005 (time 1788428125.52 total 60.05 last 76.59) total: 4500 open/close in 68.15 seconds: 66.04 ops/second - /unlink 1266 (time 1788428149.06 total 10.00 last 126.58) - /unlink 2266 (time 1788428159.07 total 20.01 last 99.90) - /unlink 3235 (time 1788428169.09 total 30.03 last 96.74) - /unlink 4451 (time 1788428179.09 total 40.03 last 121.59) total: 4500 /unlink in 40.74 seconds: 110.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 642 (time 1788428194.85 total 10.02 last 64.08) - open/close 1207 (time 1788428204.87 total 20.03 last 56.41) - open/close 1727 (time 1788428214.88 total 30.04 last 51.95) - open/close 2460 (time 1788428224.88 total 40.05 last 73.24) - open/close 3243 (time 1788428234.89 total 50.06 last 78.22) - open/close 3947 (time 1788428244.90 total 60.07 last 70.36) total: 4500 open/close in 68.91 seconds: 65.30 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1340 (time 1788428269.35 total 10.01 last 133.93) - /unlink 2550 (time 1788428279.37 total 20.02 last 120.81) - /unlink 3611 (time 1788428289.37 total 30.02 last 106.07) total: 4500 /unlink in 37.61 seconds: 119.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 491 (time 1788428313.64 total 10.01 last 49.07) - open/close 1046 (time 1788428323.65 total 20.01 last 55.48) - open/close 1564 (time 1788428333.65 total 30.01 last 51.77) - open/close 2237 (time 1788428343.66 total 40.02 last 67.23) - open/close 2881 (time 1788428353.67 total 50.03 last 64.36) - open/close 3580 (time 1788428363.68 total 60.05 last 69.80) - open/close 4247 (time 1788428373.69 total 70.05 last 66.67) total: 4500 open/close in 74.40 seconds: 60.49 ops/second - /unlink 1308 (time 1788428393.65 total 10.01 last 130.61) - /unlink 2473 (time 1788428403.66 total 20.03 last 116.36) - /unlink 3387 (time 1788428413.67 total 30.04 last 91.30) total: 4500 /unlink in 39.66 seconds: 113.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 559 (time 1788428438.76 total 10.02 last 55.80) - open/close 1234 (time 1788428448.77 total 20.02 last 67.47) - open/close 1857 (time 1788428458.77 total 30.02 last 62.28) - open/close 2456 (time 1788428468.79 total 40.04 last 59.81) - open/close 3129 (time 1788428478.79 total 50.04 last 67.30) - open/close 3725 (time 1788428488.79 total 60.04 last 59.59) - open/close 4226 (time 1788428498.80 total 70.06 last 50.03) total: 4500 open/close in 76.59 seconds: 58.76 ops/second - /unlink 1174 (time 1788428522.71 total 10.00 last 117.35) - /unlink 2107 (time 1788428532.71 total 20.01 last 93.26) - /unlink 3554 (time 1788428542.72 total 30.01 last 144.66) total: 4500 /unlink in 37.36 seconds: 120.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 646 (time 1788428565.71 total 10.01 last 64.53) - open/close 1332 (time 1788428575.72 total 20.02 last 68.51) - open/close 1932 (time 1788428585.73 total 30.03 last 59.95) - open/close 2541 (time 1788428595.74 total 40.04 last 60.86) - open/close 3191 (time 1788428605.75 total 50.05 last 64.95) - open/close 3895 (time 1788428615.75 total 60.05 last 70.40) total: 4500 open/close in 68.85 seconds: 65.36 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1459 (time 1788428639.33 total 10.00 last 145.89) - /unlink 2756 (time 1788428649.33 total 20.00 last 129.66) - /unlink 4095 (time 1788428659.33 total 30.00 last 133.90) total: 4500 /unlink in 32.81 seconds: 137.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 742 (time 1788428677.43 total 10.00 last 74.19) - open/close 1435 (time 1788428687.44 total 20.00 last 69.28) - open/close 2216 (time 1788428697.44 total 30.01 last 78.08) - open/close 2903 (time 1788428707.44 total 40.01 last 68.68) - open/close 3619 (time 1788428717.45 total 50.02 last 71.54) - open/close 4307 (time 1788428727.45 total 60.02 last 68.78) total: 4500 open/close in 62.70 seconds: 71.77 ops/second - /unlink 1398 (time 1788428745.88 total 10.00 last 139.75) - /unlink 2555 (time 1788428755.89 total 20.01 last 115.62) - /unlink 3790 (time 1788428765.89 total 30.01 last 123.50) total: 4500 /unlink in 35.92 seconds: 125.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 690 (time 1788428786.30 total 10.01 last 68.95) - open/close 1293 (time 1788428796.31 total 20.02 last 60.23) - open/close 1971 (time 1788428806.31 total 30.02 last 67.76) - open/close 2775 (time 1788428816.32 total 40.03 last 80.36) - open/close 3403 (time 1788428826.33 total 50.04 last 62.72) - open/close 3979 (time 1788428836.34 total 60.05 last 57.54) total: 4500 open/close in 66.78 seconds: 67.38 ops/second - /unlink 1481 (time 1788428857.64 total 10.00 last 148.09) - /unlink 2784 (time 1788428867.65 total 20.01 last 130.23) - /unlink 3910 (time 1788428877.65 total 30.01 last 112.58) total: 4500 /unlink in 36.30 seconds: 123.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 657 (time 1788428900.82 total 10.00 last 65.68) - open/close 1327 (time 1788428910.83 total 20.01 last 66.96) - open/close 1977 (time 1788428920.84 total 30.02 last 64.94) - open/close 2682 (time 1788428930.84 total 40.02 last 70.47) - open/close 3369 (time 1788428940.85 total 50.03 last 68.63) - open/close 4030 (time 1788428950.86 total 60.04 last 66.03) total: 4500 open/close in 66.80 seconds: 67.37 ops/second - /unlink 1298 (time 1788428972.35 total 10.01 last 129.72) - /unlink 2568 (time 1788428982.36 total 20.01 last 126.96) - /unlink 3740 (time 1788428992.38 total 30.03 last 116.96) total: 4500 /unlink in 35.97 seconds: 125.10 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 566 (time 1788429013.00 total 10.01 last 56.54) - open/close 1306 (time 1788429023.01 total 20.02 last 73.93) - open/close 2131 (time 1788429033.02 total 30.02 last 82.46) - open/close 2943 (time 1788429043.04 total 40.05 last 81.01) - open/close 3486 (time 1788429053.05 total 50.06 last 54.25) - open/close 4168 (time 1788429063.06 total 60.07 last 68.13) total: 4500 open/close in 66.07 seconds: 68.11 ops/second - /unlink 1197 (time 1788429085.39 total 10.00 last 119.65) - /unlink 2497 (time 1788429095.39 total 20.01 last 129.99) - /unlink 3732 (time 1788429105.39 total 30.01 last 123.47) total: 4500 /unlink in 36.03 seconds: 124.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 612 (time 1788429126.48 total 10.01 last 61.13) - open/close 1303 (time 1788429136.49 total 20.02 last 69.02) - open/close 2045 (time 1788429146.50 total 30.02 last 74.20) - open/close 2802 (time 1788429156.50 total 40.02 last 75.69) - open/close 3556 (time 1788429166.52 total 50.05 last 75.21) - open/close 4212 (time 1788429176.54 total 60.07 last 65.48) total: 4500 open/close in 64.65 seconds: 69.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1226 (time 1788429196.08 total 10.00 last 122.57) - /unlink 2492 (time 1788429206.09 total 20.01 last 126.55) - /unlink 3527 (time 1788429216.10 total 30.02 last 103.40) total: 4500 /unlink in 38.99 seconds: 115.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 653 (time 1788429239.89 total 10.01 last 65.24) - open/close 1387 (time 1788429249.91 total 20.03 last 73.27) - open/close 2232 (time 1788429259.91 total 30.03 last 84.48) - open/close 2928 (time 1788429269.93 total 40.04 last 69.49) - open/close 3492 (time 1788429279.95 total 50.07 last 56.26) - open/close 4139 (time 1788429289.96 total 60.07 last 64.66) total: 4500 open/close in 65.60 seconds: 68.60 ops/second - /unlink 1160 (time 1788429310.03 total 10.02 last 115.82) - /unlink 2308 (time 1788429320.03 total 20.02 last 114.74) - /unlink 3765 (time 1788429330.04 total 30.02 last 145.67) total: 4500 /unlink in 35.51 seconds: 126.74 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 705 (time 1788429350.18 total 10.00 last 70.48) - open/close 1353 (time 1788429360.18 total 20.00 last 64.80) - open/close 2052 (time 1788429370.19 total 30.01 last 69.85) - open/close 2720 (time 1788429380.20 total 40.02 last 66.74) - open/close 3359 (time 1788429390.21 total 50.03 last 63.84) - open/close 4087 (time 1788429400.21 total 60.03 last 72.77) total: 4500 open/close in 65.08 seconds: 69.14 ops/second - /unlink 1434 (time 1788429419.78 total 10.00 last 143.38) - /unlink 2832 (time 1788429429.79 total 20.01 last 139.67) - /unlink 4341 (time 1788429439.79 total 30.01 last 150.89) total: 4500 /unlink in 31.21 seconds: 144.16 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11349: 95252 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg233-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg233-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg233-server stop mds service on oleg233-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg233-server stop mds service on oleg233-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg233-server unloading modules via unload_modules_local on: 'oleg233-server' oleg233-server: oleg233-server.virtnet: executing unload_modules_local oleg233-server: kmemleak disabled modules unloaded.