== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 09:47:00 (1788443220) Stopping clients: oleg104-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg104-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg104-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg104-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg104-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg104-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg104-server oleg104-server: oleg104-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg104-server' oleg104-server: oleg104-server.virtnet: executing load_modules_local oleg104-server: kmemleak disabled oleg104-server: Loading modules from /home/green/git/lustre-release/lustre oleg104-server: detected 4 online CPUs by sysfs oleg104-server: MODOPTS_LIBCFS= oleg104-server: Force libcfs to create 2 CPU partitions oleg104-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg104-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg104-server: oleg104-server.virtnet: executing set_default_debug -1 all pdsh@oleg104-client: oleg104-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg104-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg104-server: oleg104-server.virtnet: executing set_default_debug -1 all pdsh@oleg104-client: oleg104-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg104-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg104-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg104-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg104-server: oleg104-server.virtnet: executing set_default_debug -1 all pdsh@oleg104-client: oleg104-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg104-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg104-client.virtnet: -o user_xattr,flock 192.168.201.104@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 519 (time 1788443431.59 total 10.00 last 51.90) - open/close 1286 (time 1788443441.60 total 20.01 last 76.63) - open/close 2054 (time 1788443451.60 total 30.01 last 76.78) - open/close 2858 (time 1788443461.61 total 40.02 last 80.33) - open/close 3649 (time 1788443471.61 total 50.03 last 79.07) - open/close 4343 (time 1788443481.64 total 60.06 last 69.19) total: 4500 open/close in 62.51 seconds: 71.99 ops/second - /unlink 1483 (time 1788443499.27 total 10.01 last 148.19) - /unlink 2916 (time 1788443509.28 total 20.01 last 143.20) - /unlink 4484 (time 1788443519.28 total 30.02 last 156.75) total: 4500 /unlink in 30.09 seconds: 149.56 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 818 (time 1788443534.30 total 10.01 last 81.68) - open/close 1540 (time 1788443544.31 total 20.02 last 72.15) - open/close 2184 (time 1788443554.31 total 30.03 last 64.35) - open/close 2944 (time 1788443564.32 total 40.03 last 75.98) - open/close 3552 (time 1788443574.32 total 50.04 last 60.78) - open/close 4088 (time 1788443584.33 total 60.04 last 53.55) total: 4500 open/close in 65.73 seconds: 68.46 ops/second - /unlink 1340 (time 1788443605.46 total 10.03 last 133.65) - /unlink 2925 (time 1788443615.47 total 20.03 last 158.42) - /unlink 4112 (time 1788443625.47 total 30.04 last 118.64) total: 4500 /unlink in 32.52 seconds: 138.38 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 778 (time 1788443643.35 total 10.00 last 77.79) - open/close 1513 (time 1788443653.35 total 20.01 last 73.46) - open/close 2254 (time 1788443663.35 total 30.01 last 74.10) - open/close 3014 (time 1788443673.36 total 40.01 last 75.93) - open/close 3824 (time 1788443683.36 total 50.02 last 80.99) total: 4500 open/close in 56.92 seconds: 79.06 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1559 (time 1788443704.30 total 10.00 last 155.85) - /unlink 2774 (time 1788443714.31 total 20.01 last 121.48) - /unlink 4335 (time 1788443724.31 total 30.01 last 156.10) total: 4500 /unlink in 31.26 seconds: 143.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 922 (time 1788443739.44 total 10.01 last 92.11) - open/close 1790 (time 1788443749.44 total 20.02 last 86.76) - open/close 2622 (time 1788443759.45 total 30.03 last 83.11) - open/close 3424 (time 1788443769.46 total 40.04 last 80.12) - open/close 4184 (time 1788443779.47 total 50.04 last 75.96) total: 4500 open/close in 54.20 seconds: 83.02 ops/second - /unlink 1589 (time 1788443798.53 total 10.01 last 158.81) - /unlink 3258 (time 1788443808.54 total 20.01 last 166.83) total: 4500 /unlink in 28.26 seconds: 159.26 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 893 (time 1788443832.16 total 10.02 last 89.12) - open/close 1719 (time 1788443842.16 total 20.02 last 82.60) - open/close 2570 (time 1788443852.18 total 30.04 last 84.93) - open/close 3500 (time 1788443862.21 total 40.06 last 92.79) - open/close 4311 (time 1788443872.22 total 50.08 last 80.99) total: 4500 open/close in 52.45 seconds: 85.80 ops/second - /unlink 1659 (time 1788443889.09 total 10.00 last 165.86) - /unlink 3316 (time 1788443899.09 total 20.00 last 165.66) total: 4500 /unlink in 27.42 seconds: 164.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 877 (time 1788443920.69 total 10.00 last 87.67) - open/close 1769 (time 1788443930.70 total 20.01 last 89.16) - open/close 2608 (time 1788443940.71 total 30.02 last 83.80) - open/close 3199 (time 1788443950.74 total 40.05 last 58.93) - open/close 3731 (time 1788443960.75 total 50.06 last 53.12) - open/close 4246 (time 1788443970.76 total 60.07 last 51.45) total: 4500 open/close in 63.21 seconds: 71.19 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1627 (time 1788443988.16 total 10.01 last 162.49) - /unlink 2857 (time 1788443998.16 total 20.02 last 122.94) - /unlink 4003 (time 1788444008.17 total 30.02 last 114.57) total: 4500 /unlink in 33.26 seconds: 135.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 915 (time 1788444025.63 total 10.01 last 91.39) - open/close 1740 (time 1788444035.64 total 20.03 last 82.37) - open/close 2609 (time 1788444045.64 total 30.03 last 86.90) - open/close 3447 (time 1788444055.66 total 40.04 last 83.67) - open/close 4003 (time 1788444065.66 total 50.05 last 55.56) total: 4500 open/close in 59.63 seconds: 75.46 ops/second - /unlink 1439 (time 1788444090.17 total 10.00 last 143.90) - /unlink 2638 (time 1788444100.17 total 20.00 last 119.85) - /unlink 4213 (time 1788444110.17 total 30.00 last 157.50) total: 4500 /unlink in 31.77 seconds: 141.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 807 (time 1788444127.59 total 10.02 last 80.56) - open/close 1549 (time 1788444137.60 total 20.02 last 74.16) - open/close 2286 (time 1788444147.60 total 30.02 last 73.69) - open/close 2971 (time 1788444157.63 total 40.06 last 68.28) - open/close 3668 (time 1788444167.64 total 50.06 last 69.65) - open/close 4491 (time 1788444177.65 total 60.08 last 82.20) total: 4500 open/close in 60.19 seconds: 74.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1639 (time 1788444192.75 total 10.00 last 163.89) - /unlink 3185 (time 1788444202.75 total 20.01 last 154.53) total: 4500 /unlink in 28.81 seconds: 156.17 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 903 (time 1788444226.53 total 10.00 last 90.28) - open/close 1681 (time 1788444236.55 total 20.02 last 77.64) - open/close 2299 (time 1788444246.55 total 30.03 last 61.78) - open/close 3016 (time 1788444256.56 total 40.04 last 71.62) - open/close 3695 (time 1788444266.58 total 50.05 last 67.81) - open/close 4484 (time 1788444276.59 total 60.06 last 78.78) total: 4500 open/close in 60.30 seconds: 74.63 ops/second - /unlink 1686 (time 1788444291.07 total 10.00 last 168.55) - /unlink 3288 (time 1788444301.08 total 20.01 last 160.03) - /unlink 4377 (time 1788444311.09 total 30.02 last 108.83) total: 4500 /unlink in 31.32 seconds: 143.68 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 947 (time 1788444327.74 total 10.01 last 94.61) - open/close 1741 (time 1788444337.77 total 20.04 last 79.18) - open/close 2592 (time 1788444347.77 total 30.04 last 85.09) - open/close 3567 (time 1788444357.78 total 40.05 last 97.39) - open/close 4384 (time 1788444367.81 total 50.08 last 81.49) total: 4500 open/close in 51.75 seconds: 86.96 ops/second - /unlink 1642 (time 1788444382.99 total 10.01 last 164.08) - /unlink 3261 (time 1788444392.99 total 20.01 last 161.85) total: 4500 /unlink in 26.76 seconds: 168.17 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 887 (time 1788444414.24 total 10.01 last 88.64) - open/close 1761 (time 1788444424.26 total 20.02 last 87.27) - open/close 2489 (time 1788444434.26 total 30.02 last 72.80) - open/close 3338 (time 1788444444.27 total 40.04 last 84.76) - open/close 4232 (time 1788444454.29 total 50.05 last 89.27) total: 4500 open/close in 53.05 seconds: 84.82 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1813 (time 1788444472.18 total 10.00 last 181.23) - /unlink 3197 (time 1788444482.19 total 20.01 last 138.36) total: 4500 /unlink in 28.36 seconds: 158.67 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 912 (time 1788444505.57 total 10.00 last 91.20) - open/close 1802 (time 1788444515.58 total 20.00 last 88.96) - open/close 2726 (time 1788444525.58 total 30.01 last 92.36) - open/close 3559 (time 1788444535.58 total 40.01 last 83.30) - open/close 4493 (time 1788444545.58 total 50.01 last 93.39) total: 4500 open/close in 50.06 seconds: 89.89 ops/second - /unlink 1634 (time 1788444560.59 total 10.00 last 163.34) - /unlink 3173 (time 1788444570.60 total 20.01 last 153.86) total: 4500 /unlink in 28.26 seconds: 159.26 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 854 (time 1788444594.21 total 10.00 last 85.40) - open/close 1634 (time 1788444604.23 total 20.02 last 77.87) - open/close 2235 (time 1788444614.24 total 30.03 last 60.01) - open/close 3007 (time 1788444624.25 total 40.04 last 77.15) - open/close 3929 (time 1788444634.26 total 50.05 last 92.08) total: 4500 open/close in 57.42 seconds: 78.38 ops/second - /unlink 1568 (time 1788444656.28 total 10.00 last 156.78) - /unlink 3095 (time 1788444666.28 total 20.01 last 152.62) - /unlink 4197 (time 1788444676.28 total 30.01 last 110.16) total: 4500 /unlink in 32.28 seconds: 139.40 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 822 (time 1788444694.61 total 10.01 last 82.15) - open/close 1572 (time 1788444704.62 total 20.01 last 74.95) - open/close 2225 (time 1788444714.63 total 30.03 last 65.21) - open/close 2810 (time 1788444724.64 total 40.03 last 58.49) - open/close 3404 (time 1788444734.64 total 50.03 last 59.39) - open/close 4008 (time 1788444744.65 total 60.05 last 60.31) total: 4500 open/close in 66.62 seconds: 67.55 ops/second - /unlink 1574 (time 1788444765.53 total 10.00 last 157.32) - /unlink 3270 (time 1788444775.54 total 20.01 last 169.57) total: 4500 /unlink in 28.16 seconds: 159.81 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 756 (time 1788444799.72 total 10.01 last 75.49) - open/close 1573 (time 1788444809.72 total 20.02 last 81.68) - open/close 2399 (time 1788444819.74 total 30.03 last 82.46) - open/close 3155 (time 1788444829.74 total 40.04 last 75.58) - open/close 3938 (time 1788444839.76 total 50.05 last 78.19) total: 4500 open/close in 56.14 seconds: 80.15 ops/second - /unlink 1518 (time 1788444860.33 total 10.01 last 151.60) - /unlink 2796 (time 1788444870.33 total 20.02 last 127.76) - /unlink 4237 (time 1788444880.34 total 30.02 last 144.09) total: 4500 /unlink in 31.97 seconds: 140.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 807 (time 1788444897.70 total 10.01 last 80.63) - open/close 1601 (time 1788444907.70 total 20.01 last 79.38) - open/close 2358 (time 1788444917.71 total 30.02 last 75.64) - open/close 3035 (time 1788444927.72 total 40.02 last 67.66) - open/close 3738 (time 1788444937.72 total 50.03 last 70.28) total: 4500 open/close in 59.73 seconds: 75.34 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1340 (time 1788444961.99 total 10.01 last 133.85) - /unlink 2729 (time 1788444972.00 total 20.03 last 138.65) - /unlink 4188 (time 1788444982.00 total 30.03 last 145.89) total: 4500 /unlink in 32.21 seconds: 139.72 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 768 (time 1788445000.07 total 10.02 last 76.67) - open/close 1546 (time 1788445010.07 total 20.02 last 77.78) - open/close 2330 (time 1788445020.08 total 30.03 last 78.34) - open/close 3112 (time 1788445030.10 total 40.04 last 78.09) - open/close 3743 (time 1788445040.10 total 50.05 last 63.06) - open/close 4368 (time 1788445050.12 total 60.06 last 62.41) total: 4500 open/close in 62.87 seconds: 71.58 ops/second - /unlink 1478 (time 1788445069.75 total 10.00 last 147.76) - /unlink 2873 (time 1788445079.75 total 20.00 last 139.49) - /unlink 4346 (time 1788445089.75 total 30.01 last 147.26) total: 4500 /unlink in 31.49 seconds: 142.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 695 (time 1788445106.96 total 10.01 last 69.45) - open/close 1484 (time 1788445116.96 total 20.01 last 78.88) - open/close 2285 (time 1788445126.97 total 30.02 last 80.05) - open/close 3046 (time 1788445136.98 total 40.03 last 76.02) - open/close 3834 (time 1788445146.98 total 50.03 last 78.79) total: 4500 open/close in 58.21 seconds: 77.30 ops/second - /unlink 1700 (time 1788445169.78 total 10.00 last 169.97) - /unlink 3189 (time 1788445179.78 total 20.01 last 148.83) total: 4500 /unlink in 29.27 seconds: 153.73 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95239 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg104-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg104-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg104-server stop mds service on oleg104-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg104-server stop mds service on oleg104-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg104-server unloading modules via unload_modules_local on: 'oleg104-server' oleg104-server: oleg104-server.virtnet: executing unload_modules_local oleg104-server: kmemleak disabled modules unloaded.