== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 21:35:36 (1789522536) Stopping clients: oleg257-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg257-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg257-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg257-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg257-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg257-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg257-server oleg257-server: oleg257-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg257-server' oleg257-server: oleg257-server.virtnet: executing load_modules_local oleg257-server: kmemleak disabled oleg257-server: Loading modules from /home/green/git/lustre-release/lustre oleg257-server: detected 4 online CPUs by sysfs oleg257-server: MODOPTS_LIBCFS= oleg257-server: Force libcfs to create 2 CPU partitions oleg257-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg257-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg257-server: oleg257-server.virtnet: executing set_default_debug -1 all pdsh@oleg257-client: oleg257-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg257-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg257-server: oleg257-server.virtnet: executing set_default_debug -1 all pdsh@oleg257-client: oleg257-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg257-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg257-server: oleg257-server.virtnet: executing set_default_debug -1 all pdsh@oleg257-client: oleg257-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg257-client.virtnet: -o user_xattr,flock 192.168.202.157@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 744 (time 1789522749.53 total 10.00 last 74.37) - open/close 1488 (time 1789522759.55 total 20.02 last 74.27) - open/close 2220 (time 1789522769.56 total 30.03 last 73.11) - open/close 2995 (time 1789522779.57 total 40.04 last 77.48) - open/close 3737 (time 1789522789.57 total 50.04 last 74.15) total: 4500 open/close in 59.47 seconds: 75.67 ops/second - /unlink 1649 (time 1789522813.39 total 10.00 last 164.82) - /unlink 3192 (time 1789522823.39 total 20.01 last 154.25) total: 4500 /unlink in 28.86 seconds: 155.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 768 (time 1789522846.69 total 10.01 last 76.76) - open/close 1597 (time 1789522856.71 total 20.02 last 82.77) - open/close 2316 (time 1789522866.71 total 30.02 last 71.88) - open/close 3051 (time 1789522876.71 total 40.03 last 73.47) - open/close 3818 (time 1789522886.72 total 50.04 last 76.65) total: 4500 open/close in 58.35 seconds: 77.13 ops/second - /unlink 1526 (time 1789522909.45 total 10.01 last 152.52) - /unlink 3267 (time 1789522919.45 total 20.01 last 174.08) total: 4500 /unlink in 27.35 seconds: 164.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 728 (time 1789522941.57 total 10.01 last 72.74) - open/close 1476 (time 1789522951.58 total 20.02 last 74.73) - open/close 2237 (time 1789522961.60 total 30.03 last 76.01) - open/close 2953 (time 1789522971.60 total 40.03 last 71.59) - open/close 3642 (time 1789522981.61 total 50.04 last 68.84) - open/close 4366 (time 1789522991.62 total 60.05 last 72.31) total: 4500 open/close in 61.76 seconds: 72.86 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1319 (time 1789523007.88 total 10.00 last 131.87) - /unlink 2625 (time 1789523017.88 total 20.00 last 130.59) - /unlink 4027 (time 1789523027.88 total 30.01 last 140.17) total: 4500 /unlink in 33.60 seconds: 133.95 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 688 (time 1789523046.92 total 10.01 last 68.75) - open/close 1396 (time 1789523056.92 total 20.01 last 70.79) - open/close 1983 (time 1789523066.92 total 30.01 last 58.68) - open/close 2631 (time 1789523076.95 total 40.04 last 64.62) - open/close 3366 (time 1789523086.95 total 50.04 last 73.48) - open/close 4075 (time 1789523096.95 total 60.05 last 70.89) total: 4500 open/close in 65.63 seconds: 68.56 ops/second - /unlink 1563 (time 1789523116.34 total 10.00 last 156.28) - /unlink 3096 (time 1789523126.35 total 20.01 last 153.22) total: 4500 /unlink in 29.27 seconds: 153.74 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 806 (time 1789523149.68 total 10.00 last 80.56) - open/close 1524 (time 1789523159.69 total 20.02 last 71.72) - open/close 2296 (time 1789523169.71 total 30.04 last 77.02) - open/close 3073 (time 1789523179.71 total 40.04 last 77.68) - open/close 3870 (time 1789523189.71 total 50.04 last 79.69) total: 4500 open/close in 57.78 seconds: 77.89 ops/second - /unlink 1405 (time 1789523211.54 total 10.01 last 140.41) - /unlink 2770 (time 1789523221.54 total 20.01 last 136.50) - /unlink 4152 (time 1789523231.54 total 30.01 last 138.19) total: 4500 /unlink in 32.49 seconds: 138.50 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 864 (time 1789523247.39 total 10.00 last 86.37) - open/close 1769 (time 1789523257.40 total 20.01 last 90.45) - open/close 2577 (time 1789523267.41 total 30.02 last 80.73) - open/close 3403 (time 1789523277.42 total 40.03 last 82.47) - open/close 4131 (time 1789523287.43 total 50.04 last 72.78) total: 4500 open/close in 54.78 seconds: 82.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1649 (time 1789523306.34 total 10.00 last 164.86) - /unlink 2941 (time 1789523316.34 total 20.00 last 129.19) - /unlink 4361 (time 1789523326.35 total 30.01 last 141.96) total: 4500 /unlink in 31.05 seconds: 144.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 818 (time 1789523341.59 total 10.00 last 81.76) - open/close 1562 (time 1789523351.60 total 20.01 last 74.39) - open/close 2313 (time 1789523361.60 total 30.01 last 75.05) - open/close 3146 (time 1789523371.60 total 40.01 last 83.30) - open/close 3994 (time 1789523381.61 total 50.02 last 84.75) total: 4500 open/close in 56.18 seconds: 80.10 ops/second - /unlink 1505 (time 1789523401.42 total 10.00 last 150.48) - /unlink 3252 (time 1789523411.43 total 20.01 last 174.53) total: 4500 /unlink in 26.97 seconds: 166.83 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 891 (time 1789523431.93 total 10.03 last 88.81) - open/close 1810 (time 1789523441.93 total 20.03 last 91.90) - open/close 2661 (time 1789523451.94 total 30.04 last 85.02) - open/close 3494 (time 1789523461.95 total 40.06 last 83.17) - open/close 4385 (time 1789523471.96 total 50.07 last 89.02) total: 4500 open/close in 51.36 seconds: 87.62 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1633 (time 1789523486.49 total 10.00 last 163.24) - /unlink 3335 (time 1789523496.49 total 20.01 last 170.14) total: 4500 /unlink in 27.39 seconds: 164.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 915 (time 1789523517.37 total 10.01 last 91.41) - open/close 1784 (time 1789523527.37 total 20.01 last 86.90) - open/close 2632 (time 1789523537.37 total 30.01 last 84.79) - open/close 3603 (time 1789523547.38 total 40.02 last 97.00) total: 4500 open/close in 49.48 seconds: 90.94 ops/second - /unlink 1704 (time 1789523570.66 total 10.00 last 170.36) - /unlink 3426 (time 1789523580.66 total 20.00 last 172.19) total: 4500 /unlink in 25.79 seconds: 174.48 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 908 (time 1789523599.58 total 10.01 last 90.70) - open/close 1799 (time 1789523609.58 total 20.01 last 89.10) - open/close 2686 (time 1789523619.59 total 30.02 last 88.62) - open/close 3555 (time 1789523629.60 total 40.03 last 86.87) - open/close 4492 (time 1789523639.60 total 50.03 last 93.66) total: 4500 open/close in 50.11 seconds: 89.80 ops/second - /unlink 1640 (time 1789523653.17 total 10.00 last 164.00) - /unlink 3231 (time 1789523663.17 total 20.01 last 159.02) total: 4500 /unlink in 26.84 seconds: 167.66 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 908 (time 1789523683.45 total 10.01 last 90.73) - open/close 1858 (time 1789523693.45 total 20.01 last 94.95) - open/close 2738 (time 1789523703.46 total 30.02 last 87.97) - open/close 3576 (time 1789523713.46 total 40.02 last 83.77) - open/close 4436 (time 1789523723.47 total 50.03 last 85.90) total: 4500 open/close in 50.75 seconds: 88.67 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1770 (time 1789523737.31 total 10.01 last 176.80) - /unlink 3588 (time 1789523747.31 total 20.02 last 181.71) total: 4500 /unlink in 25.01 seconds: 179.92 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 873 (time 1789523766.27 total 10.01 last 87.20) - open/close 1763 (time 1789523776.28 total 20.02 last 88.91) - open/close 2687 (time 1789523786.29 total 30.03 last 92.35) - open/close 3558 (time 1789523796.30 total 40.04 last 86.96) total: 4500 open/close in 49.88 seconds: 90.22 ops/second - /unlink 1609 (time 1789523819.96 total 10.01 last 160.71) - /unlink 3175 (time 1789523829.96 total 20.01 last 156.56) total: 4500 /unlink in 28.68 seconds: 156.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 962 (time 1789523852.58 total 10.00 last 96.17) - open/close 1864 (time 1789523862.59 total 20.01 last 90.13) - open/close 2746 (time 1789523872.59 total 30.01 last 88.18) - open/close 3635 (time 1789523882.59 total 40.02 last 88.85) - open/close 4485 (time 1789523892.60 total 50.03 last 84.90) total: 4500 open/close in 50.16 seconds: 89.72 ops/second - /unlink 1749 (time 1789523905.81 total 10.00 last 174.87) - /unlink 3694 (time 1789523915.81 total 20.00 last 194.47) total: 4500 /unlink in 23.38 seconds: 192.48 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 970 (time 1789523932.39 total 10.00 last 96.99) - open/close 1995 (time 1789523942.40 total 20.01 last 102.45) - open/close 2952 (time 1789523952.41 total 30.01 last 95.62) - open/close 3998 (time 1789523962.42 total 40.03 last 104.43) total: 4500 open/close in 45.22 seconds: 99.51 ops/second - /unlink 1600 (time 1789523980.78 total 10.00 last 159.96) - /unlink 3434 (time 1789523990.78 total 20.00 last 183.38) total: 4500 /unlink in 25.39 seconds: 177.21 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 974 (time 1789524009.62 total 10.01 last 97.34) - open/close 1877 (time 1789524019.62 total 20.01 last 90.27) - open/close 2814 (time 1789524029.63 total 30.01 last 93.65) - open/close 3722 (time 1789524039.63 total 40.02 last 90.76) total: 4500 open/close in 48.11 seconds: 93.54 ops/second - /unlink 1883 (time 1789524060.79 total 10.00 last 188.28) - /unlink 3745 (time 1789524070.79 total 20.00 last 186.18) total: 4500 /unlink in 23.43 seconds: 192.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 927 (time 1789524087.72 total 10.01 last 92.59) - open/close 1903 (time 1789524097.72 total 20.01 last 97.59) - open/close 3022 (time 1789524107.73 total 30.02 last 111.79) - open/close 4188 (time 1789524117.73 total 40.02 last 116.58) total: 4500 open/close in 42.45 seconds: 106.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 2038 (time 1789524133.03 total 10.00 last 203.73) - /unlink 4202 (time 1789524143.03 total 20.01 last 216.36) total: 4500 /unlink in 21.66 seconds: 207.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 1038 (time 1789524157.86 total 10.01 last 103.69) - open/close 2050 (time 1789524167.87 total 20.02 last 101.13) - open/close 3058 (time 1789524177.87 total 30.02 last 100.78) - open/close 4034 (time 1789524187.88 total 40.03 last 97.48) total: 4500 open/close in 44.65 seconds: 100.78 ops/second - /unlink 2020 (time 1789524205.23 total 10.00 last 201.99) - /unlink 3858 (time 1789524215.23 total 20.00 last 183.77) total: 4500 /unlink in 23.44 seconds: 191.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 994 (time 1789524232.04 total 10.00 last 99.38) - open/close 2034 (time 1789524242.04 total 20.00 last 103.98) - open/close 3151 (time 1789524252.04 total 30.00 last 111.70) - open/close 4232 (time 1789524262.04 total 40.01 last 108.05) total: 4500 open/close in 42.31 seconds: 106.37 ops/second - /unlink 2292 (time 1789524276.54 total 10.01 last 229.07) - /unlink 4444 (time 1789524286.54 total 20.01 last 215.12) total: 4500 /unlink in 20.51 seconds: 219.36 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11350: 95327 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg257-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg257-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg257-server stop mds service on oleg257-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg257-server stop mds service on oleg257-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg257-server unloading modules via unload_modules_local on: 'oleg257-server' oleg257-server: oleg257-server.virtnet: executing unload_modules_local oleg257-server: kmemleak disabled modules unloaded.