== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 10:06:42 (1790777202) Stopping clients: oleg104-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg104-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg104-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg104-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg104-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg104-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg104-server oleg104-server: oleg104-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg104-server' oleg104-server: oleg104-server.virtnet: executing load_modules_local oleg104-server: kmemleak disabled oleg104-server: Loading modules from /home/green/git/lustre-release/lustre oleg104-server: detected 4 online CPUs by sysfs oleg104-server: MODOPTS_LIBCFS= oleg104-server: Force libcfs to create 2 CPU partitions oleg104-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg104-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg104-server: oleg104-server.virtnet: executing set_default_debug -1 all pdsh@oleg104-client: oleg104-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg104-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg104-server: oleg104-server.virtnet: executing set_default_debug -1 all pdsh@oleg104-client: oleg104-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg104-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg104-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg104-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg104-server: oleg104-server.virtnet: executing set_default_debug -1 all pdsh@oleg104-client: oleg104-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg104-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg104-client.virtnet: -o user_xattr,flock 192.168.201.104@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 864 (time 1790777405.03 total 10.01 last 86.35) - open/close 1814 (time 1790777415.04 total 20.01 last 94.93) - open/close 2699 (time 1790777425.04 total 30.01 last 88.50) - open/close 3512 (time 1790777435.06 total 40.03 last 81.14) - open/close 4297 (time 1790777445.06 total 50.04 last 78.48) total: 4500 open/close in 53.29 seconds: 84.44 ops/second - /unlink 1752 (time 1790777465.00 total 10.01 last 175.02) - /unlink 3212 (time 1790777475.00 total 20.01 last 145.96) total: 4500 /unlink in 27.60 seconds: 163.07 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 749 (time 1790777497.88 total 10.00 last 74.90) - open/close 1656 (time 1790777507.88 total 20.00 last 90.66) - open/close 2456 (time 1790777517.90 total 30.02 last 79.90) - open/close 3379 (time 1790777527.90 total 40.02 last 92.22) - open/close 4253 (time 1790777537.91 total 50.04 last 87.31) total: 4500 open/close in 53.28 seconds: 84.47 ops/second - /unlink 1487 (time 1790777554.99 total 10.00 last 148.70) - /unlink 2654 (time 1790777565.00 total 20.00 last 116.66) - /unlink 4158 (time 1790777575.00 total 30.00 last 150.40) total: 4500 /unlink in 31.80 seconds: 141.52 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 947 (time 1790777591.70 total 10.02 last 94.56) - open/close 1788 (time 1790777601.70 total 20.02 last 84.07) - open/close 2707 (time 1790777611.71 total 30.03 last 91.84) - open/close 3576 (time 1790777621.72 total 40.03 last 86.86) - open/close 4404 (time 1790777631.72 total 50.04 last 82.74) total: 4500 open/close in 51.09 seconds: 88.09 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1472 (time 1790777648.42 total 10.00 last 147.16) - /unlink 3250 (time 1790777658.43 total 20.01 last 177.72) total: 4500 /unlink in 27.29 seconds: 164.89 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 860 (time 1790777680.42 total 10.00 last 85.96) - open/close 1650 (time 1790777690.42 total 20.00 last 79.00) - open/close 2548 (time 1790777700.43 total 30.01 last 89.71) - open/close 3385 (time 1790777710.43 total 40.02 last 83.64) - open/close 4347 (time 1790777720.44 total 50.02 last 96.18) total: 4500 open/close in 51.69 seconds: 87.06 ops/second - /unlink 1569 (time 1790777736.44 total 10.03 last 156.49) - /unlink 3255 (time 1790777746.44 total 20.03 last 168.58) total: 4500 /unlink in 27.67 seconds: 162.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 906 (time 1790777769.10 total 10.00 last 90.60) - open/close 1663 (time 1790777779.11 total 20.01 last 75.65) - open/close 2559 (time 1790777789.12 total 30.02 last 89.51) - open/close 3442 (time 1790777799.13 total 40.03 last 88.18) - open/close 4259 (time 1790777809.13 total 50.03 last 81.70) total: 4500 open/close in 52.40 seconds: 85.87 ops/second - /unlink 1874 (time 1790777826.37 total 10.01 last 187.29) - /unlink 3401 (time 1790777836.37 total 20.01 last 152.67) total: 4500 /unlink in 27.96 seconds: 160.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 772 (time 1790777859.12 total 10.01 last 77.14) - open/close 1697 (time 1790777869.13 total 20.01 last 92.47) - open/close 2487 (time 1790777879.14 total 30.02 last 78.90) - open/close 3145 (time 1790777889.15 total 40.04 last 65.72) - open/close 3963 (time 1790777899.16 total 50.05 last 81.71) total: 4500 open/close in 56.81 seconds: 79.21 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1425 (time 1790777920.49 total 10.01 last 142.42) - /unlink 2673 (time 1790777930.51 total 20.03 last 124.50) - /unlink 4033 (time 1790777940.52 total 30.04 last 135.93) total: 4500 /unlink in 33.51 seconds: 134.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 802 (time 1790777958.90 total 10.00 last 80.17) - open/close 1611 (time 1790777968.91 total 20.02 last 80.79) - open/close 2481 (time 1790777978.92 total 30.02 last 86.96) - open/close 3236 (time 1790777988.93 total 40.03 last 75.42) - open/close 4188 (time 1790777998.93 total 50.04 last 95.17) total: 4500 open/close in 54.13 seconds: 83.13 ops/second - /unlink 1409 (time 1790778017.87 total 10.01 last 140.78) - /unlink 2688 (time 1790778027.88 total 20.01 last 127.82) - /unlink 4125 (time 1790778037.88 total 30.02 last 143.63) total: 4500 /unlink in 32.64 seconds: 137.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 828 (time 1790778056.17 total 10.00 last 82.78) - open/close 1613 (time 1790778066.18 total 20.01 last 78.46) - open/close 2422 (time 1790778076.18 total 30.02 last 80.84) - open/close 3154 (time 1790778086.18 total 40.02 last 73.20) - open/close 3789 (time 1790778096.19 total 50.02 last 63.47) - open/close 4436 (time 1790778106.20 total 60.03 last 64.65) total: 4500 open/close in 60.72 seconds: 74.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1544 (time 1790778122.18 total 10.01 last 154.31) - /unlink 2691 (time 1790778132.18 total 20.01 last 114.70) - /unlink 4093 (time 1790778142.19 total 30.01 last 140.12) total: 4500 /unlink in 32.97 seconds: 136.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 797 (time 1790778159.97 total 10.01 last 79.66) - open/close 1512 (time 1790778169.97 total 20.01 last 71.48) - open/close 2182 (time 1790778179.98 total 30.01 last 66.97) - open/close 3102 (time 1790778189.98 total 40.02 last 91.97) - open/close 3763 (time 1790778199.99 total 50.03 last 66.03) total: 4500 open/close in 59.59 seconds: 75.52 ops/second - /unlink 1396 (time 1790778224.75 total 10.00 last 139.54) - /unlink 3029 (time 1790778234.75 total 20.01 last 163.26) total: 4500 /unlink in 29.09 seconds: 154.69 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 750 (time 1790778259.01 total 10.01 last 74.96) - open/close 1518 (time 1790778269.03 total 20.03 last 76.64) - open/close 2329 (time 1790778279.03 total 30.03 last 81.05) - open/close 3080 (time 1790778289.04 total 40.04 last 75.05) - open/close 3836 (time 1790778299.04 total 50.04 last 75.58) total: 4500 open/close in 57.94 seconds: 77.66 ops/second - /unlink 1383 (time 1790778321.77 total 10.01 last 138.17) - /unlink 2920 (time 1790778331.78 total 20.01 last 153.63) total: 4500 /unlink in 29.89 seconds: 150.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 841 (time 1790778357.23 total 10.00 last 84.09) - open/close 1583 (time 1790778367.23 total 20.01 last 74.16) - open/close 2292 (time 1790778377.23 total 30.01 last 70.89) - open/close 3063 (time 1790778387.23 total 40.01 last 77.10) - open/close 3814 (time 1790778397.24 total 50.01 last 75.09) - open/close 4487 (time 1790778407.24 total 60.02 last 67.25) total: 4500 open/close in 60.20 seconds: 74.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1567 (time 1790778422.81 total 10.01 last 156.56) - /unlink 3105 (time 1790778432.81 total 20.01 last 153.78) - /unlink 4483 (time 1790778442.81 total 30.01 last 137.78) total: 4500 /unlink in 30.26 seconds: 148.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 804 (time 1790778457.77 total 10.01 last 80.34) - open/close 1528 (time 1790778467.78 total 20.02 last 72.33) - open/close 2388 (time 1790778477.79 total 30.03 last 85.92) - open/close 3243 (time 1790778487.79 total 40.03 last 85.49) - open/close 3954 (time 1790778497.80 total 50.04 last 71.05) total: 4500 open/close in 58.35 seconds: 77.12 ops/second - /unlink 1522 (time 1790778521.88 total 10.01 last 152.10) - /unlink 2825 (time 1790778531.89 total 20.01 last 130.22) - /unlink 4437 (time 1790778541.89 total 30.02 last 161.10) total: 4500 /unlink in 30.96 seconds: 145.37 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 732 (time 1790778558.25 total 10.00 last 73.17) - open/close 1626 (time 1790778568.26 total 20.01 last 89.30) - open/close 2405 (time 1790778578.27 total 30.02 last 77.83) - open/close 3206 (time 1790778588.30 total 40.05 last 79.88) - open/close 3930 (time 1790778598.31 total 50.06 last 72.34) total: 4500 open/close in 58.04 seconds: 77.53 ops/second - /unlink 1602 (time 1790778621.96 total 10.01 last 160.09) - /unlink 2925 (time 1790778631.96 total 20.01 last 132.29) - /unlink 4080 (time 1790778641.96 total 30.01 last 115.45) total: 4500 /unlink in 33.72 seconds: 133.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 841 (time 1790778661.53 total 10.01 last 84.05) - open/close 1654 (time 1790778671.54 total 20.01 last 81.23) - open/close 2366 (time 1790778681.54 total 30.02 last 71.19) - open/close 2935 (time 1790778691.54 total 40.02 last 56.89) - open/close 3859 (time 1790778701.55 total 50.02 last 92.34) total: 4500 open/close in 57.88 seconds: 77.74 ops/second - /unlink 1414 (time 1790778724.04 total 10.00 last 141.39) - /unlink 2664 (time 1790778734.05 total 20.01 last 124.91) - /unlink 3986 (time 1790778744.05 total 30.01 last 132.16) total: 4500 /unlink in 34.23 seconds: 131.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 575 (time 1790778764.09 total 10.01 last 57.46) - open/close 1237 (time 1790778774.10 total 20.01 last 66.19) - open/close 1918 (time 1790778784.10 total 30.01 last 68.07) - open/close 2601 (time 1790778794.11 total 40.03 last 68.20) - open/close 3288 (time 1790778804.13 total 50.04 last 68.61) - open/close 3872 (time 1790778814.14 total 60.05 last 58.35) total: 4500 open/close in 69.40 seconds: 64.84 ops/second - /unlink 1544 (time 1790778838.32 total 10.00 last 154.33) - /unlink 3031 (time 1790778848.32 total 20.01 last 148.62) total: 4500 /unlink in 28.39 seconds: 158.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 656 (time 1790778872.23 total 10.01 last 65.56) - open/close 1403 (time 1790778882.24 total 20.02 last 74.63) - open/close 2152 (time 1790778892.24 total 30.02 last 74.86) - open/close 2810 (time 1790778902.25 total 40.03 last 65.75) - open/close 3584 (time 1790778912.25 total 50.03 last 77.39) - open/close 4387 (time 1790778922.25 total 60.03 last 80.30) total: 4500 open/close in 61.44 seconds: 73.24 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1399 (time 1790778939.25 total 10.02 last 139.58) - /unlink 2724 (time 1790778949.25 total 20.03 last 132.45) - /unlink 4068 (time 1790778959.26 total 30.03 last 134.34) total: 4500 /unlink in 33.84 seconds: 132.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 749 (time 1790778978.23 total 10.00 last 74.88) - open/close 1480 (time 1790778988.24 total 20.01 last 73.03) - open/close 2306 (time 1790778998.24 total 30.01 last 82.59) - open/close 3085 (time 1790779008.24 total 40.02 last 77.89) - open/close 3844 (time 1790779018.25 total 50.02 last 75.83) total: 4500 open/close in 57.23 seconds: 78.63 ops/second - /unlink 1607 (time 1790779040.03 total 10.00 last 160.67) - /unlink 3165 (time 1790779050.03 total 20.00 last 155.80) total: 4500 /unlink in 29.08 seconds: 154.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 701 (time 1790779073.89 total 10.01 last 70.00) - open/close 1480 (time 1790779083.90 total 20.02 last 77.84) - open/close 2334 (time 1790779093.90 total 30.02 last 85.38) - open/close 3144 (time 1790779103.90 total 40.03 last 80.98) - open/close 3863 (time 1790779113.90 total 50.03 last 71.88) total: 4500 open/close in 59.25 seconds: 75.95 ops/second - /unlink 1482 (time 1790779138.70 total 10.00 last 148.15) - /unlink 2860 (time 1790779148.70 total 20.00 last 137.78) - /unlink 4451 (time 1790779158.71 total 30.01 last 158.99) total: 4500 /unlink in 30.51 seconds: 147.51 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11358: 95469 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg104-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg104-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg104-server stop mds service on oleg104-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg104-server stop mds service on oleg104-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg104-server unloading modules via unload_modules_local on: 'oleg104-server' oleg104-server: oleg104-server.virtnet: executing unload_modules_local oleg104-server: kmemleak disabled modules unloaded.