== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 15:39:06 (1787859546) Stopping clients: oleg149-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg149-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg149-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg149-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg149-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg149-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg149-server oleg149-server: oleg149-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1049: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg149-server' oleg149-server: oleg149-server.virtnet: executing load_modules_local oleg149-server: Loading modules from /home/green/git/lustre-release/lustre oleg149-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1049: echo: write error: Device or resource busy oleg149-server: detected 4 online CPUs by sysfs oleg149-server: MODOPTS_LIBCFS= oleg149-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg149-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg149-server: oleg149-server.virtnet: executing set_default_debug -1 all pdsh@oleg149-client: oleg149-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg149-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg149-server: oleg149-server.virtnet: executing set_default_debug -1 all pdsh@oleg149-client: oleg149-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg149-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg149-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg149-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg149-server: oleg149-server.virtnet: executing set_default_debug -1 all pdsh@oleg149-client: oleg149-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg149-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg149-client.virtnet: -o user_xattr,flock 192.168.201.149@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 912 (time 1787859738.06 total 10.00 last 91.19) - open/close 1676 (time 1787859748.08 total 20.03 last 76.19) - open/close 2641 (time 1787859758.09 total 30.03 last 96.49) - open/close 3440 (time 1787859768.09 total 40.03 last 79.86) - open/close 4217 (time 1787859778.09 total 50.04 last 77.68) total: 4500 open/close in 53.43 seconds: 84.23 ops/second - /unlink 1660 (time 1787859795.88 total 10.00 last 165.98) - /unlink 3060 (time 1787859805.89 total 20.01 last 139.90) - /unlink 4331 (time 1787859815.90 total 30.01 last 127.04) total: 4500 /unlink in 31.39 seconds: 143.36 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 866 (time 1787859832.17 total 10.01 last 86.53) - open/close 1576 (time 1787859842.17 total 20.01 last 70.99) - open/close 2421 (time 1787859852.19 total 30.03 last 84.35) - open/close 3176 (time 1787859862.20 total 40.03 last 75.45) - open/close 3890 (time 1787859872.21 total 50.05 last 71.31) total: 4500 open/close in 57.33 seconds: 78.50 ops/second - /unlink 981 (time 1787859895.68 total 10.01 last 98.05) - /unlink 2563 (time 1787859905.68 total 20.01 last 158.16) - /unlink 4077 (time 1787859915.69 total 30.01 last 151.32) total: 4500 /unlink in 33.19 seconds: 135.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 800 (time 1787859933.89 total 10.00 last 79.97) - open/close 1678 (time 1787859943.90 total 20.01 last 87.73) - open/close 2483 (time 1787859953.90 total 30.01 last 80.47) - open/close 3344 (time 1787859963.91 total 40.02 last 86.07) - open/close 4096 (time 1787859973.92 total 50.03 last 75.08) total: 4500 open/close in 55.48 seconds: 81.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1576 (time 1787859993.55 total 10.00 last 157.60) - /unlink 3025 (time 1787860003.55 total 20.00 last 144.87) total: 4500 /unlink in 29.94 seconds: 150.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 831 (time 1787860028.34 total 10.01 last 83.03) - open/close 1769 (time 1787860038.34 total 20.01 last 93.76) - open/close 2449 (time 1787860048.35 total 30.02 last 67.97) - open/close 3081 (time 1787860058.35 total 40.02 last 63.16) - open/close 3835 (time 1787860068.36 total 50.03 last 75.37) total: 4500 open/close in 57.79 seconds: 77.87 ops/second - /unlink 1616 (time 1787860091.14 total 10.02 last 161.24) - /unlink 2994 (time 1787860101.15 total 20.03 last 137.75) total: 4500 /unlink in 29.66 seconds: 151.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 725 (time 1787860126.04 total 10.05 last 72.15) - open/close 1556 (time 1787860136.04 total 20.05 last 83.08) - open/close 2361 (time 1787860146.08 total 30.09 last 80.20) - open/close 3210 (time 1787860156.09 total 40.09 last 84.87) - open/close 4019 (time 1787860166.09 total 50.09 last 80.89) total: 4500 open/close in 56.88 seconds: 79.11 ops/second - /unlink 1692 (time 1787860188.08 total 10.01 last 169.00) - /unlink 3109 (time 1787860198.09 total 20.02 last 141.64) - /unlink 4470 (time 1787860208.09 total 30.02 last 136.05) total: 4500 /unlink in 30.31 seconds: 148.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 824 (time 1787860222.82 total 10.01 last 82.36) - open/close 1651 (time 1787860232.82 total 20.01 last 82.69) - open/close 2427 (time 1787860242.84 total 30.02 last 77.50) - open/close 3205 (time 1787860252.84 total 40.02 last 77.80) - open/close 3771 (time 1787860262.84 total 50.02 last 56.59) total: 4500 open/close in 59.11 seconds: 76.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1790 (time 1787860286.95 total 10.02 last 178.70) - /unlink 3243 (time 1787860296.95 total 20.02 last 145.25) total: 4500 /unlink in 26.35 seconds: 170.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 788 (time 1787860317.62 total 10.01 last 78.72) - open/close 1617 (time 1787860327.65 total 20.04 last 82.65) - open/close 2515 (time 1787860337.68 total 30.06 last 89.59) - open/close 3309 (time 1787860347.68 total 40.07 last 79.36) - open/close 4007 (time 1787860357.72 total 50.10 last 69.57) total: 4500 open/close in 56.29 seconds: 79.94 ops/second - /unlink 1471 (time 1787860378.56 total 10.01 last 147.02) - /unlink 2614 (time 1787860388.57 total 20.01 last 114.24) - /unlink 4268 (time 1787860398.57 total 30.01 last 165.34) total: 4500 /unlink in 31.68 seconds: 142.06 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 600 (time 1787860415.23 total 10.01 last 59.97) - open/close 1367 (time 1787860425.24 total 20.01 last 76.68) - open/close 2235 (time 1787860435.24 total 30.01 last 86.80) - open/close 3193 (time 1787860445.24 total 40.02 last 95.73) - open/close 4156 (time 1787860455.25 total 50.03 last 96.20) total: 4500 open/close in 54.48 seconds: 82.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1565 (time 1787860475.29 total 10.01 last 156.41) - /unlink 3184 (time 1787860485.30 total 20.01 last 161.85) total: 4500 /unlink in 27.99 seconds: 160.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 783 (time 1787860508.90 total 10.01 last 78.26) - open/close 1581 (time 1787860518.90 total 20.01 last 79.76) - open/close 2250 (time 1787860528.90 total 30.01 last 66.87) - open/close 3000 (time 1787860538.91 total 40.02 last 74.97) - open/close 3877 (time 1787860548.92 total 50.03 last 87.63) total: 4500 open/close in 57.72 seconds: 77.96 ops/second - /unlink 1646 (time 1787860571.39 total 10.01 last 164.50) - /unlink 2934 (time 1787860581.39 total 20.01 last 128.75) - /unlink 4152 (time 1787860591.39 total 30.01 last 121.78) total: 4500 /unlink in 32.13 seconds: 140.06 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 765 (time 1787860607.99 total 10.01 last 76.41) - open/close 1539 (time 1787860617.99 total 20.02 last 77.36) - open/close 2383 (time 1787860627.99 total 30.02 last 84.38) - open/close 3090 (time 1787860638.00 total 40.02 last 70.66) - open/close 3991 (time 1787860648.00 total 50.03 last 90.08) total: 4500 open/close in 56.56 seconds: 79.57 ops/second - /unlink 1637 (time 1787860669.22 total 10.00 last 163.67) - /unlink 3131 (time 1787860679.22 total 20.00 last 149.36) total: 4500 /unlink in 27.78 seconds: 161.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 766 (time 1787860702.19 total 10.02 last 76.47) - open/close 1545 (time 1787860712.20 total 20.02 last 77.86) - open/close 2473 (time 1787860722.20 total 30.02 last 92.79) - open/close 3198 (time 1787860732.21 total 40.03 last 72.45) - open/close 4001 (time 1787860742.21 total 50.04 last 80.24) total: 4500 open/close in 55.80 seconds: 80.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1638 (time 1787860762.14 total 10.00 last 163.72) - /unlink 3123 (time 1787860772.14 total 20.01 last 148.49) total: 4500 /unlink in 29.33 seconds: 153.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 869 (time 1787860796.54 total 10.01 last 86.86) - open/close 1630 (time 1787860806.55 total 20.01 last 76.09) - open/close 2500 (time 1787860816.56 total 30.02 last 86.85) - open/close 3317 (time 1787860826.57 total 40.03 last 81.65) - open/close 3944 (time 1787860836.57 total 50.03 last 62.67) total: 4500 open/close in 58.72 seconds: 76.64 ops/second - /unlink 1500 (time 1787860860.12 total 10.00 last 149.94) - /unlink 3041 (time 1787860870.12 total 20.01 last 154.08) - /unlink 4391 (time 1787860880.13 total 30.01 last 134.95) total: 4500 /unlink in 30.89 seconds: 145.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 777 (time 1787860894.99 total 10.01 last 77.64) - open/close 1524 (time 1787860904.99 total 20.01 last 74.67) - open/close 2248 (time 1787860914.99 total 30.01 last 72.38) - open/close 2998 (time 1787860925.00 total 40.02 last 74.94) - open/close 3758 (time 1787860935.01 total 50.03 last 75.95) total: 4500 open/close in 58.50 seconds: 76.93 ops/second - /unlink 1708 (time 1787860957.91 total 10.00 last 170.72) - /unlink 3276 (time 1787860967.91 total 20.01 last 156.79) total: 4500 /unlink in 28.06 seconds: 160.38 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 731 (time 1787860991.71 total 10.02 last 72.95) - open/close 1409 (time 1787861001.73 total 20.04 last 67.65) - open/close 2223 (time 1787861011.74 total 30.05 last 81.37) - open/close 3118 (time 1787861021.74 total 40.05 last 89.43) - open/close 4070 (time 1787861031.75 total 50.06 last 95.13) total: 4500 open/close in 54.71 seconds: 82.24 ops/second - /unlink 1645 (time 1787861050.57 total 10.00 last 164.45) - /unlink 3246 (time 1787861060.57 total 20.00 last 160.09) total: 4500 /unlink in 27.92 seconds: 161.18 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 864 (time 1787861083.33 total 10.00 last 86.40) - open/close 1688 (time 1787861093.33 total 20.00 last 82.39) - open/close 2588 (time 1787861103.34 total 30.00 last 89.97) - open/close 3345 (time 1787861113.35 total 40.02 last 75.59) - open/close 4141 (time 1787861123.36 total 50.03 last 79.50) total: 4500 open/close in 54.78 seconds: 82.15 ops/second - /unlink 1733 (time 1787861143.17 total 10.00 last 173.24) - /unlink 3100 (time 1787861153.17 total 20.01 last 136.67) total: 4500 /unlink in 28.98 seconds: 155.27 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 653 (time 1787861177.88 total 10.00 last 65.27) - open/close 1433 (time 1787861187.89 total 20.01 last 77.92) - open/close 2339 (time 1787861197.89 total 30.02 last 90.59) - open/close 3141 (time 1787861207.89 total 40.02 last 80.18) - open/close 3839 (time 1787861217.91 total 50.03 last 69.70) total: 4500 open/close in 58.39 seconds: 77.07 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1321 (time 1787861241.08 total 10.00 last 132.08) - /unlink 2797 (time 1787861251.08 total 20.00 last 147.60) - /unlink 4321 (time 1787861261.10 total 30.01 last 152.23) total: 4500 /unlink in 31.62 seconds: 142.30 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 862 (time 1787861277.31 total 10.01 last 86.16) - open/close 1613 (time 1787861287.34 total 20.04 last 74.81) - open/close 2308 (time 1787861297.36 total 30.05 last 69.42) - open/close 3098 (time 1787861307.37 total 40.07 last 78.85) - open/close 3980 (time 1787861317.39 total 50.09 last 88.04) total: 4500 open/close in 56.61 seconds: 79.48 ops/second - /unlink 1452 (time 1787861337.71 total 10.01 last 145.12) - /unlink 3076 (time 1787861347.71 total 20.01 last 162.38) total: 4500 /unlink in 28.58 seconds: 157.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 793 (time 1787861371.64 total 10.00 last 79.26) - open/close 1719 (time 1787861381.65 total 20.01 last 92.58) - open/close 2422 (time 1787861391.65 total 30.01 last 70.25) - open/close 3334 (time 1787861401.66 total 40.02 last 91.15) - open/close 4097 (time 1787861411.67 total 50.03 last 76.24) total: 4500 open/close in 55.25 seconds: 81.44 ops/second - /unlink 1481 (time 1787861432.36 total 10.00 last 148.07) - /unlink 2862 (time 1787861442.36 total 20.00 last 138.06) - /unlink 4150 (time 1787861452.36 total 30.01 last 128.77) total: 4500 /unlink in 32.35 seconds: 139.09 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11369: 95242 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg149-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg149-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg149-server stop mds service on oleg149-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg149-server stop mds service on oleg149-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg149-server unloading modules via unload_modules_local on: 'oleg149-server' oleg149-server: oleg149-server.virtnet: executing unload_modules_local oleg149-server: modules unloaded.