== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 00:45:51 (1789620351) Stopping clients: oleg645-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg645-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg645-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg645-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg645-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg645-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg645-server oleg645-server: oleg645-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg645-server' oleg645-server: oleg645-server.virtnet: executing load_modules_local oleg645-server: kmemleak disabled oleg645-server: Loading modules from /home/green/git/lustre-release/lustre oleg645-server: detected 4 online CPUs by sysfs oleg645-server: MODOPTS_LIBCFS= oleg645-server: Force libcfs to create 2 CPU partitions oleg645-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg645-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg645-server: oleg645-server.virtnet: executing set_default_debug -1 all pdsh@oleg645-client: oleg645-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg645-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg645-server: oleg645-server.virtnet: executing set_default_debug -1 all pdsh@oleg645-client: oleg645-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg645-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg645-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg645-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg645-server: oleg645-server.virtnet: executing set_default_debug -1 all pdsh@oleg645-client: oleg645-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg645-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg645-client.virtnet: -o user_xattr,flock 192.168.206.145@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 797 (time 1789620560.44 total 10.02 last 79.52) - open/close 1585 (time 1789620570.45 total 20.02 last 78.78) - open/close 2345 (time 1789620580.46 total 30.04 last 75.91) - open/close 3143 (time 1789620590.46 total 40.04 last 79.76) - open/close 3915 (time 1789620600.47 total 50.04 last 77.17) total: 4500 open/close in 58.66 seconds: 76.72 ops/second - /unlink 1236 (time 1789620623.48 total 10.00 last 123.60) - /unlink 2557 (time 1789620633.49 total 20.01 last 132.00) - /unlink 3795 (time 1789620643.49 total 30.01 last 123.75) total: 4500 /unlink in 35.46 seconds: 126.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 676 (time 1789620663.32 total 10.02 last 67.44) - open/close 1447 (time 1789620673.34 total 20.04 last 77.00) - open/close 2137 (time 1789620683.34 total 30.04 last 68.97) - open/close 2793 (time 1789620693.35 total 40.05 last 65.53) - open/close 3504 (time 1789620703.37 total 50.07 last 70.99) - open/close 4238 (time 1789620713.37 total 60.07 last 73.37) total: 4500 open/close in 62.96 seconds: 71.47 ops/second - /unlink 1506 (time 1789620730.26 total 10.01 last 150.52) - /unlink 2941 (time 1789620740.26 total 20.01 last 143.48) - /unlink 4497 (time 1789620750.26 total 30.01 last 155.53) total: 4500 /unlink in 30.03 seconds: 149.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 751 (time 1789620764.10 total 10.00 last 75.10) - open/close 1539 (time 1789620774.11 total 20.00 last 78.79) - open/close 2306 (time 1789620784.12 total 30.02 last 76.56) - open/close 3041 (time 1789620794.14 total 40.04 last 73.38) - open/close 3793 (time 1789620804.19 total 50.08 last 74.84) - open/close 4435 (time 1789620814.19 total 60.08 last 64.20) total: 4500 open/close in 61.24 seconds: 73.49 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1546 (time 1789620829.81 total 10.01 last 154.50) - /unlink 2998 (time 1789620839.82 total 20.01 last 145.13) - /unlink 4208 (time 1789620849.82 total 30.02 last 120.94) total: 4500 /unlink in 32.99 seconds: 136.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 722 (time 1789620868.85 total 10.02 last 72.08) - open/close 1454 (time 1789620878.85 total 20.02 last 73.16) - open/close 2209 (time 1789620888.86 total 30.03 last 75.43) - open/close 2977 (time 1789620898.87 total 40.04 last 76.74) - open/close 3757 (time 1789620908.88 total 50.05 last 77.95) total: 4500 open/close in 59.36 seconds: 75.81 ops/second - /unlink 1411 (time 1789620932.24 total 10.00 last 141.07) - /unlink 2691 (time 1789620942.24 total 20.00 last 127.99) - /unlink 4032 (time 1789620952.24 total 30.01 last 134.05) total: 4500 /unlink in 33.24 seconds: 135.37 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 846 (time 1789620970.29 total 10.01 last 84.53) - open/close 1588 (time 1789620980.29 total 20.01 last 74.18) - open/close 2359 (time 1789620990.30 total 30.01 last 77.09) - open/close 3075 (time 1789621000.31 total 40.03 last 71.49) - open/close 3830 (time 1789621010.32 total 50.04 last 75.40) total: 4500 open/close in 58.53 seconds: 76.88 ops/second - /unlink 1355 (time 1789621033.14 total 10.00 last 135.48) - /unlink 2698 (time 1789621043.14 total 20.00 last 134.26) - /unlink 4109 (time 1789621053.14 total 30.00 last 141.10) total: 4500 /unlink in 32.49 seconds: 138.52 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 765 (time 1789621070.56 total 10.01 last 76.44) - open/close 1578 (time 1789621080.56 total 20.02 last 81.23) - open/close 2379 (time 1789621090.57 total 30.02 last 80.05) - open/close 3192 (time 1789621100.57 total 40.02 last 81.30) - open/close 3953 (time 1789621110.57 total 50.02 last 76.09) total: 4500 open/close in 56.90 seconds: 79.09 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1419 (time 1789621131.74 total 10.00 last 141.88) - /unlink 2768 (time 1789621141.74 total 20.01 last 134.84) - /unlink 4316 (time 1789621151.74 total 30.01 last 154.79) total: 4500 /unlink in 31.22 seconds: 144.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 848 (time 1789621167.08 total 10.01 last 84.76) - open/close 1638 (time 1789621177.08 total 20.01 last 78.97) - open/close 2466 (time 1789621187.10 total 30.02 last 82.69) - open/close 3275 (time 1789621197.10 total 40.03 last 80.85) - open/close 4053 (time 1789621207.11 total 50.03 last 77.76) total: 4500 open/close in 55.55 seconds: 81.00 ops/second - /unlink 1588 (time 1789621226.53 total 10.00 last 158.76) - /unlink 3151 (time 1789621236.53 total 20.01 last 156.25) total: 4500 /unlink in 28.68 seconds: 156.89 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 814 (time 1789621259.11 total 10.01 last 81.30) - open/close 1640 (time 1789621269.12 total 20.03 last 82.46) - open/close 2414 (time 1789621279.13 total 30.04 last 77.31) - open/close 3289 (time 1789621289.14 total 40.05 last 87.43) - open/close 4111 (time 1789621299.16 total 50.07 last 82.05) total: 4500 open/close in 54.34 seconds: 82.81 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1591 (time 1789621317.38 total 10.01 last 158.97) - /unlink 3063 (time 1789621327.38 total 20.01 last 147.17) total: 4500 /unlink in 29.81 seconds: 150.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 814 (time 1789621351.33 total 10.00 last 81.37) - open/close 1597 (time 1789621361.35 total 20.02 last 78.14) - open/close 2376 (time 1789621371.36 total 30.03 last 77.86) - open/close 3178 (time 1789621381.36 total 40.04 last 80.15) - open/close 3987 (time 1789621391.37 total 50.04 last 80.85) total: 4500 open/close in 56.57 seconds: 79.55 ops/second - /unlink 1553 (time 1789621411.73 total 10.01 last 155.09) - /unlink 2989 (time 1789621421.73 total 20.01 last 143.59) total: 4500 /unlink in 29.84 seconds: 150.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 837 (time 1789621445.28 total 10.00 last 83.67) - open/close 1636 (time 1789621455.30 total 20.01 last 79.81) - open/close 2429 (time 1789621465.30 total 30.02 last 79.29) - open/close 3257 (time 1789621475.30 total 40.02 last 82.78) - open/close 4097 (time 1789621485.30 total 50.02 last 83.96) total: 4500 open/close in 55.13 seconds: 81.63 ops/second - /unlink 1557 (time 1789621504.02 total 10.00 last 155.70) - /unlink 2884 (time 1789621514.02 total 20.00 last 132.64) - /unlink 4426 (time 1789621524.03 total 30.01 last 154.12) total: 4500 /unlink in 30.59 seconds: 147.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 847 (time 1789621538.34 total 10.00 last 84.67) - open/close 1701 (time 1789621548.35 total 20.01 last 85.37) - open/close 2536 (time 1789621558.37 total 30.03 last 83.29) - open/close 3370 (time 1789621568.38 total 40.04 last 83.35) - open/close 4226 (time 1789621578.38 total 50.04 last 85.56) total: 4500 open/close in 53.17 seconds: 84.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1644 (time 1789621594.72 total 10.00 last 164.35) - /unlink 3223 (time 1789621604.72 total 20.00 last 157.88) total: 4500 /unlink in 28.71 seconds: 156.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 799 (time 1789621627.81 total 10.01 last 79.84) - open/close 1561 (time 1789621637.81 total 20.01 last 76.20) - open/close 2251 (time 1789621647.83 total 30.02 last 68.91) - open/close 3081 (time 1789621657.83 total 40.03 last 82.93) - open/close 3934 (time 1789621667.85 total 50.05 last 85.13) total: 4500 open/close in 57.24 seconds: 78.61 ops/second - /unlink 1649 (time 1789621688.49 total 10.01 last 164.79) - /unlink 3230 (time 1789621698.49 total 20.01 last 158.07) total: 4500 /unlink in 28.79 seconds: 156.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 779 (time 1789621721.22 total 10.00 last 77.89) - open/close 1598 (time 1789621731.22 total 20.00 last 81.90) - open/close 2464 (time 1789621741.23 total 30.02 last 86.46) - open/close 3306 (time 1789621751.25 total 40.03 last 84.09) - open/close 4142 (time 1789621761.26 total 50.04 last 83.50) total: 4500 open/close in 54.39 seconds: 82.73 ops/second - /unlink 1610 (time 1789621779.02 total 10.00 last 160.93) - /unlink 3165 (time 1789621789.02 total 20.01 last 155.44) total: 4500 /unlink in 27.98 seconds: 160.83 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 810 (time 1789621810.46 total 10.00 last 80.99) - open/close 1587 (time 1789621820.46 total 20.01 last 77.65) - open/close 2416 (time 1789621830.46 total 30.01 last 82.88) - open/close 3196 (time 1789621840.47 total 40.01 last 77.97) - open/close 4029 (time 1789621850.48 total 50.02 last 83.22) total: 4500 open/close in 55.53 seconds: 81.03 ops/second - /unlink 1564 (time 1789621869.19 total 10.00 last 156.39) - /unlink 3128 (time 1789621879.20 total 20.01 last 156.29) total: 4500 /unlink in 28.66 seconds: 156.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 827 (time 1789621901.57 total 10.01 last 82.64) - open/close 1676 (time 1789621911.58 total 20.02 last 84.82) - open/close 2537 (time 1789621921.59 total 30.03 last 86.00) - open/close 3398 (time 1789621931.60 total 40.04 last 86.03) - open/close 4259 (time 1789621941.60 total 50.04 last 86.08) total: 4500 open/close in 52.76 seconds: 85.30 ops/second - /unlink 1674 (time 1789621957.94 total 10.00 last 167.36) - /unlink 3377 (time 1789621967.94 total 20.00 last 170.30) total: 4500 /unlink in 27.00 seconds: 166.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 813 (time 1789621988.54 total 10.00 last 81.30) - open/close 1656 (time 1789621998.55 total 20.02 last 84.17) - open/close 2533 (time 1789622008.56 total 30.02 last 87.64) - open/close 3417 (time 1789622018.57 total 40.03 last 88.30) - open/close 4265 (time 1789622028.58 total 50.04 last 84.74) total: 4500 open/close in 52.84 seconds: 85.17 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1668 (time 1789622044.91 total 10.00 last 166.75) - /unlink 3251 (time 1789622054.91 total 20.01 last 158.26) total: 4500 /unlink in 27.41 seconds: 164.19 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 790 (time 1789622076.67 total 10.01 last 78.89) - open/close 1522 (time 1789622086.68 total 20.03 last 73.10) - open/close 2281 (time 1789622096.68 total 30.03 last 75.90) - open/close 3112 (time 1789622106.68 total 40.03 last 83.07) - open/close 3960 (time 1789622116.69 total 50.04 last 84.73) total: 4500 open/close in 56.97 seconds: 78.98 ops/second - /unlink 1569 (time 1789622137.15 total 10.00 last 156.89) - /unlink 3111 (time 1789622147.16 total 20.01 last 154.02) total: 4500 /unlink in 28.93 seconds: 155.55 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 828 (time 1789622169.40 total 10.01 last 82.70) - open/close 1570 (time 1789622179.41 total 20.02 last 74.16) - open/close 2420 (time 1789622189.41 total 30.03 last 84.93) - open/close 3214 (time 1789622199.42 total 40.03 last 79.37) - open/close 4080 (time 1789622209.42 total 50.03 last 86.57) total: 4500 open/close in 55.19 seconds: 81.53 ops/second - /unlink 1545 (time 1789622228.37 total 10.00 last 154.45) - /unlink 2990 (time 1789622238.37 total 20.01 last 144.43) - /unlink 4456 (time 1789622248.38 total 30.01 last 146.55) total: 4500 /unlink in 30.39 seconds: 148.07 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11350: 95106 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg645-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg645-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg645-server stop mds service on oleg645-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg645-server stop mds service on oleg645-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg645-server unloading modules via unload_modules_local on: 'oleg645-server' oleg645-server: oleg645-server.virtnet: executing unload_modules_local oleg645-server: kmemleak disabled modules unloaded.