== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 00:02:18 (1789185738) Stopping clients: oleg131-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg131-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg131-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg131-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg131-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg131-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg131-server oleg131-server: oleg131-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg131-server' oleg131-server: oleg131-server.virtnet: executing load_modules_local oleg131-server: kmemleak disabled oleg131-server: Loading modules from /home/green/git/lustre-release/lustre oleg131-server: detected 4 online CPUs by sysfs oleg131-server: MODOPTS_LIBCFS= oleg131-server: Force libcfs to create 2 CPU partitions oleg131-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg131-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg131-server: oleg131-server.virtnet: executing set_default_debug -1 all pdsh@oleg131-client: oleg131-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg131-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg131-server: oleg131-server.virtnet: executing set_default_debug -1 all pdsh@oleg131-client: oleg131-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg131-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg131-server: oleg131-server.virtnet: executing set_default_debug -1 all pdsh@oleg131-client: oleg131-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg131-client.virtnet: -o user_xattr,flock 192.168.201.131@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 810 (time 1789185945.10 total 10.01 last 80.95) - open/close 1546 (time 1789185955.11 total 20.01 last 73.56) - open/close 2247 (time 1789185965.13 total 30.03 last 69.96) - open/close 3012 (time 1789185975.13 total 40.04 last 76.48) - open/close 3700 (time 1789185985.13 total 50.04 last 68.76) - open/close 4410 (time 1789185995.14 total 60.05 last 70.95) total: 4500 open/close in 61.20 seconds: 73.53 ops/second - /unlink 1366 (time 1789186010.79 total 10.01 last 136.51) - /unlink 2705 (time 1789186020.79 total 20.01 last 133.87) - /unlink 4100 (time 1789186030.80 total 30.01 last 139.44) total: 4500 /unlink in 32.64 seconds: 137.87 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 785 (time 1789186048.26 total 10.00 last 78.46) - open/close 1478 (time 1789186058.26 total 20.01 last 69.28) - open/close 2198 (time 1789186068.27 total 30.02 last 71.88) - open/close 2982 (time 1789186078.28 total 40.03 last 78.35) - open/close 3800 (time 1789186088.28 total 50.03 last 81.79) total: 4500 open/close in 59.32 seconds: 75.86 ops/second - /unlink 1421 (time 1789186111.81 total 10.00 last 142.10) - /unlink 2670 (time 1789186121.81 total 20.00 last 124.87) - /unlink 4017 (time 1789186131.81 total 30.01 last 134.66) total: 4500 /unlink in 33.29 seconds: 135.18 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 768 (time 1789186150.23 total 10.00 last 76.76) - open/close 1507 (time 1789186160.24 total 20.01 last 73.88) - open/close 2265 (time 1789186170.25 total 30.02 last 75.71) - open/close 3006 (time 1789186180.25 total 40.02 last 74.08) - open/close 3731 (time 1789186190.25 total 50.02 last 72.50) total: 4500 open/close in 59.80 seconds: 75.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1573 (time 1789186213.86 total 10.01 last 157.16) - /unlink 3061 (time 1789186223.86 total 20.01 last 148.77) total: 4500 /unlink in 29.74 seconds: 151.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 717 (time 1789186248.03 total 10.00 last 71.69) - open/close 1488 (time 1789186258.04 total 20.01 last 77.07) - open/close 2173 (time 1789186268.04 total 30.01 last 68.44) - open/close 2959 (time 1789186278.05 total 40.02 last 78.57) - open/close 3624 (time 1789186288.07 total 50.03 last 66.38) - open/close 4264 (time 1789186298.07 total 60.04 last 63.99) total: 4500 open/close in 63.65 seconds: 70.70 ops/second - /unlink 1311 (time 1789186316.54 total 10.00 last 131.08) - /unlink 2790 (time 1789186326.54 total 20.00 last 147.86) - /unlink 4045 (time 1789186336.54 total 30.01 last 125.48) total: 4500 /unlink in 33.11 seconds: 135.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 627 (time 1789186355.24 total 10.00 last 62.67) - open/close 1321 (time 1789186365.24 total 20.01 last 69.38) - open/close 2020 (time 1789186375.25 total 30.01 last 69.87) - open/close 2745 (time 1789186385.28 total 40.04 last 72.27) - open/close 3629 (time 1789186395.29 total 50.05 last 88.34) - open/close 4352 (time 1789186405.29 total 60.05 last 72.29) total: 4500 open/close in 61.65 seconds: 72.99 ops/second - /unlink 1671 (time 1789186421.28 total 10.00 last 167.10) - /unlink 3131 (time 1789186431.30 total 20.01 last 145.81) total: 4500 /unlink in 29.61 seconds: 151.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 843 (time 1789186455.19 total 10.01 last 84.23) - open/close 1732 (time 1789186465.20 total 20.01 last 88.86) - open/close 2682 (time 1789186475.20 total 30.02 last 94.93) - open/close 3544 (time 1789186485.21 total 40.02 last 86.20) - open/close 4366 (time 1789186495.21 total 50.02 last 82.19) total: 4500 open/close in 51.62 seconds: 87.18 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1627 (time 1789186511.10 total 10.00 last 162.63) - /unlink 3165 (time 1789186521.12 total 20.02 last 153.58) total: 4500 /unlink in 28.82 seconds: 156.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 829 (time 1789186543.76 total 10.01 last 82.84) - open/close 1589 (time 1789186553.76 total 20.01 last 75.99) - open/close 2428 (time 1789186563.76 total 30.01 last 83.90) - open/close 3209 (time 1789186573.77 total 40.02 last 77.99) - open/close 4048 (time 1789186583.78 total 50.03 last 83.84) total: 4500 open/close in 54.31 seconds: 82.86 ops/second - /unlink 1515 (time 1789186600.97 total 10.01 last 151.34) - /unlink 3075 (time 1789186610.97 total 20.01 last 155.97) total: 4500 /unlink in 28.54 seconds: 157.68 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 763 (time 1789186634.06 total 10.01 last 76.25) - open/close 1561 (time 1789186644.08 total 20.02 last 79.69) - open/close 2332 (time 1789186654.08 total 30.03 last 77.07) - open/close 3140 (time 1789186664.09 total 40.03 last 80.76) - open/close 4022 (time 1789186674.10 total 50.04 last 88.11) total: 4500 open/close in 55.82 seconds: 80.62 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1891 (time 1789186693.51 total 10.01 last 188.89) - /unlink 3662 (time 1789186703.51 total 20.01 last 177.05) total: 4500 /unlink in 24.72 seconds: 182.05 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 815 (time 1789186722.36 total 10.01 last 81.41) - open/close 1541 (time 1789186732.36 total 20.02 last 72.57) - open/close 2311 (time 1789186742.36 total 30.02 last 76.98) - open/close 3079 (time 1789186752.37 total 40.02 last 76.78) - open/close 4034 (time 1789186762.37 total 50.03 last 95.44) total: 4500 open/close in 55.10 seconds: 81.68 ops/second - /unlink 1567 (time 1789186781.37 total 10.00 last 156.67) - /unlink 3129 (time 1789186791.38 total 20.01 last 156.04) total: 4500 /unlink in 28.73 seconds: 156.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 838 (time 1789186814.11 total 10.01 last 83.72) - open/close 1712 (time 1789186824.11 total 20.02 last 87.35) - open/close 2519 (time 1789186834.15 total 30.05 last 80.40) - open/close 3401 (time 1789186844.16 total 40.07 last 88.06) - open/close 4276 (time 1789186854.18 total 50.08 last 87.39) total: 4500 open/close in 52.62 seconds: 85.52 ops/second - /unlink 1840 (time 1789186869.57 total 10.00 last 183.98) - /unlink 3624 (time 1789186879.57 total 20.00 last 178.37) total: 4500 /unlink in 25.19 seconds: 178.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 923 (time 1789186898.32 total 10.00 last 92.28) - open/close 1871 (time 1789186908.32 total 20.00 last 94.79) - open/close 2716 (time 1789186918.32 total 30.01 last 84.45) - open/close 3643 (time 1789186928.34 total 40.02 last 92.57) total: 4500 open/close in 49.07 seconds: 91.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1812 (time 1789186950.69 total 10.00 last 181.16) - /unlink 3793 (time 1789186960.69 total 20.01 last 198.02) total: 4500 /unlink in 23.84 seconds: 188.78 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 873 (time 1789186978.13 total 10.01 last 87.25) - open/close 1867 (time 1789186988.13 total 20.01 last 99.39) - open/close 2761 (time 1789186998.14 total 30.01 last 89.35) - open/close 3659 (time 1789187008.14 total 40.01 last 89.80) total: 4500 open/close in 48.47 seconds: 92.84 ops/second - /unlink 1735 (time 1789187029.98 total 10.00 last 173.43) - /unlink 3606 (time 1789187039.98 total 20.01 last 187.05) total: 4500 /unlink in 25.30 seconds: 177.87 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 878 (time 1789187058.44 total 10.00 last 87.77) - open/close 1783 (time 1789187068.46 total 20.03 last 90.28) - open/close 2549 (time 1789187078.47 total 30.03 last 76.57) - open/close 3287 (time 1789187088.47 total 40.03 last 73.79) - open/close 4226 (time 1789187098.47 total 50.03 last 93.89) total: 4500 open/close in 52.86 seconds: 85.13 ops/second - /unlink 1686 (time 1789187114.70 total 10.00 last 168.56) - /unlink 3263 (time 1789187124.70 total 20.01 last 157.65) total: 4500 /unlink in 27.18 seconds: 165.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 986 (time 1789187145.30 total 10.02 last 98.39) - open/close 1949 (time 1789187155.30 total 20.03 last 96.24) - open/close 2954 (time 1789187165.31 total 30.03 last 100.47) - open/close 3949 (time 1789187175.32 total 40.04 last 99.41) total: 4500 open/close in 46.04 seconds: 97.75 ops/second - /unlink 1899 (time 1789187194.24 total 10.00 last 189.86) - /unlink 3754 (time 1789187204.24 total 20.01 last 185.45) total: 4500 /unlink in 23.86 seconds: 188.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 1030 (time 1789187221.08 total 10.00 last 102.98) - open/close 1981 (time 1789187231.09 total 20.00 last 95.08) - open/close 2938 (time 1789187241.10 total 30.01 last 95.59) - open/close 3849 (time 1789187251.10 total 40.02 last 91.10) total: 4500 open/close in 47.21 seconds: 95.32 ops/second - /unlink 1934 (time 1789187271.22 total 10.00 last 193.35) - /unlink 3639 (time 1789187281.22 total 20.01 last 170.45) total: 4500 /unlink in 24.70 seconds: 182.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 1001 (time 1789187299.47 total 10.01 last 100.01) - open/close 1954 (time 1789187309.47 total 20.01 last 95.30) - open/close 2916 (time 1789187319.48 total 30.02 last 96.05) - open/close 3857 (time 1789187329.49 total 40.03 last 94.07) total: 4500 open/close in 46.90 seconds: 95.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 2171 (time 1789187348.98 total 10.00 last 217.02) - /unlink 4281 (time 1789187358.98 total 20.01 last 210.95) total: 4500 /unlink in 21.17 seconds: 212.56 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 1055 (time 1789187373.06 total 10.00 last 105.48) - open/close 2091 (time 1789187383.06 total 20.00 last 103.57) - open/close 3053 (time 1789187393.06 total 30.01 last 96.19) - open/close 4006 (time 1789187403.06 total 40.01 last 95.29) total: 4500 open/close in 45.38 seconds: 99.15 ops/second - /unlink 1782 (time 1789187421.92 total 10.00 last 178.20) - /unlink 3495 (time 1789187431.92 total 20.00 last 171.27) total: 4500 /unlink in 25.36 seconds: 177.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 884 (time 1789187450.42 total 10.01 last 88.35) - open/close 1844 (time 1789187460.43 total 20.01 last 95.93) - open/close 2791 (time 1789187470.43 total 30.01 last 94.68) - open/close 3803 (time 1789187480.44 total 40.02 last 101.13) total: 4500 open/close in 46.16 seconds: 97.49 ops/second - /unlink 2165 (time 1789187499.14 total 10.00 last 216.40) - /unlink 4220 (time 1789187509.15 total 20.01 last 205.45) total: 4500 /unlink in 21.40 seconds: 210.32 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11350: 95301 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg131-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg131-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg131-server stop mds service on oleg131-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg131-server stop mds service on oleg131-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg131-server unloading modules via unload_modules_local on: 'oleg131-server' oleg131-server: oleg131-server.virtnet: executing unload_modules_local oleg131-server: kmemleak disabled modules unloaded.