== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 23:27:07 (1788578827) Stopping clients: oleg133-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg133-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg133-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg133-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg133-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg133-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg133-server oleg133-server: oleg133-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg133-server' oleg133-server: oleg133-server.virtnet: executing load_modules_local oleg133-server: kmemleak disabled oleg133-server: Loading modules from /home/green/git/lustre-release/lustre oleg133-server: detected 4 online CPUs by sysfs oleg133-server: MODOPTS_LIBCFS= oleg133-server: Force libcfs to create 2 CPU partitions oleg133-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg133-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg133-server: oleg133-server.virtnet: executing set_default_debug -1 all pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg133-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg133-server: oleg133-server.virtnet: executing set_default_debug -1 all pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg133-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg133-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg133-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg133-server: oleg133-server.virtnet: executing set_default_debug -1 all pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg133-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg133-client.virtnet: -o user_xattr,flock 192.168.201.133@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 943 (time 1788579033.00 total 10.00 last 94.26) - open/close 1724 (time 1788579043.00 total 20.01 last 78.07) - open/close 2580 (time 1788579053.01 total 30.01 last 85.59) - open/close 3495 (time 1788579063.01 total 40.01 last 91.46) - open/close 4218 (time 1788579073.01 total 50.02 last 72.27) total: 4500 open/close in 53.92 seconds: 83.46 ops/second - /unlink 1624 (time 1788579091.75 total 10.00 last 162.39) - /unlink 3184 (time 1788579101.76 total 20.00 last 156.00) total: 4500 /unlink in 28.81 seconds: 156.21 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 976 (time 1788579125.22 total 10.00 last 97.57) - open/close 1874 (time 1788579135.22 total 20.01 last 89.75) - open/close 2778 (time 1788579145.23 total 30.01 last 90.35) - open/close 3643 (time 1788579155.24 total 40.02 last 86.41) total: 4500 open/close in 49.62 seconds: 90.68 ops/second - /unlink 1513 (time 1788579178.75 total 10.00 last 151.29) - /unlink 3298 (time 1788579188.75 total 20.00 last 178.44) total: 4500 /unlink in 27.47 seconds: 163.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 838 (time 1788579210.64 total 10.00 last 83.79) - open/close 1588 (time 1788579220.64 total 20.00 last 74.99) - open/close 2527 (time 1788579230.65 total 30.01 last 93.85) - open/close 3390 (time 1788579240.65 total 40.01 last 86.27) - open/close 4162 (time 1788579250.66 total 50.02 last 77.11) total: 4500 open/close in 54.89 seconds: 81.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1380 (time 1788579270.30 total 10.00 last 137.97) - /unlink 2554 (time 1788579280.30 total 20.00 last 117.39) - /unlink 3922 (time 1788579290.30 total 30.01 last 136.71) total: 4500 /unlink in 34.80 seconds: 129.30 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 663 (time 1788579310.73 total 10.00 last 66.30) - open/close 1364 (time 1788579320.74 total 20.01 last 70.04) - open/close 2018 (time 1788579330.75 total 30.02 last 65.31) - open/close 2721 (time 1788579340.77 total 40.04 last 70.17) - open/close 3518 (time 1788579350.79 total 50.06 last 79.58) - open/close 4162 (time 1788579360.81 total 60.08 last 64.25) total: 4500 open/close in 64.06 seconds: 70.25 ops/second - /unlink 1098 (time 1788579379.77 total 10.00 last 109.77) - /unlink 2236 (time 1788579389.77 total 20.01 last 113.72) - /unlink 3721 (time 1788579399.78 total 30.02 last 148.37) total: 4500 /unlink in 35.00 seconds: 128.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 767 (time 1788579419.23 total 10.01 last 76.62) - open/close 1459 (time 1788579429.23 total 20.01 last 69.18) - open/close 2237 (time 1788579439.25 total 30.03 last 77.68) - open/close 2806 (time 1788579449.27 total 40.05 last 56.77) - open/close 3511 (time 1788579459.28 total 50.06 last 70.43) - open/close 4284 (time 1788579469.29 total 60.07 last 77.20) total: 4500 open/close in 62.46 seconds: 72.04 ops/second - /unlink 1548 (time 1788579486.76 total 10.00 last 154.75) - /unlink 3079 (time 1788579496.76 total 20.01 last 153.06) total: 4500 /unlink in 29.66 seconds: 151.72 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 825 (time 1788579522.26 total 10.02 last 82.35) - open/close 1667 (time 1788579532.27 total 20.03 last 84.10) - open/close 2631 (time 1788579542.28 total 30.03 last 96.36) - open/close 3532 (time 1788579552.28 total 40.04 last 90.02) - open/close 4333 (time 1788579562.29 total 50.04 last 80.09) total: 4500 open/close in 51.83 seconds: 86.82 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1524 (time 1788579577.83 total 10.00 last 152.38) - /unlink 3177 (time 1788579587.84 total 20.01 last 165.22) total: 4500 /unlink in 27.67 seconds: 162.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 842 (time 1788579610.29 total 10.01 last 84.15) - open/close 1738 (time 1788579620.30 total 20.02 last 89.51) - open/close 2468 (time 1788579630.30 total 30.02 last 72.97) - open/close 3158 (time 1788579640.31 total 40.02 last 68.97) - open/close 3833 (time 1788579650.31 total 50.03 last 67.44) - open/close 4471 (time 1788579660.33 total 60.05 last 63.69) total: 4500 open/close in 60.48 seconds: 74.40 ops/second - /unlink 1625 (time 1788579675.92 total 10.00 last 162.45) - /unlink 3076 (time 1788579685.93 total 20.01 last 144.93) total: 4500 /unlink in 29.88 seconds: 150.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 794 (time 1788579711.14 total 10.02 last 79.22) - open/close 1612 (time 1788579721.17 total 20.04 last 81.62) - open/close 2374 (time 1788579731.17 total 30.05 last 76.17) - open/close 3217 (time 1788579741.17 total 40.05 last 84.28) - open/close 4050 (time 1788579751.18 total 50.06 last 83.25) total: 4500 open/close in 55.17 seconds: 81.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1664 (time 1788579770.86 total 10.00 last 166.40) - /unlink 3266 (time 1788579780.86 total 20.00 last 160.19) total: 4500 /unlink in 27.88 seconds: 161.40 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 874 (time 1788579804.05 total 10.00 last 87.38) - open/close 1821 (time 1788579814.06 total 20.00 last 94.68) - open/close 2604 (time 1788579824.07 total 30.01 last 78.24) - open/close 3469 (time 1788579834.07 total 40.01 last 86.48) - open/close 4329 (time 1788579844.07 total 50.02 last 85.98) total: 4500 open/close in 51.99 seconds: 86.56 ops/second - /unlink 1322 (time 1788579861.07 total 10.01 last 132.06) - /unlink 2789 (time 1788579871.07 total 20.01 last 146.65) - /unlink 4395 (time 1788579881.07 total 30.02 last 160.55) total: 4500 /unlink in 30.87 seconds: 145.78 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 848 (time 1788579896.67 total 10.01 last 84.74) - open/close 1648 (time 1788579906.67 total 20.01 last 79.95) - open/close 2481 (time 1788579916.69 total 30.03 last 83.19) - open/close 3363 (time 1788579926.69 total 40.03 last 88.18) - open/close 4249 (time 1788579936.71 total 50.04 last 88.45) total: 4500 open/close in 52.95 seconds: 84.99 ops/second - /unlink 1781 (time 1788579954.19 total 10.00 last 178.09) - /unlink 3376 (time 1788579964.20 total 20.00 last 159.44) total: 4500 /unlink in 28.56 seconds: 157.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 763 (time 1788579989.41 total 10.01 last 76.24) - open/close 1445 (time 1788579999.42 total 20.01 last 68.18) - open/close 2251 (time 1788580009.44 total 30.03 last 80.42) - open/close 3108 (time 1788580019.45 total 40.04 last 85.61) - open/close 4073 (time 1788580029.46 total 50.05 last 96.39) total: 4500 open/close in 54.68 seconds: 82.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1643 (time 1788580048.28 total 10.00 last 164.29) - /unlink 3089 (time 1788580058.29 total 20.01 last 144.49) total: 4500 /unlink in 29.15 seconds: 154.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 812 (time 1788580082.14 total 10.01 last 81.16) - open/close 1667 (time 1788580092.15 total 20.01 last 85.45) - open/close 2545 (time 1788580102.15 total 30.02 last 87.73) - open/close 3456 (time 1788580112.16 total 40.03 last 91.03) - open/close 4378 (time 1788580122.18 total 50.05 last 91.99) total: 4500 open/close in 51.46 seconds: 87.45 ops/second - /unlink 1965 (time 1788580137.75 total 10.00 last 196.49) - /unlink 3646 (time 1788580147.75 total 20.00 last 168.09) total: 4500 /unlink in 24.77 seconds: 181.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 836 (time 1788580166.96 total 10.01 last 83.54) - open/close 1616 (time 1788580176.97 total 20.02 last 77.94) - open/close 2459 (time 1788580186.97 total 30.02 last 84.27) - open/close 3431 (time 1788580196.98 total 40.03 last 97.11) - open/close 4096 (time 1788580206.99 total 50.04 last 66.42) total: 4500 open/close in 54.42 seconds: 82.69 ops/second - /unlink 1777 (time 1788580225.84 total 10.00 last 177.66) - /unlink 3360 (time 1788580235.85 total 20.02 last 158.09) total: 4500 /unlink in 27.34 seconds: 164.62 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 906 (time 1788580257.69 total 10.00 last 90.59) - open/close 1801 (time 1788580267.69 total 20.01 last 89.45) - open/close 2745 (time 1788580277.70 total 30.01 last 94.36) - open/close 3546 (time 1788580287.70 total 40.02 last 80.04) - open/close 4289 (time 1788580297.71 total 50.02 last 74.28) total: 4500 open/close in 52.38 seconds: 85.91 ops/second - /unlink 1411 (time 1788580314.88 total 10.01 last 140.95) - /unlink 3048 (time 1788580324.88 total 20.01 last 163.67) total: 4500 /unlink in 28.97 seconds: 155.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 803 (time 1788580349.95 total 10.00 last 80.28) - open/close 1556 (time 1788580359.97 total 20.01 last 75.21) - open/close 2395 (time 1788580369.97 total 30.02 last 83.86) - open/close 3289 (time 1788580380.00 total 40.04 last 89.18) - open/close 3869 (time 1788580390.00 total 50.05 last 57.96) total: 4500 open/close in 58.48 seconds: 76.95 ops/second - /unlink 1916 (time 1788580411.93 total 10.00 last 191.57) - /unlink 3494 (time 1788580421.93 total 20.01 last 157.70) total: 4500 /unlink in 26.35 seconds: 170.78 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 787 (time 1788580442.23 total 10.07 last 78.12) - open/close 1586 (time 1788580452.24 total 20.08 last 79.85) - open/close 2385 (time 1788580462.26 total 30.10 last 79.72) - open/close 3292 (time 1788580472.34 total 40.18 last 89.97) - open/close 4099 (time 1788580482.35 total 50.19 last 80.62) total: 4500 open/close in 55.21 seconds: 81.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1433 (time 1788580502.52 total 10.01 last 143.12) - /unlink 3001 (time 1788580512.52 total 20.01 last 156.78) total: 4500 /unlink in 29.81 seconds: 150.95 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 935 (time 1788580536.68 total 10.00 last 93.49) - open/close 1655 (time 1788580546.69 total 20.01 last 71.96) - open/close 2519 (time 1788580556.69 total 30.01 last 86.34) - open/close 3375 (time 1788580566.71 total 40.03 last 85.46) - open/close 4211 (time 1788580576.72 total 50.04 last 83.52) total: 4500 open/close in 53.32 seconds: 84.40 ops/second - /unlink 1671 (time 1788580594.73 total 10.00 last 167.09) - /unlink 3284 (time 1788580604.74 total 20.01 last 161.09) total: 4500 /unlink in 27.56 seconds: 163.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 819 (time 1788580627.22 total 10.00 last 81.87) - open/close 1777 (time 1788580637.22 total 20.00 last 95.80) - open/close 2737 (time 1788580647.23 total 30.01 last 95.96) - open/close 3545 (time 1788580657.25 total 40.03 last 80.64) - open/close 4393 (time 1788580667.26 total 50.04 last 84.69) total: 4500 open/close in 51.29 seconds: 87.73 ops/second - /unlink 1288 (time 1788580683.58 total 10.00 last 128.75) - /unlink 2292 (time 1788580693.58 total 20.01 last 100.38) - /unlink 3883 (time 1788580703.58 total 30.01 last 159.10) total: 4500 /unlink in 34.35 seconds: 131.01 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95202 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg133-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg133-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg133-server stop mds service on oleg133-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg133-server stop mds service on oleg133-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg133-server unloading modules via unload_modules_local on: 'oleg133-server' oleg133-server: oleg133-server.virtnet: executing unload_modules_local oleg133-server: kmemleak disabled modules unloaded.