== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 04:12:42 (1788855162) Stopping clients: oleg241-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg241-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg241-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg241-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg241-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg241-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg241-server oleg241-server: oleg241-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg241-server' oleg241-server: oleg241-server.virtnet: executing load_modules_local oleg241-server: kmemleak disabled oleg241-server: Loading modules from /home/green/git/lustre-release/lustre oleg241-server: detected 4 online CPUs by sysfs oleg241-server: MODOPTS_LIBCFS= oleg241-server: Force libcfs to create 2 CPU partitions oleg241-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg241-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg241-server: oleg241-server.virtnet: executing set_default_debug -1 all pdsh@oleg241-client: oleg241-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg241-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg241-server: oleg241-server.virtnet: executing set_default_debug -1 all pdsh@oleg241-client: oleg241-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg241-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg241-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg241-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg241-server: oleg241-server.virtnet: executing set_default_debug -1 all pdsh@oleg241-client: oleg241-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg241-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg241-client.virtnet: -o user_xattr,flock 192.168.202.141@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 827 (time 1788855388.69 total 10.01 last 82.60) - open/close 1583 (time 1788855398.69 total 20.01 last 75.59) - open/close 2262 (time 1788855408.69 total 30.02 last 67.88) - open/close 2968 (time 1788855418.70 total 40.02 last 70.55) - open/close 3753 (time 1788855428.71 total 50.04 last 78.41) total: 4500 open/close in 58.02 seconds: 77.56 ops/second - /unlink 1799 (time 1788855451.59 total 10.00 last 179.84) - /unlink 3399 (time 1788855461.59 total 20.00 last 159.99) total: 4500 /unlink in 26.34 seconds: 170.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 666 (time 1788855484.30 total 10.05 last 66.27) - open/close 1325 (time 1788855494.31 total 20.06 last 65.84) - open/close 2089 (time 1788855504.31 total 30.06 last 76.37) - open/close 2743 (time 1788855514.32 total 40.07 last 65.34) - open/close 3730 (time 1788855524.32 total 50.07 last 98.68) total: 4500 open/close in 58.69 seconds: 76.67 ops/second - /unlink 1128 (time 1788855548.74 total 10.00 last 112.78) - /unlink 2237 (time 1788855558.75 total 20.01 last 110.81) - /unlink 3456 (time 1788855568.76 total 30.02 last 121.79) total: 4500 /unlink in 38.75 seconds: 116.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 731 (time 1788855592.92 total 10.03 last 72.88) - open/close 1504 (time 1788855602.92 total 20.03 last 77.30) - open/close 2322 (time 1788855612.92 total 30.03 last 81.77) - open/close 3138 (time 1788855622.93 total 40.04 last 81.55) - open/close 3932 (time 1788855632.94 total 50.06 last 79.27) total: 4500 open/close in 56.71 seconds: 79.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1660 (time 1788855654.40 total 10.00 last 166.00) - /unlink 3248 (time 1788855664.41 total 20.01 last 158.70) total: 4500 /unlink in 27.64 seconds: 162.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 805 (time 1788855686.92 total 10.02 last 80.36) - open/close 1494 (time 1788855696.94 total 20.03 last 68.79) - open/close 2151 (time 1788855706.95 total 30.05 last 65.62) - open/close 2833 (time 1788855716.95 total 40.05 last 68.19) - open/close 3538 (time 1788855726.97 total 50.07 last 70.37) - open/close 4138 (time 1788855736.98 total 60.07 last 59.97) total: 4500 open/close in 65.23 seconds: 68.99 ops/second - /unlink 1212 (time 1788855757.84 total 10.00 last 121.19) - /unlink 2831 (time 1788855767.85 total 20.01 last 161.79) - /unlink 4310 (time 1788855777.85 total 30.01 last 147.90) total: 4500 /unlink in 31.89 seconds: 141.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 773 (time 1788855794.57 total 10.03 last 77.04) - open/close 1686 (time 1788855804.58 total 20.04 last 91.27) - open/close 2492 (time 1788855814.60 total 30.06 last 80.38) - open/close 3191 (time 1788855824.62 total 40.08 last 69.77) - open/close 3980 (time 1788855834.63 total 50.09 last 78.83) total: 4500 open/close in 55.74 seconds: 80.73 ops/second - /unlink 1434 (time 1788855854.87 total 10.01 last 143.31) - /unlink 2832 (time 1788855864.87 total 20.01 last 139.73) total: 4500 /unlink in 29.83 seconds: 150.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 927 (time 1788855890.10 total 10.01 last 92.65) - open/close 1719 (time 1788855900.11 total 20.02 last 79.10) - open/close 2633 (time 1788855910.12 total 30.03 last 91.30) - open/close 3314 (time 1788855920.14 total 40.04 last 68.01) - open/close 4067 (time 1788855930.14 total 50.04 last 75.29) total: 4500 open/close in 58.34 seconds: 77.14 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1382 (time 1788855953.43 total 10.00 last 138.17) - /unlink 2958 (time 1788855963.45 total 20.02 last 157.32) - /unlink 4452 (time 1788855973.45 total 30.02 last 149.37) total: 4500 /unlink in 30.50 seconds: 147.54 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 573 (time 1788855991.38 total 10.02 last 57.16) - open/close 1281 (time 1788856001.38 total 20.03 last 70.78) - open/close 1967 (time 1788856011.39 total 30.04 last 68.50) - open/close 2641 (time 1788856021.41 total 40.06 last 67.27) - open/close 3245 (time 1788856031.43 total 50.08 last 60.27) - open/close 4048 (time 1788856041.44 total 60.09 last 80.27) total: 4500 open/close in 66.17 seconds: 68.01 ops/second - /unlink 1749 (time 1788856063.53 total 10.00 last 174.85) - /unlink 3475 (time 1788856073.53 total 20.00 last 172.59) total: 4500 /unlink in 26.35 seconds: 170.78 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 851 (time 1788856093.81 total 10.00 last 85.06) - open/close 1716 (time 1788856103.81 total 20.01 last 86.44) - open/close 2689 (time 1788856113.81 total 30.01 last 97.30) - open/close 3552 (time 1788856123.82 total 40.01 last 86.27) - open/close 4420 (time 1788856133.82 total 50.02 last 86.77) total: 4500 open/close in 51.14 seconds: 87.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1476 (time 1788856150.26 total 10.00 last 147.60) - /unlink 2660 (time 1788856160.26 total 20.00 last 118.38) - /unlink 3876 (time 1788856170.27 total 30.01 last 121.50) total: 4500 /unlink in 34.30 seconds: 131.19 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 1025 (time 1788856191.02 total 10.01 last 102.44) - open/close 1864 (time 1788856201.03 total 20.02 last 83.78) - open/close 2583 (time 1788856211.06 total 30.05 last 71.68) - open/close 3052 (time 1788856221.07 total 40.06 last 46.86) - open/close 3577 (time 1788856231.09 total 50.08 last 52.40) - open/close 4052 (time 1788856241.12 total 60.11 last 47.34) total: 4500 open/close in 65.71 seconds: 68.48 ops/second - /unlink 1419 (time 1788856262.17 total 10.00 last 141.89) - /unlink 3141 (time 1788856272.17 total 20.00 last 172.15) total: 4500 /unlink in 27.96 seconds: 160.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 795 (time 1788856295.58 total 10.02 last 79.38) - open/close 1618 (time 1788856305.61 total 20.04 last 82.13) - open/close 2431 (time 1788856315.61 total 30.04 last 81.27) - open/close 3240 (time 1788856325.61 total 40.04 last 80.89) - open/close 4081 (time 1788856335.62 total 50.05 last 84.05) total: 4500 open/close in 55.47 seconds: 81.12 ops/second - /unlink 1174 (time 1788856355.99 total 10.01 last 117.33) - /unlink 2492 (time 1788856366.00 total 20.02 last 131.63) - /unlink 3519 (time 1788856376.00 total 30.02 last 102.70) total: 4500 /unlink in 37.75 seconds: 119.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 601 (time 1788856400.50 total 10.00 last 60.10) - open/close 1189 (time 1788856410.51 total 20.01 last 58.77) - open/close 1856 (time 1788856420.51 total 30.01 last 66.67) - open/close 2656 (time 1788856430.51 total 40.01 last 79.98) - open/close 3496 (time 1788856440.52 total 50.02 last 83.95) - open/close 4261 (time 1788856450.53 total 60.03 last 76.45) total: 4500 open/close in 63.66 seconds: 70.69 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1298 (time 1788856469.74 total 10.00 last 129.78) - /unlink 2524 (time 1788856479.74 total 20.01 last 122.55) - /unlink 3948 (time 1788856489.75 total 30.01 last 142.31) total: 4500 /unlink in 33.08 seconds: 136.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 962 (time 1788856507.42 total 10.00 last 96.18) - open/close 1810 (time 1788856517.43 total 20.00 last 84.79) - open/close 2694 (time 1788856527.43 total 30.01 last 88.36) - open/close 3377 (time 1788856537.44 total 40.02 last 68.23) - open/close 4011 (time 1788856547.44 total 50.02 last 63.38) total: 4500 open/close in 59.48 seconds: 75.66 ops/second - /unlink 942 (time 1788856573.61 total 10.01 last 94.11) - /unlink 2222 (time 1788856583.61 total 20.01 last 127.94) - /unlink 3703 (time 1788856593.62 total 30.02 last 148.03) total: 4500 /unlink in 35.98 seconds: 125.09 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 928 (time 1788856615.03 total 10.00 last 92.80) - open/close 1763 (time 1788856625.05 total 20.02 last 83.35) - open/close 2558 (time 1788856635.09 total 30.06 last 79.14) - open/close 3235 (time 1788856645.11 total 40.08 last 67.57) - open/close 3895 (time 1788856655.12 total 50.09 last 65.96) total: 4500 open/close in 58.62 seconds: 76.77 ops/second - /unlink 1327 (time 1788856678.38 total 10.00 last 132.68) - /unlink 2855 (time 1788856688.39 total 20.01 last 152.70) - /unlink 4347 (time 1788856698.39 total 30.01 last 149.15) total: 4500 /unlink in 31.48 seconds: 142.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 783 (time 1788856715.60 total 10.01 last 78.21) - open/close 1391 (time 1788856725.61 total 20.02 last 60.73) - open/close 2121 (time 1788856735.63 total 30.04 last 72.85) - open/close 2795 (time 1788856745.64 total 40.05 last 67.38) - open/close 3457 (time 1788856755.64 total 50.05 last 66.18) - open/close 4162 (time 1788856765.65 total 60.06 last 70.44) total: 4500 open/close in 63.43 seconds: 70.95 ops/second - /unlink 1423 (time 1788856783.79 total 10.01 last 142.12) - /unlink 2938 (time 1788856793.80 total 20.02 last 151.32) total: 4500 /unlink in 29.02 seconds: 155.05 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 751 (time 1788856818.39 total 10.00 last 75.08) - open/close 1526 (time 1788856828.41 total 20.02 last 77.38) - open/close 2212 (time 1788856838.42 total 30.03 last 68.49) - open/close 2881 (time 1788856848.43 total 40.04 last 66.86) - open/close 3768 (time 1788856858.44 total 50.05 last 88.64) total: 4500 open/close in 58.13 seconds: 77.41 ops/second - /unlink 1451 (time 1788856881.50 total 10.03 last 144.68) - /unlink 2982 (time 1788856891.51 total 20.03 last 153.08) - /unlink 4448 (time 1788856901.51 total 30.03 last 146.57) total: 4500 /unlink in 30.78 seconds: 146.21 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 560 (time 1788856917.76 total 10.01 last 55.94) - open/close 1142 (time 1788856927.80 total 20.05 last 57.97) - open/close 1790 (time 1788856937.81 total 30.07 last 64.69) - open/close 2284 (time 1788856947.83 total 40.08 last 49.32) - open/close 2866 (time 1788856957.84 total 50.09 last 58.15) - open/close 3458 (time 1788856967.85 total 60.10 last 59.15) - open/close 4166 (time 1788856977.86 total 70.11 last 70.73) total: 4500 open/close in 74.17 seconds: 60.67 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1406 (time 1788856997.13 total 10.00 last 140.53) - /unlink 2732 (time 1788857007.13 total 20.01 last 132.55) - /unlink 4112 (time 1788857017.14 total 30.01 last 137.98) total: 4500 /unlink in 32.22 seconds: 139.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 641 (time 1788857034.76 total 10.00 last 64.08) - open/close 1442 (time 1788857044.77 total 20.01 last 80.05) - open/close 2258 (time 1788857054.77 total 30.02 last 81.54) - open/close 3078 (time 1788857064.78 total 40.02 last 81.93) - open/close 3934 (time 1788857074.78 total 50.03 last 85.58) total: 4500 open/close in 55.85 seconds: 80.57 ops/second - /unlink 1475 (time 1788857095.05 total 10.00 last 147.43) - /unlink 3057 (time 1788857105.05 total 20.01 last 158.19) - /unlink 4238 (time 1788857115.06 total 30.01 last 118.07) total: 4500 /unlink in 31.88 seconds: 141.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 672 (time 1788857133.60 total 10.01 last 67.14) - open/close 1207 (time 1788857143.60 total 20.01 last 53.49) - open/close 1780 (time 1788857153.60 total 30.02 last 57.27) - open/close 2572 (time 1788857163.60 total 40.02 last 79.19) - open/close 3404 (time 1788857173.61 total 50.03 last 83.13) - open/close 4208 (time 1788857183.64 total 60.06 last 80.16) total: 4500 open/close in 64.39 seconds: 69.89 ops/second - /unlink 1666 (time 1788857203.37 total 10.00 last 166.54) - /unlink 3090 (time 1788857213.39 total 20.02 last 142.19) - /unlink 4437 (time 1788857223.40 total 30.03 last 134.57) total: 4500 /unlink in 30.70 seconds: 146.56 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11350: 95343 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg241-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg241-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg241-server stop mds service on oleg241-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg241-server stop mds service on oleg241-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg241-server unloading modules via unload_modules_local on: 'oleg241-server' oleg241-server: oleg241-server.virtnet: executing unload_modules_local oleg241-server: kmemleak disabled modules unloaded.