== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 23:25:07 (1788837907) Stopping clients: oleg143-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg143-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg143-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg143-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg143-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg143-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg143-server oleg143-server: oleg143-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing load_modules_local oleg143-server: kmemleak disabled oleg143-server: Loading modules from /home/green/git/lustre-release/lustre oleg143-server: detected 4 online CPUs by sysfs oleg143-server: MODOPTS_LIBCFS= oleg143-server: Force libcfs to create 2 CPU partitions oleg143-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg143-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg143-server: oleg143-server.virtnet: executing set_default_debug -1 all pdsh@oleg143-client: oleg143-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg143-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg143-server: oleg143-server.virtnet: executing set_default_debug -1 all pdsh@oleg143-client: oleg143-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg143-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg143-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg143-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg143-server: oleg143-server.virtnet: executing set_default_debug -1 all pdsh@oleg143-client: oleg143-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg143-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg143-client.virtnet: -o user_xattr,flock 192.168.201.143@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 707 (time 1788838115.01 total 10.00 last 70.69) - open/close 1458 (time 1788838125.01 total 20.01 last 75.06) - open/close 2146 (time 1788838135.02 total 30.01 last 68.79) - open/close 2895 (time 1788838145.02 total 40.01 last 74.89) - open/close 3618 (time 1788838155.05 total 50.04 last 72.07) - open/close 4319 (time 1788838165.06 total 60.05 last 70.02) total: 4500 open/close in 62.73 seconds: 71.73 ops/second - /unlink 1221 (time 1788838183.49 total 10.00 last 122.05) - /unlink 2341 (time 1788838193.50 total 20.01 last 111.94) - /unlink 3589 (time 1788838203.50 total 30.01 last 124.79) total: 4500 /unlink in 36.99 seconds: 121.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 700 (time 1788838225.55 total 10.00 last 69.99) - open/close 1358 (time 1788838235.55 total 20.00 last 65.80) - open/close 2089 (time 1788838245.55 total 30.01 last 73.05) - open/close 2746 (time 1788838255.56 total 40.01 last 65.68) - open/close 3425 (time 1788838265.59 total 50.04 last 67.69) - open/close 4101 (time 1788838275.60 total 60.05 last 67.54) total: 4500 open/close in 66.80 seconds: 67.36 ops/second - /unlink 1354 (time 1788838298.65 total 10.01 last 135.30) - /unlink 2634 (time 1788838308.65 total 20.01 last 127.95) - /unlink 3938 (time 1788838318.66 total 30.02 last 130.31) total: 4500 /unlink in 35.02 seconds: 128.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 754 (time 1788838338.71 total 10.01 last 75.36) - open/close 1468 (time 1788838348.71 total 20.01 last 71.35) - open/close 2108 (time 1788838358.72 total 30.02 last 63.96) - open/close 2861 (time 1788838368.74 total 40.04 last 75.13) - open/close 3619 (time 1788838378.75 total 50.05 last 75.76) - open/close 4247 (time 1788838388.76 total 60.06 last 62.73) total: 4500 open/close in 63.45 seconds: 70.92 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1421 (time 1788838407.63 total 10.00 last 142.04) - /unlink 2507 (time 1788838417.65 total 20.02 last 108.38) - /unlink 3481 (time 1788838427.65 total 30.03 last 97.32) total: 4500 /unlink in 38.06 seconds: 118.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 839 (time 1788838452.11 total 10.00 last 83.88) - open/close 1589 (time 1788838462.12 total 20.01 last 74.92) - open/close 2352 (time 1788838472.12 total 30.02 last 76.27) - open/close 3099 (time 1788838482.13 total 40.03 last 74.60) - open/close 3916 (time 1788838492.15 total 50.05 last 81.54) total: 4500 open/close in 58.41 seconds: 77.05 ops/second - /unlink 1122 (time 1788838515.78 total 10.00 last 112.20) - /unlink 2473 (time 1788838525.79 total 20.01 last 134.96) - /unlink 4006 (time 1788838535.79 total 30.01 last 153.29) total: 4500 /unlink in 34.00 seconds: 132.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 639 (time 1788838555.34 total 10.00 last 63.89) - open/close 1242 (time 1788838565.35 total 20.01 last 60.23) - open/close 1938 (time 1788838575.38 total 30.04 last 69.44) - open/close 2680 (time 1788838585.38 total 40.04 last 74.18) - open/close 3554 (time 1788838595.39 total 50.05 last 87.28) - open/close 4363 (time 1788838605.40 total 60.06 last 80.84) total: 4500 open/close in 61.70 seconds: 72.94 ops/second - /unlink 1449 (time 1788838622.34 total 10.00 last 144.86) - /unlink 2848 (time 1788838632.34 total 20.00 last 139.89) - /unlink 4305 (time 1788838642.34 total 30.01 last 145.60) total: 4500 /unlink in 31.71 seconds: 141.89 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 728 (time 1788838658.90 total 10.00 last 72.78) - open/close 1424 (time 1788838668.91 total 20.01 last 69.53) - open/close 2143 (time 1788838678.91 total 30.02 last 71.88) - open/close 2953 (time 1788838688.93 total 40.03 last 80.89) - open/close 3756 (time 1788838698.94 total 50.04 last 80.21) - open/close 4463 (time 1788838708.95 total 60.05 last 70.61) total: 4500 open/close in 60.69 seconds: 74.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1215 (time 1788838724.72 total 10.00 last 121.45) - /unlink 2314 (time 1788838734.73 total 20.01 last 109.78) - /unlink 3342 (time 1788838744.75 total 30.03 last 102.60) total: 4500 /unlink in 38.25 seconds: 117.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 631 (time 1788838768.59 total 10.01 last 63.05) - open/close 1427 (time 1788838778.61 total 20.03 last 79.46) - open/close 2181 (time 1788838788.62 total 30.04 last 75.31) - open/close 2833 (time 1788838798.63 total 40.05 last 65.11) - open/close 3491 (time 1788838808.64 total 50.06 last 65.77) - open/close 4158 (time 1788838818.65 total 60.06 last 66.64) total: 4500 open/close in 64.50 seconds: 69.77 ops/second - /unlink 1433 (time 1788838837.75 total 10.00 last 143.23) - /unlink 2712 (time 1788838847.75 total 20.01 last 127.84) - /unlink 3878 (time 1788838857.75 total 30.01 last 116.58) total: 4500 /unlink in 35.77 seconds: 125.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 731 (time 1788838878.72 total 10.01 last 73.06) - open/close 1378 (time 1788838888.74 total 20.03 last 64.55) - open/close 2150 (time 1788838898.75 total 30.03 last 77.17) - open/close 2884 (time 1788838908.77 total 40.05 last 73.26) - open/close 3597 (time 1788838918.77 total 50.06 last 71.28) - open/close 4320 (time 1788838928.78 total 60.06 last 72.25) total: 4500 open/close in 62.03 seconds: 72.55 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1327 (time 1788838945.54 total 10.01 last 132.54) - /unlink 2709 (time 1788838955.54 total 20.01 last 138.17) - /unlink 4177 (time 1788838965.54 total 30.02 last 146.75) total: 4500 /unlink in 32.19 seconds: 139.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 739 (time 1788838982.73 total 10.01 last 73.84) - open/close 1441 (time 1788838992.74 total 20.02 last 70.13) - open/close 2200 (time 1788839002.74 total 30.02 last 75.89) - open/close 2994 (time 1788839012.75 total 40.03 last 79.29) - open/close 3644 (time 1788839022.76 total 50.04 last 64.96) - open/close 4238 (time 1788839032.77 total 60.06 last 59.29) total: 4500 open/close in 65.09 seconds: 69.13 ops/second - /unlink 1407 (time 1788839054.05 total 10.00 last 140.67) - /unlink 2597 (time 1788839064.05 total 20.00 last 119.00) - /unlink 4095 (time 1788839074.05 total 30.00 last 149.78) total: 4500 /unlink in 33.13 seconds: 135.82 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 713 (time 1788839092.22 total 10.00 last 71.29) - open/close 1379 (time 1788839102.22 total 20.01 last 66.56) - open/close 2030 (time 1788839112.23 total 30.01 last 65.06) - open/close 2783 (time 1788839122.26 total 40.04 last 75.08) - open/close 3530 (time 1788839132.26 total 50.05 last 74.68) - open/close 4266 (time 1788839142.27 total 60.05 last 73.54) total: 4500 open/close in 63.42 seconds: 70.96 ops/second - /unlink 1399 (time 1788839161.11 total 10.01 last 139.79) - /unlink 2830 (time 1788839171.12 total 20.01 last 143.03) - /unlink 4296 (time 1788839181.12 total 30.02 last 146.54) total: 4500 /unlink in 31.78 seconds: 141.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 738 (time 1788839197.13 total 10.01 last 73.71) - open/close 1439 (time 1788839207.13 total 20.01 last 70.09) - open/close 2261 (time 1788839217.14 total 30.02 last 82.12) - open/close 2986 (time 1788839227.14 total 40.03 last 72.45) - open/close 3673 (time 1788839237.15 total 50.03 last 68.68) - open/close 4367 (time 1788839247.15 total 60.04 last 69.38) total: 4500 open/close in 61.71 seconds: 72.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1261 (time 1788839263.13 total 10.00 last 126.07) - /unlink 2519 (time 1788839273.15 total 20.02 last 125.60) - /unlink 3603 (time 1788839283.16 total 30.03 last 108.31) total: 4500 /unlink in 38.43 seconds: 117.10 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 589 (time 1788839307.50 total 10.00 last 58.88) - open/close 1141 (time 1788839317.51 total 20.01 last 55.18) - open/close 1550 (time 1788839327.54 total 30.04 last 40.76) - open/close 2197 (time 1788839337.57 total 40.07 last 64.53) - open/close 2938 (time 1788839347.57 total 50.07 last 74.09) - open/close 3660 (time 1788839357.57 total 60.07 last 72.19) - open/close 4292 (time 1788839367.57 total 70.07 last 63.17) total: 4500 open/close in 73.27 seconds: 61.42 ops/second - /unlink 1268 (time 1788839385.77 total 10.01 last 126.73) - /unlink 2392 (time 1788839395.77 total 20.01 last 112.37) - /unlink 3562 (time 1788839405.77 total 30.01 last 116.97) total: 4500 /unlink in 38.89 seconds: 115.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 582 (time 1788839429.80 total 10.00 last 58.18) - open/close 1166 (time 1788839439.82 total 20.02 last 58.32) - open/close 1784 (time 1788839449.82 total 30.02 last 61.80) - open/close 2495 (time 1788839459.83 total 40.03 last 71.05) - open/close 3060 (time 1788839469.86 total 50.06 last 56.30) - open/close 3740 (time 1788839479.88 total 60.08 last 67.90) - open/close 4276 (time 1788839489.90 total 70.10 last 53.49) total: 4500 open/close in 73.04 seconds: 61.61 ops/second - /unlink 1308 (time 1788839508.13 total 10.00 last 130.79) - /unlink 2682 (time 1788839518.13 total 20.00 last 137.38) - /unlink 3642 (time 1788839528.13 total 30.00 last 95.98) total: 4500 /unlink in 38.13 seconds: 118.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 700 (time 1788839552.02 total 10.01 last 69.90) - open/close 1288 (time 1788839562.03 total 20.02 last 58.79) - open/close 1962 (time 1788839572.03 total 30.03 last 67.34) - open/close 2695 (time 1788839582.04 total 40.03 last 73.25) - open/close 3390 (time 1788839592.05 total 50.04 last 69.42) - open/close 4097 (time 1788839602.06 total 60.05 last 70.67) total: 4500 open/close in 66.13 seconds: 68.05 ops/second - /unlink 1220 (time 1788839623.53 total 10.01 last 121.90) - /unlink 2601 (time 1788839633.54 total 20.02 last 137.98) - /unlink 3929 (time 1788839643.55 total 30.03 last 132.69) total: 4500 /unlink in 34.30 seconds: 131.21 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 646 (time 1788839663.52 total 10.01 last 64.54) - open/close 1333 (time 1788839673.54 total 20.03 last 68.59) - open/close 1927 (time 1788839683.56 total 30.05 last 59.24) - open/close 2644 (time 1788839693.57 total 40.06 last 71.64) - open/close 3297 (time 1788839703.58 total 50.07 last 65.23) - open/close 3988 (time 1788839713.59 total 60.08 last 69.02) total: 4500 open/close in 68.18 seconds: 66.00 ops/second - /unlink 1198 (time 1788839738.12 total 10.01 last 119.72) - /unlink 2504 (time 1788839748.12 total 20.01 last 130.55) - /unlink 3673 (time 1788839758.14 total 30.03 last 116.73) total: 4500 /unlink in 36.86 seconds: 122.07 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 688 (time 1788839780.60 total 10.00 last 68.77) - open/close 1322 (time 1788839790.60 total 20.01 last 63.38) - open/close 1936 (time 1788839800.64 total 30.04 last 61.20) - open/close 2474 (time 1788839810.65 total 40.05 last 53.76) - open/close 3018 (time 1788839820.67 total 50.07 last 54.28) - open/close 3527 (time 1788839830.68 total 60.09 last 50.82) - open/close 4069 (time 1788839840.69 total 70.10 last 54.15) total: 4500 open/close in 77.63 seconds: 57.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1321 (time 1788839863.59 total 10.01 last 132.03) - /unlink 2337 (time 1788839873.59 total 20.01 last 101.58) - /unlink 3459 (time 1788839883.59 total 30.01 last 112.13) total: 4500 /unlink in 39.14 seconds: 114.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 683 (time 1788839908.67 total 10.00 last 68.29) - open/close 1383 (time 1788839918.68 total 20.01 last 69.93) - open/close 2043 (time 1788839928.69 total 30.02 last 65.93) - open/close 2738 (time 1788839938.70 total 40.03 last 69.47) - open/close 3366 (time 1788839948.71 total 50.04 last 62.73) - open/close 3923 (time 1788839958.71 total 60.04 last 55.69) total: 4500 open/close in 67.66 seconds: 66.51 ops/second - /unlink 1299 (time 1788839981.77 total 10.00 last 129.90) - /unlink 2653 (time 1788839991.77 total 20.00 last 135.37) - /unlink 3867 (time 1788840001.77 total 30.00 last 121.39) total: 4500 /unlink in 35.86 seconds: 125.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 641 (time 1788840022.90 total 10.02 last 63.98) - open/close 1377 (time 1788840032.94 total 20.06 last 73.32) - open/close 2090 (time 1788840042.95 total 30.07 last 71.17) - open/close 2750 (time 1788840052.96 total 40.08 last 65.96) - open/close 3405 (time 1788840062.99 total 50.11 last 65.31) - open/close 4088 (time 1788840073.00 total 60.13 last 68.19) total: 4500 open/close in 65.68 seconds: 68.51 ops/second - /unlink 1193 (time 1788840093.82 total 10.00 last 119.27) - /unlink 2094 (time 1788840103.83 total 20.01 last 90.02) - /unlink 2912 (time 1788840113.83 total 30.02 last 81.76) - /unlink 3897 (time 1788840123.84 total 40.02 last 98.43) total: 4500 /unlink in 45.18 seconds: 99.61 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95460 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg143-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg143-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg143-server stop mds service on oleg143-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg143-server stop mds service on oleg143-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg143-server unloading modules via unload_modules_local on: 'oleg143-server' oleg143-server: oleg143-server.virtnet: executing unload_modules_local oleg143-server: kmemleak disabled modules unloaded.