== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 13:20:43 (1788974443) Stopping clients: oleg235-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg235-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg235-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg235-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg235-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg235-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg235-server oleg235-server: oleg235-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg235-server' oleg235-server: oleg235-server.virtnet: executing load_modules_local oleg235-server: kmemleak disabled oleg235-server: Loading modules from /home/green/git/lustre-release/lustre oleg235-server: detected 4 online CPUs by sysfs oleg235-server: MODOPTS_LIBCFS= oleg235-server: Force libcfs to create 2 CPU partitions oleg235-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg235-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg235-server: oleg235-server.virtnet: executing set_default_debug -1 all pdsh@oleg235-client: oleg235-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg235-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg235-server: oleg235-server.virtnet: executing set_default_debug -1 all pdsh@oleg235-client: oleg235-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg235-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg235-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg235-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg235-server: oleg235-server.virtnet: executing set_default_debug -1 all pdsh@oleg235-client: oleg235-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg235-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg235-client.virtnet: -o user_xattr,flock 192.168.202.135@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 737 (time 1788974665.11 total 10.00 last 73.67) - open/close 1532 (time 1788974675.12 total 20.02 last 79.35) - open/close 2148 (time 1788974685.13 total 30.02 last 61.59) - open/close 2800 (time 1788974695.13 total 40.03 last 65.15) - open/close 3436 (time 1788974705.15 total 50.05 last 63.48) - open/close 4182 (time 1788974715.15 total 60.05 last 74.59) total: 4500 open/close in 65.56 seconds: 68.63 ops/second - /unlink 1049 (time 1788974736.01 total 10.01 last 104.80) - /unlink 2349 (time 1788974746.01 total 20.01 last 129.98) - /unlink 3665 (time 1788974756.01 total 30.01 last 131.56) total: 4500 /unlink in 36.74 seconds: 122.48 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 889 (time 1788974777.86 total 10.00 last 88.88) - open/close 1565 (time 1788974787.87 total 20.01 last 67.54) - open/close 2213 (time 1788974797.87 total 30.01 last 64.78) - open/close 2932 (time 1788974807.88 total 40.02 last 71.85) - open/close 3744 (time 1788974817.88 total 50.02 last 81.18) - open/close 4466 (time 1788974827.89 total 60.03 last 72.14) total: 4500 open/close in 60.47 seconds: 74.41 ops/second - /unlink 1356 (time 1788974843.93 total 10.00 last 135.58) - /unlink 2778 (time 1788974853.94 total 20.01 last 142.12) - /unlink 3975 (time 1788974863.94 total 30.01 last 119.66) total: 4500 /unlink in 34.11 seconds: 131.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 753 (time 1788974882.98 total 10.01 last 75.21) - open/close 1514 (time 1788974892.99 total 20.02 last 76.06) - open/close 2222 (time 1788974902.99 total 30.02 last 70.77) - open/close 2906 (time 1788974912.99 total 40.03 last 68.37) - open/close 3707 (time 1788974923.00 total 50.03 last 80.06) - open/close 4315 (time 1788974933.01 total 60.04 last 60.75) total: 4500 open/close in 63.82 seconds: 70.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1318 (time 1788974952.65 total 10.00 last 131.74) - /unlink 2660 (time 1788974962.65 total 20.01 last 134.18) - /unlink 3920 (time 1788974972.65 total 30.01 last 125.93) total: 4500 /unlink in 34.76 seconds: 129.44 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 651 (time 1788974992.51 total 10.00 last 65.07) - open/close 1357 (time 1788975002.51 total 20.01 last 70.58) - open/close 1980 (time 1788975012.52 total 30.02 last 62.25) - open/close 2706 (time 1788975022.53 total 40.03 last 72.52) - open/close 3310 (time 1788975032.55 total 50.04 last 60.30) - open/close 3997 (time 1788975042.55 total 60.05 last 68.66) total: 4500 open/close in 66.92 seconds: 67.25 ops/second - /unlink 1152 (time 1788975064.29 total 10.00 last 115.18) - /unlink 2240 (time 1788975074.29 total 20.00 last 108.79) - /unlink 3828 (time 1788975084.29 total 30.01 last 158.75) total: 4500 /unlink in 34.34 seconds: 131.03 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 664 (time 1788975104.41 total 10.01 last 66.31) - open/close 1392 (time 1788975114.41 total 20.02 last 72.78) - open/close 2168 (time 1788975124.42 total 30.02 last 77.56) - open/close 2905 (time 1788975134.42 total 40.03 last 73.64) - open/close 3624 (time 1788975144.43 total 50.03 last 71.88) - open/close 4420 (time 1788975154.43 total 60.04 last 79.54) total: 4500 open/close in 61.03 seconds: 73.73 ops/second - /unlink 1299 (time 1788975170.02 total 10.00 last 129.88) - /unlink 2359 (time 1788975180.03 total 20.01 last 105.90) - /unlink 3638 (time 1788975190.03 total 30.01 last 127.90) total: 4500 /unlink in 36.24 seconds: 124.16 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 740 (time 1788975211.43 total 10.00 last 73.98) - open/close 1514 (time 1788975221.44 total 20.01 last 77.35) - open/close 2284 (time 1788975231.45 total 30.02 last 76.91) - open/close 3017 (time 1788975241.47 total 40.04 last 73.14) - open/close 3630 (time 1788975251.48 total 50.05 last 61.28) - open/close 4274 (time 1788975261.48 total 60.05 last 64.38) total: 4500 open/close in 63.17 seconds: 71.23 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1216 (time 1788975279.64 total 10.02 last 121.37) - /unlink 2153 (time 1788975289.65 total 20.02 last 93.66) - /unlink 3130 (time 1788975299.65 total 30.03 last 97.67) - /unlink 4482 (time 1788975309.65 total 40.03 last 135.19) total: 4500 /unlink in 40.26 seconds: 111.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 748 (time 1788975325.31 total 10.01 last 74.75) - open/close 1414 (time 1788975335.33 total 20.03 last 66.47) - open/close 2105 (time 1788975345.36 total 30.06 last 68.85) - open/close 2699 (time 1788975355.37 total 40.07 last 59.37) - open/close 3446 (time 1788975365.37 total 50.07 last 74.67) - open/close 4292 (time 1788975375.38 total 60.08 last 84.54) total: 4500 open/close in 63.54 seconds: 70.82 ops/second - /unlink 1597 (time 1788975393.46 total 10.02 last 159.46) - /unlink 2736 (time 1788975403.47 total 20.02 last 113.80) - /unlink 3891 (time 1788975413.47 total 30.03 last 115.46) total: 4500 /unlink in 34.24 seconds: 131.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 592 (time 1788975433.36 total 10.03 last 59.03) - open/close 1274 (time 1788975443.40 total 20.07 last 67.93) - open/close 1993 (time 1788975453.40 total 30.07 last 71.87) - open/close 2751 (time 1788975463.41 total 40.08 last 75.74) - open/close 3468 (time 1788975473.42 total 50.08 last 71.69) - open/close 4121 (time 1788975483.42 total 60.09 last 65.27) total: 4500 open/close in 65.78 seconds: 68.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1321 (time 1788975503.99 total 10.01 last 132.03) - /unlink 2726 (time 1788975513.99 total 20.01 last 140.44) - /unlink 4066 (time 1788975523.99 total 30.01 last 133.99) total: 4500 /unlink in 33.17 seconds: 135.66 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 632 (time 1788975542.60 total 10.02 last 63.09) - open/close 1360 (time 1788975552.60 total 20.02 last 72.79) - open/close 2014 (time 1788975562.60 total 30.02 last 65.36) - open/close 2545 (time 1788975572.62 total 40.03 last 53.04) - open/close 3138 (time 1788975582.62 total 50.04 last 59.28) - open/close 3810 (time 1788975592.63 total 60.05 last 67.15) - open/close 4448 (time 1788975602.64 total 70.06 last 63.70) total: 4500 open/close in 70.85 seconds: 63.51 ops/second - /unlink 1411 (time 1788975618.59 total 10.00 last 141.05) - /unlink 2928 (time 1788975628.59 total 20.00 last 151.68) - /unlink 4349 (time 1788975638.59 total 30.01 last 142.08) total: 4500 /unlink in 31.76 seconds: 141.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 562 (time 1788975656.14 total 10.00 last 56.18) - open/close 1138 (time 1788975666.15 total 20.01 last 57.53) - open/close 1869 (time 1788975676.16 total 30.02 last 73.08) - open/close 2577 (time 1788975686.16 total 40.02 last 70.77) - open/close 3384 (time 1788975696.17 total 50.03 last 80.63) - open/close 4171 (time 1788975706.18 total 60.04 last 78.63) total: 4500 open/close in 64.40 seconds: 69.88 ops/second - /unlink 1405 (time 1788975724.84 total 10.00 last 140.47) - /unlink 2924 (time 1788975734.84 total 20.01 last 151.83) total: 4500 /unlink in 29.81 seconds: 150.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 578 (time 1788975759.74 total 10.01 last 57.76) - open/close 1286 (time 1788975769.75 total 20.02 last 70.73) - open/close 2076 (time 1788975779.75 total 30.02 last 78.96) - open/close 2857 (time 1788975789.76 total 40.02 last 78.09) - open/close 3499 (time 1788975799.76 total 50.03 last 64.16) - open/close 4201 (time 1788975809.77 total 60.04 last 70.15) total: 4500 open/close in 63.98 seconds: 70.33 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1340 (time 1788975828.71 total 10.01 last 133.92) - /unlink 2500 (time 1788975838.71 total 20.01 last 115.98) - /unlink 3964 (time 1788975848.73 total 30.03 last 146.11) total: 4500 /unlink in 34.72 seconds: 129.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 568 (time 1788975869.27 total 10.04 last 56.60) - open/close 1230 (time 1788975879.27 total 20.04 last 66.19) - open/close 1900 (time 1788975889.27 total 30.04 last 66.95) - open/close 2536 (time 1788975899.28 total 40.05 last 63.58) - open/close 3339 (time 1788975909.29 total 50.06 last 80.23) - open/close 4055 (time 1788975919.30 total 60.07 last 71.51) total: 4500 open/close in 66.57 seconds: 67.60 ops/second - /unlink 1107 (time 1788975940.20 total 10.00 last 110.69) - /unlink 2222 (time 1788975950.20 total 20.01 last 111.44) - /unlink 3197 (time 1788975960.22 total 30.02 last 97.39) - /unlink 4205 (time 1788975970.22 total 40.02 last 100.77) total: 4500 /unlink in 42.40 seconds: 106.14 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 732 (time 1788975988.15 total 10.01 last 73.13) - open/close 1416 (time 1788975998.16 total 20.02 last 68.36) - open/close 2009 (time 1788976008.16 total 30.02 last 59.26) - open/close 2581 (time 1788976018.17 total 40.03 last 57.17) - open/close 3022 (time 1788976028.17 total 50.03 last 44.09) - open/close 3694 (time 1788976038.17 total 60.03 last 67.20) - open/close 4384 (time 1788976048.19 total 70.05 last 68.89) total: 4500 open/close in 71.60 seconds: 62.85 ops/second - /unlink 1263 (time 1788976065.22 total 10.00 last 126.27) - /unlink 2189 (time 1788976075.22 total 20.01 last 92.58) - /unlink 3318 (time 1788976085.22 total 30.01 last 112.89) total: 4500 /unlink in 38.16 seconds: 117.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 696 (time 1788976108.39 total 10.00 last 69.59) - open/close 1415 (time 1788976118.41 total 20.02 last 71.78) - open/close 2098 (time 1788976128.42 total 30.03 last 68.25) - open/close 2678 (time 1788976138.42 total 40.03 last 57.97) - open/close 3478 (time 1788976148.43 total 50.04 last 79.96) - open/close 4208 (time 1788976158.43 total 60.04 last 73.00) total: 4500 open/close in 64.01 seconds: 70.30 ops/second - /unlink 1196 (time 1788976176.96 total 10.02 last 119.41) - /unlink 2501 (time 1788976186.97 total 20.02 last 130.49) - /unlink 3687 (time 1788976196.97 total 30.03 last 118.49) total: 4500 /unlink in 37.93 seconds: 118.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 707 (time 1788976220.18 total 10.00 last 70.68) - open/close 1337 (time 1788976230.19 total 20.02 last 62.92) - open/close 2073 (time 1788976240.19 total 30.02 last 73.57) - open/close 2738 (time 1788976250.21 total 40.03 last 66.42) - open/close 3327 (time 1788976260.21 total 50.03 last 58.89) - open/close 3993 (time 1788976270.22 total 60.04 last 66.56) total: 4500 open/close in 66.63 seconds: 67.53 ops/second - /unlink 1322 (time 1788976291.31 total 10.01 last 132.03) - /unlink 2544 (time 1788976301.31 total 20.01 last 122.17) - /unlink 3684 (time 1788976311.32 total 30.02 last 113.97) total: 4500 /unlink in 37.60 seconds: 119.68 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 700 (time 1788976334.23 total 10.01 last 69.96) - open/close 1359 (time 1788976344.23 total 20.01 last 65.90) - open/close 2144 (time 1788976354.24 total 30.01 last 78.44) - open/close 3009 (time 1788976364.26 total 40.04 last 86.30) - open/close 3753 (time 1788976374.26 total 50.04 last 74.39) total: 4500 open/close in 58.89 seconds: 76.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1355 (time 1788976397.99 total 10.00 last 135.46) - /unlink 2686 (time 1788976407.99 total 20.01 last 133.05) - /unlink 3771 (time 1788976418.00 total 30.01 last 108.47) total: 4500 /unlink in 36.75 seconds: 122.44 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 761 (time 1788976440.30 total 10.01 last 76.02) - open/close 1359 (time 1788976450.31 total 20.02 last 59.74) - open/close 1922 (time 1788976460.31 total 30.02 last 56.29) - open/close 2606 (time 1788976470.33 total 40.04 last 68.27) - open/close 3218 (time 1788976480.34 total 50.05 last 61.15) - open/close 3887 (time 1788976490.35 total 60.07 last 66.79) total: 4500 open/close in 69.71 seconds: 64.56 ops/second - /unlink 1182 (time 1788976515.81 total 10.00 last 118.18) - /unlink 2334 (time 1788976525.81 total 20.01 last 115.16) - /unlink 3734 (time 1788976535.81 total 30.01 last 139.99) total: 4500 /unlink in 36.41 seconds: 123.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 676 (time 1788976557.46 total 10.01 last 67.51) - open/close 1371 (time 1788976567.47 total 20.02 last 69.46) - open/close 2173 (time 1788976577.48 total 30.03 last 80.15) - open/close 2942 (time 1788976587.48 total 40.03 last 76.83) - open/close 3662 (time 1788976597.50 total 50.05 last 71.87) - open/close 4457 (time 1788976607.51 total 60.06 last 79.46) total: 4500 open/close in 60.56 seconds: 74.30 ops/second - /unlink 1381 (time 1788976622.78 total 10.00 last 138.06) - /unlink 2428 (time 1788976632.79 total 20.01 last 104.65) - /unlink 3446 (time 1788976642.80 total 30.02 last 101.73) total: 4500 /unlink in 39.10 seconds: 115.08 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11350: 95275 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg235-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg235-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg235-server stop mds service on oleg235-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg235-server stop mds service on oleg235-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg235-server unloading modules via unload_modules_local on: 'oleg235-server' oleg235-server: oleg235-server.virtnet: executing unload_modules_local oleg235-server: kmemleak disabled modules unloaded.