== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 09:06:14 (1788527174) Stopping clients: oleg235-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg235-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg235-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg235-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg235-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg235-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg235-server oleg235-server: oleg235-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg235-server' oleg235-server: oleg235-server.virtnet: executing load_modules_local oleg235-server: kmemleak disabled oleg235-server: Loading modules from /home/green/git/lustre-release/lustre oleg235-server: detected 4 online CPUs by sysfs oleg235-server: MODOPTS_LIBCFS= oleg235-server: Force libcfs to create 2 CPU partitions oleg235-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg235-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg235-server: oleg235-server.virtnet: executing set_default_debug -1 all pdsh@oleg235-client: oleg235-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg235-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg235-server: oleg235-server.virtnet: executing set_default_debug -1 all pdsh@oleg235-client: oleg235-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg235-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg235-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg235-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg235-server: oleg235-server.virtnet: executing set_default_debug -1 all pdsh@oleg235-client: oleg235-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg235-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg235-client.virtnet: -o user_xattr,flock 192.168.202.135@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 627 (time 1788527406.02 total 10.00 last 62.69) - open/close 1333 (time 1788527416.03 total 20.01 last 70.56) - open/close 2106 (time 1788527426.04 total 30.01 last 77.25) - open/close 2809 (time 1788527436.04 total 40.02 last 70.25) - open/close 3429 (time 1788527446.05 total 50.03 last 61.94) - open/close 4000 (time 1788527456.06 total 60.04 last 57.07) total: 4500 open/close in 67.61 seconds: 66.56 ops/second - /unlink 1397 (time 1788527478.92 total 10.00 last 139.64) - /unlink 2777 (time 1788527488.92 total 20.01 last 137.97) - /unlink 4130 (time 1788527498.92 total 30.01 last 135.30) total: 4500 /unlink in 33.26 seconds: 135.30 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 544 (time 1788527517.79 total 10.02 last 54.29) - open/close 1147 (time 1788527527.79 total 20.03 last 60.25) - open/close 1892 (time 1788527537.80 total 30.04 last 74.43) - open/close 2640 (time 1788527547.83 total 40.06 last 74.62) - open/close 3314 (time 1788527557.84 total 50.07 last 67.34) - open/close 4200 (time 1788527567.84 total 60.08 last 88.55) total: 4500 open/close in 63.15 seconds: 71.25 ops/second - /unlink 1697 (time 1788527585.21 total 10.01 last 169.56) - /unlink 3266 (time 1788527595.21 total 20.01 last 156.87) total: 4500 /unlink in 28.34 seconds: 158.76 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 823 (time 1788527618.90 total 10.01 last 82.23) - open/close 1647 (time 1788527628.91 total 20.01 last 82.36) - open/close 2437 (time 1788527638.91 total 30.02 last 78.97) - open/close 3070 (time 1788527648.92 total 40.03 last 63.23) - open/close 3877 (time 1788527658.93 total 50.04 last 80.64) total: 4500 open/close in 57.76 seconds: 77.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1526 (time 1788527680.96 total 10.00 last 152.55) - /unlink 3114 (time 1788527690.96 total 20.01 last 158.74) total: 4500 /unlink in 28.45 seconds: 158.18 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 886 (time 1788527714.66 total 10.01 last 88.51) - open/close 1746 (time 1788527724.67 total 20.02 last 85.88) - open/close 2498 (time 1788527734.69 total 30.05 last 75.03) - open/close 3099 (time 1788527744.70 total 40.05 last 60.08) - open/close 3850 (time 1788527754.71 total 50.06 last 75.03) total: 4500 open/close in 59.01 seconds: 76.25 ops/second - /unlink 1404 (time 1788527778.60 total 10.00 last 140.36) - /unlink 2624 (time 1788527788.60 total 20.01 last 121.97) - /unlink 4111 (time 1788527798.61 total 30.01 last 148.57) total: 4500 /unlink in 32.89 seconds: 136.83 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 711 (time 1788527817.49 total 10.03 last 70.91) - open/close 1304 (time 1788527827.50 total 20.03 last 59.25) - open/close 1988 (time 1788527837.50 total 30.04 last 68.39) - open/close 2728 (time 1788527847.52 total 40.05 last 73.87) - open/close 3527 (time 1788527857.53 total 50.07 last 79.76) - open/close 4371 (time 1788527867.54 total 60.08 last 84.35) total: 4500 open/close in 61.43 seconds: 73.26 ops/second - /unlink 1280 (time 1788527884.06 total 10.00 last 127.96) - /unlink 2509 (time 1788527894.06 total 20.00 last 122.90) - /unlink 3976 (time 1788527904.06 total 30.00 last 146.69) total: 4500 /unlink in 33.22 seconds: 135.44 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 689 (time 1788527923.53 total 10.00 last 68.87) - open/close 1383 (time 1788527933.53 total 20.01 last 69.38) - open/close 2097 (time 1788527943.54 total 30.01 last 71.36) - open/close 2795 (time 1788527953.54 total 40.02 last 69.77) - open/close 3225 (time 1788527963.55 total 50.02 last 42.98) - open/close 3852 (time 1788527973.59 total 60.06 last 62.44) total: 4500 open/close in 69.93 seconds: 64.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1473 (time 1788527998.85 total 10.01 last 147.22) - /unlink 2688 (time 1788528008.86 total 20.01 last 121.41) - /unlink 4140 (time 1788528018.86 total 30.02 last 145.14) total: 4500 /unlink in 32.54 seconds: 138.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 834 (time 1788528036.01 total 10.00 last 83.38) - open/close 1667 (time 1788528046.01 total 20.01 last 83.26) - open/close 2577 (time 1788528056.02 total 30.01 last 90.98) - open/close 3344 (time 1788528066.02 total 40.02 last 76.65) - open/close 4077 (time 1788528076.02 total 50.02 last 73.30) total: 4500 open/close in 55.65 seconds: 80.86 ops/second - /unlink 1421 (time 1788528097.57 total 10.00 last 142.09) - /unlink 2748 (time 1788528107.58 total 20.01 last 132.53) - /unlink 4476 (time 1788528117.58 total 30.01 last 172.78) total: 4500 /unlink in 30.22 seconds: 148.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 816 (time 1788528132.03 total 10.01 last 81.54) - open/close 1664 (time 1788528142.04 total 20.01 last 84.76) - open/close 2446 (time 1788528152.04 total 30.01 last 78.18) - open/close 3116 (time 1788528162.05 total 40.03 last 66.91) - open/close 3712 (time 1788528172.06 total 50.03 last 59.58) total: 4500 open/close in 59.44 seconds: 75.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1761 (time 1788528195.57 total 10.00 last 176.05) - /unlink 3328 (time 1788528205.58 total 20.02 last 156.45) total: 4500 /unlink in 27.82 seconds: 161.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 656 (time 1788528229.07 total 10.01 last 65.54) - open/close 1515 (time 1788528239.08 total 20.01 last 85.86) - open/close 2295 (time 1788528249.09 total 30.03 last 77.91) - open/close 3027 (time 1788528259.09 total 40.03 last 73.20) - open/close 3614 (time 1788528269.10 total 50.03 last 58.65) - open/close 4039 (time 1788528279.11 total 60.05 last 42.44) total: 4500 open/close in 67.26 seconds: 66.90 ops/second - /unlink 1675 (time 1788528301.99 total 10.00 last 167.48) - /unlink 2920 (time 1788528312.01 total 20.01 last 124.33) - /unlink 4421 (time 1788528322.01 total 30.02 last 150.08) total: 4500 /unlink in 31.04 seconds: 144.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 678 (time 1788528338.57 total 10.00 last 67.79) - open/close 1311 (time 1788528348.57 total 20.01 last 63.27) - open/close 2096 (time 1788528358.58 total 30.01 last 78.46) - open/close 2991 (time 1788528368.58 total 40.02 last 89.46) - open/close 3713 (time 1788528378.60 total 50.03 last 72.10) - open/close 4472 (time 1788528388.63 total 60.07 last 75.63) total: 4500 open/close in 60.57 seconds: 74.29 ops/second - /unlink 1334 (time 1788528403.91 total 10.00 last 133.39) - /unlink 2738 (time 1788528413.92 total 20.01 last 140.29) - /unlink 4301 (time 1788528423.92 total 30.01 last 156.25) total: 4500 /unlink in 31.73 seconds: 141.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 855 (time 1788528440.37 total 10.01 last 85.41) - open/close 1563 (time 1788528450.39 total 20.03 last 70.67) - open/close 2331 (time 1788528460.42 total 30.06 last 76.58) - open/close 3128 (time 1788528470.42 total 40.06 last 79.69) - open/close 4045 (time 1788528480.43 total 50.07 last 91.64) total: 4500 open/close in 55.62 seconds: 80.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1376 (time 1788528500.66 total 10.00 last 137.54) - /unlink 2652 (time 1788528510.66 total 20.01 last 127.54) - /unlink 3858 (time 1788528520.68 total 30.02 last 120.45) total: 4500 /unlink in 34.94 seconds: 128.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 827 (time 1788528540.23 total 10.03 last 82.49) - open/close 1635 (time 1788528550.24 total 20.03 last 80.76) - open/close 2436 (time 1788528560.25 total 30.04 last 80.02) - open/close 3134 (time 1788528570.25 total 40.05 last 69.77) - open/close 3772 (time 1788528580.29 total 50.08 last 63.59) total: 4500 open/close in 59.89 seconds: 75.14 ops/second - /unlink 1666 (time 1788528604.94 total 10.00 last 166.53) - /unlink 3163 (time 1788528614.94 total 20.01 last 149.64) - /unlink 4394 (time 1788528624.95 total 30.01 last 123.03) total: 4500 /unlink in 31.03 seconds: 145.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 928 (time 1788528641.14 total 10.00 last 92.80) - open/close 1873 (time 1788528651.16 total 20.02 last 94.33) - open/close 2733 (time 1788528661.18 total 30.03 last 85.87) - open/close 3552 (time 1788528671.18 total 40.04 last 81.86) - open/close 4407 (time 1788528681.20 total 50.05 last 85.36) total: 4500 open/close in 51.71 seconds: 87.02 ops/second - /unlink 1728 (time 1788528697.67 total 10.01 last 172.65) - /unlink 3422 (time 1788528707.67 total 20.01 last 169.37) total: 4500 /unlink in 25.83 seconds: 174.21 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 882 (time 1788528727.78 total 10.00 last 88.19) - open/close 1674 (time 1788528737.80 total 20.02 last 79.07) - open/close 2251 (time 1788528747.80 total 30.02 last 57.67) - open/close 2980 (time 1788528757.82 total 40.04 last 72.77) - open/close 3888 (time 1788528767.82 total 50.04 last 90.79) - open/close 4490 (time 1788528777.84 total 60.06 last 60.11) total: 4500 open/close in 60.27 seconds: 74.66 ops/second - /unlink 1249 (time 1788528794.21 total 10.00 last 124.86) - /unlink 2304 (time 1788528804.21 total 20.01 last 105.47) - /unlink 3749 (time 1788528814.21 total 30.01 last 144.48) total: 4500 /unlink in 34.97 seconds: 128.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 839 (time 1788528834.63 total 10.00 last 83.89) - open/close 1650 (time 1788528844.63 total 20.00 last 81.08) - open/close 2607 (time 1788528854.64 total 30.01 last 95.66) - open/close 3524 (time 1788528864.65 total 40.02 last 91.61) total: 4500 open/close in 49.67 seconds: 90.60 ops/second - /unlink 1730 (time 1788528887.93 total 10.00 last 172.93) - /unlink 3572 (time 1788528897.93 total 20.01 last 184.18) total: 4500 /unlink in 26.21 seconds: 171.66 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 961 (time 1788528918.30 total 10.00 last 96.08) - open/close 1868 (time 1788528928.31 total 20.01 last 90.63) - open/close 2801 (time 1788528938.31 total 30.01 last 93.29) - open/close 3754 (time 1788528948.32 total 40.03 last 95.17) total: 4500 open/close in 47.78 seconds: 94.19 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1650 (time 1788528969.91 total 10.00 last 164.97) - /unlink 3302 (time 1788528979.91 total 20.00 last 165.18) total: 4500 /unlink in 27.01 seconds: 166.58 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 653 (time 1788529001.63 total 10.01 last 65.22) - open/close 1530 (time 1788529011.63 total 20.01 last 87.69) - open/close 2411 (time 1788529021.63 total 30.02 last 88.06) - open/close 3278 (time 1788529031.64 total 40.02 last 86.67) - open/close 4060 (time 1788529041.65 total 50.03 last 78.11) total: 4500 open/close in 55.22 seconds: 81.49 ops/second - /unlink 1658 (time 1788529061.22 total 10.00 last 165.74) - /unlink 3003 (time 1788529071.22 total 20.01 last 134.47) - /unlink 4404 (time 1788529081.22 total 30.01 last 140.10) total: 4500 /unlink in 30.73 seconds: 146.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 824 (time 1788529095.84 total 10.00 last 82.37) - open/close 1628 (time 1788529105.84 total 20.00 last 80.40) - open/close 2560 (time 1788529115.85 total 30.01 last 93.16) - open/close 3450 (time 1788529125.85 total 40.01 last 88.94) - open/close 4400 (time 1788529135.86 total 50.02 last 94.92) total: 4500 open/close in 51.24 seconds: 87.83 ops/second - /unlink 1537 (time 1788529151.38 total 10.01 last 153.51) - /unlink 3307 (time 1788529161.38 total 20.01 last 176.99) total: 4500 /unlink in 28.02 seconds: 160.62 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95349 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg235-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg235-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg235-server stop mds service on oleg235-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg235-server stop mds service on oleg235-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg235-server unloading modules via unload_modules_local on: 'oleg235-server' oleg235-server: oleg235-server.virtnet: executing unload_modules_local oleg235-server: kmemleak disabled modules unloaded.