== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 13:36:05 (1788456965) Stopping clients: oleg643-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg643-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg643-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg643-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg643-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg643-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg643-server oleg643-server: oleg643-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg643-server' oleg643-server: oleg643-server.virtnet: executing load_modules_local oleg643-server: kmemleak disabled oleg643-server: Loading modules from /home/green/git/lustre-release/lustre oleg643-server: detected 4 online CPUs by sysfs oleg643-server: MODOPTS_LIBCFS= oleg643-server: Force libcfs to create 2 CPU partitions oleg643-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg643-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg643-server: oleg643-server.virtnet: executing set_default_debug -1 all pdsh@oleg643-client: oleg643-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg643-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg643-server: oleg643-server.virtnet: executing set_default_debug -1 all pdsh@oleg643-client: oleg643-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg643-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg643-server: oleg643-server.virtnet: executing set_default_debug -1 all pdsh@oleg643-client: oleg643-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg643-client.virtnet: -o user_xattr,flock 192.168.206.143@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 699 (time 1788457169.46 total 10.01 last 69.84) - open/close 1426 (time 1788457179.46 total 20.01 last 72.69) - open/close 2174 (time 1788457189.46 total 30.01 last 74.80) - open/close 2873 (time 1788457199.47 total 40.02 last 69.84) - open/close 3575 (time 1788457209.48 total 50.03 last 70.15) - open/close 4226 (time 1788457219.48 total 60.03 last 65.06) total: 4500 open/close in 63.84 seconds: 70.49 ops/second - /unlink 1393 (time 1788457236.78 total 10.00 last 139.27) - /unlink 2752 (time 1788457246.79 total 20.01 last 135.74) - /unlink 4037 (time 1788457256.79 total 30.02 last 128.48) total: 4500 /unlink in 33.89 seconds: 132.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 696 (time 1788457275.78 total 10.00 last 69.60) - open/close 1217 (time 1788457285.79 total 20.01 last 52.05) - open/close 1814 (time 1788457295.79 total 30.01 last 59.69) - open/close 2645 (time 1788457305.80 total 40.02 last 83.06) - open/close 3347 (time 1788457315.81 total 50.03 last 70.12) - open/close 3926 (time 1788457325.82 total 60.04 last 57.83) total: 4500 open/close in 68.76 seconds: 65.44 ops/second - /unlink 1487 (time 1788457349.66 total 10.00 last 148.66) - /unlink 2857 (time 1788457359.68 total 20.02 last 136.81) - /unlink 4177 (time 1788457369.68 total 30.02 last 131.98) total: 4500 /unlink in 32.19 seconds: 139.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 759 (time 1788457387.15 total 10.01 last 75.86) - open/close 1317 (time 1788457397.16 total 20.01 last 55.77) - open/close 1856 (time 1788457407.18 total 30.03 last 53.79) - open/close 2464 (time 1788457417.18 total 40.04 last 60.78) - open/close 3179 (time 1788457427.20 total 50.05 last 71.38) - open/close 3939 (time 1788457437.20 total 60.06 last 75.97) total: 4500 open/close in 68.39 seconds: 65.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1326 (time 1788457459.73 total 10.01 last 132.46) - /unlink 2587 (time 1788457469.74 total 20.01 last 126.07) - /unlink 3880 (time 1788457479.74 total 30.02 last 129.23) total: 4500 /unlink in 34.50 seconds: 130.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 703 (time 1788457499.94 total 10.04 last 70.04) - open/close 1372 (time 1788457509.95 total 20.04 last 66.87) - open/close 2125 (time 1788457519.95 total 30.04 last 75.29) - open/close 2900 (time 1788457529.96 total 40.06 last 77.40) - open/close 3658 (time 1788457539.97 total 50.06 last 75.76) - open/close 4428 (time 1788457549.98 total 60.07 last 76.90) total: 4500 open/close in 61.01 seconds: 73.76 ops/second - /unlink 1277 (time 1788457565.57 total 10.00 last 127.64) - /unlink 2332 (time 1788457575.57 total 20.01 last 105.46) - /unlink 3342 (time 1788457585.57 total 30.01 last 100.98) total: 4500 /unlink in 38.66 seconds: 116.39 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 679 (time 1788457609.06 total 10.02 last 67.78) - open/close 1376 (time 1788457619.06 total 20.02 last 69.69) - open/close 2018 (time 1788457629.08 total 30.04 last 64.04) - open/close 2597 (time 1788457639.12 total 40.08 last 57.67) - open/close 3188 (time 1788457649.14 total 50.11 last 58.97) - open/close 3813 (time 1788457659.15 total 60.12 last 62.44) - open/close 4318 (time 1788457669.16 total 70.12 last 50.45) total: 4500 open/close in 73.19 seconds: 61.48 ops/second - /unlink 1134 (time 1788457688.02 total 10.00 last 113.34) - /unlink 2372 (time 1788457698.03 total 20.01 last 123.78) - /unlink 3723 (time 1788457708.04 total 30.02 last 134.93) total: 4500 /unlink in 36.01 seconds: 124.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 619 (time 1788457730.05 total 10.00 last 61.89) - open/close 1279 (time 1788457740.07 total 20.02 last 65.86) - open/close 1884 (time 1788457750.07 total 30.03 last 60.48) - open/close 2570 (time 1788457760.08 total 40.03 last 68.56) - open/close 3262 (time 1788457770.08 total 50.03 last 69.19) - open/close 3908 (time 1788457780.09 total 60.04 last 64.55) total: 4500 open/close in 68.60 seconds: 65.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1216 (time 1788457804.48 total 10.01 last 121.51) - /unlink 2483 (time 1788457814.48 total 20.01 last 126.67) - /unlink 3811 (time 1788457824.49 total 30.02 last 132.65) total: 4500 /unlink in 35.43 seconds: 127.00 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 651 (time 1788457845.14 total 10.00 last 65.09) - open/close 1322 (time 1788457855.15 total 20.01 last 67.02) - open/close 1915 (time 1788457865.16 total 30.02 last 59.26) - open/close 2570 (time 1788457875.18 total 40.04 last 65.35) - open/close 3280 (time 1788457885.19 total 50.05 last 70.97) - open/close 3909 (time 1788457895.21 total 60.06 last 62.78) total: 4500 open/close in 69.78 seconds: 64.48 ops/second - /unlink 1338 (time 1788457919.85 total 10.00 last 133.74) - /unlink 2601 (time 1788457929.85 total 20.01 last 126.28) - /unlink 3697 (time 1788457939.85 total 30.01 last 109.60) total: 4500 /unlink in 37.06 seconds: 121.43 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 561 (time 1788457964.65 total 10.00 last 56.07) - open/close 1189 (time 1788457974.66 total 20.01 last 62.74) - open/close 1879 (time 1788457984.66 total 30.02 last 68.96) - open/close 2443 (time 1788457994.67 total 40.03 last 56.37) - open/close 3041 (time 1788458004.68 total 50.04 last 59.74) - open/close 3748 (time 1788458014.70 total 60.05 last 70.58) - open/close 4295 (time 1788458024.71 total 70.07 last 54.63) total: 4500 open/close in 74.16 seconds: 60.68 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1255 (time 1788458044.35 total 10.01 last 125.42) - /unlink 2591 (time 1788458054.36 total 20.02 last 133.42) - /unlink 3580 (time 1788458064.37 total 30.03 last 98.84) total: 4500 /unlink in 38.93 seconds: 115.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 566 (time 1788458088.84 total 10.01 last 56.57) - open/close 1174 (time 1788458098.86 total 20.02 last 60.71) - open/close 1812 (time 1788458108.87 total 30.03 last 63.73) - open/close 2576 (time 1788458118.88 total 40.04 last 76.31) - open/close 3274 (time 1788458128.88 total 50.05 last 69.76) - open/close 3996 (time 1788458138.89 total 60.05 last 72.19) total: 4500 open/close in 66.77 seconds: 67.40 ops/second - /unlink 1362 (time 1788458160.84 total 10.00 last 136.14) - /unlink 2704 (time 1788458170.84 total 20.01 last 134.18) - /unlink 4103 (time 1788458180.85 total 30.01 last 139.77) total: 4500 /unlink in 33.13 seconds: 135.83 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 766 (time 1788458199.25 total 10.01 last 76.54) - open/close 1426 (time 1788458209.26 total 20.02 last 65.89) - open/close 2191 (time 1788458219.27 total 30.03 last 76.42) - open/close 2855 (time 1788458229.28 total 40.04 last 66.34) - open/close 3579 (time 1788458239.28 total 50.04 last 72.40) - open/close 4408 (time 1788458249.30 total 60.06 last 82.80) total: 4500 open/close in 61.15 seconds: 73.59 ops/second - /unlink 1401 (time 1788458266.79 total 10.00 last 140.10) - /unlink 2410 (time 1788458276.79 total 20.00 last 100.90) - /unlink 3899 (time 1788458286.81 total 30.01 last 148.70) total: 4500 /unlink in 34.96 seconds: 128.72 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 675 (time 1788458307.51 total 10.01 last 67.46) - open/close 1412 (time 1788458317.51 total 20.01 last 73.66) - open/close 2123 (time 1788458327.52 total 30.02 last 71.07) - open/close 2820 (time 1788458337.54 total 40.04 last 69.51) - open/close 3541 (time 1788458347.54 total 50.04 last 72.09) - open/close 4344 (time 1788458357.55 total 60.05 last 80.24) total: 4500 open/close in 61.95 seconds: 72.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1397 (time 1788458374.40 total 10.01 last 139.61) - /unlink 2708 (time 1788458384.40 total 20.01 last 131.08) - /unlink 4110 (time 1788458394.40 total 30.01 last 140.18) total: 4500 /unlink in 32.90 seconds: 136.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 797 (time 1788458412.51 total 10.01 last 79.63) - open/close 1506 (time 1788458422.52 total 20.02 last 70.83) - open/close 2109 (time 1788458432.53 total 30.03 last 60.20) - open/close 2776 (time 1788458442.53 total 40.03 last 66.70) - open/close 3496 (time 1788458452.54 total 50.04 last 71.96) - open/close 4225 (time 1788458462.55 total 60.05 last 72.84) total: 4500 open/close in 63.46 seconds: 70.91 ops/second - /unlink 1332 (time 1788458480.69 total 10.00 last 133.17) - /unlink 2564 (time 1788458490.70 total 20.01 last 123.14) - /unlink 4007 (time 1788458500.70 total 30.01 last 144.29) total: 4500 /unlink in 33.91 seconds: 132.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 678 (time 1788458519.91 total 10.00 last 67.80) - open/close 1379 (time 1788458529.91 total 20.01 last 70.07) - open/close 2107 (time 1788458539.92 total 30.02 last 72.71) - open/close 2773 (time 1788458549.93 total 40.02 last 66.59) - open/close 3483 (time 1788458559.93 total 50.03 last 70.96) - open/close 4162 (time 1788458569.94 total 60.03 last 67.87) total: 4500 open/close in 64.72 seconds: 69.53 ops/second - /unlink 1416 (time 1788458589.91 total 10.02 last 141.34) - /unlink 2643 (time 1788458599.91 total 20.02 last 122.69) - /unlink 4174 (time 1788458609.91 total 30.02 last 153.06) total: 4500 /unlink in 33.34 seconds: 134.96 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 585 (time 1788458629.04 total 10.01 last 58.46) - open/close 1276 (time 1788458639.05 total 20.02 last 69.01) - open/close 1995 (time 1788458649.06 total 30.03 last 71.84) - open/close 2766 (time 1788458659.06 total 40.03 last 77.07) - open/close 3469 (time 1788458669.07 total 50.04 last 70.25) - open/close 4209 (time 1788458679.08 total 60.05 last 73.93) total: 4500 open/close in 64.03 seconds: 70.28 ops/second - /unlink 1423 (time 1788458697.65 total 10.01 last 142.18) - /unlink 2762 (time 1788458707.65 total 20.01 last 133.88) - /unlink 4081 (time 1788458717.66 total 30.02 last 131.82) total: 4500 /unlink in 33.30 seconds: 135.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 671 (time 1788458736.68 total 10.01 last 67.01) - open/close 1273 (time 1788458746.68 total 20.02 last 60.17) - open/close 1896 (time 1788458756.69 total 30.02 last 62.27) - open/close 2561 (time 1788458766.69 total 40.02 last 66.49) - open/close 3201 (time 1788458776.69 total 50.03 last 63.98) - open/close 3952 (time 1788458786.69 total 60.03 last 75.07) total: 4500 open/close in 67.17 seconds: 66.99 ops/second - /unlink 1147 (time 1788458807.51 total 10.00 last 114.67) - /unlink 2550 (time 1788458817.51 total 20.01 last 140.26) - /unlink 3915 (time 1788458827.51 total 30.01 last 136.49) total: 4500 /unlink in 34.03 seconds: 132.24 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 757 (time 1788458846.30 total 10.01 last 75.61) - open/close 1511 (time 1788458856.30 total 20.01 last 75.37) - open/close 2261 (time 1788458866.30 total 30.02 last 74.99) - open/close 3010 (time 1788458876.31 total 40.02 last 74.85) - open/close 3792 (time 1788458886.32 total 50.03 last 78.14) total: 4500 open/close in 59.65 seconds: 75.44 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1423 (time 1788458910.18 total 10.00 last 142.26) - /unlink 2856 (time 1788458920.19 total 20.01 last 143.19) - /unlink 4219 (time 1788458930.19 total 30.01 last 136.25) total: 4500 /unlink in 32.47 seconds: 138.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 768 (time 1788458947.23 total 10.02 last 76.66) - open/close 1546 (time 1788458957.24 total 20.03 last 77.74) - open/close 2295 (time 1788458967.24 total 30.03 last 74.90) - open/close 3036 (time 1788458977.25 total 40.04 last 74.00) - open/close 3777 (time 1788458987.27 total 50.05 last 74.01) total: 4500 open/close in 59.85 seconds: 75.18 ops/second - /unlink 1366 (time 1788459011.31 total 10.01 last 136.50) - /unlink 2745 (time 1788459021.32 total 20.01 last 137.84) - /unlink 4107 (time 1788459031.32 total 30.01 last 136.17) total: 4500 /unlink in 32.63 seconds: 137.89 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 781 (time 1788459048.72 total 10.01 last 78.05) - open/close 1487 (time 1788459058.72 total 20.02 last 70.53) - open/close 2207 (time 1788459068.73 total 30.02 last 71.94) - open/close 2970 (time 1788459078.74 total 40.03 last 76.26) - open/close 3726 (time 1788459088.74 total 50.03 last 75.59) - open/close 4434 (time 1788459098.74 total 60.03 last 70.78) total: 4500 open/close in 60.88 seconds: 73.91 ops/second - /unlink 1438 (time 1788459113.67 total 10.00 last 143.77) - /unlink 2752 (time 1788459123.67 total 20.01 last 131.35) - /unlink 4163 (time 1788459133.68 total 30.01 last 141.04) total: 4500 /unlink in 32.65 seconds: 137.84 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95252 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg643-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg643-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg643-server stop mds service on oleg643-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg643-server stop mds service on oleg643-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg643-server unloading modules via unload_modules_local on: 'oleg643-server' oleg643-server: oleg643-server.virtnet: executing unload_modules_local oleg643-server: kmemleak disabled modules unloaded.