== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 09:05:44 (1788440744) Stopping clients: oleg446-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg446-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg446-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg446-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg446-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg446-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg446-server oleg446-server: oleg446-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg446-server' oleg446-server: oleg446-server.virtnet: executing load_modules_local oleg446-server: kmemleak disabled oleg446-server: Loading modules from /home/green/git/lustre-release/lustre oleg446-server: detected 4 online CPUs by sysfs oleg446-server: MODOPTS_LIBCFS= oleg446-server: Force libcfs to create 2 CPU partitions oleg446-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg446-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg446-server: oleg446-server.virtnet: executing set_default_debug -1 all pdsh@oleg446-client: oleg446-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg446-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg446-server: oleg446-server.virtnet: executing set_default_debug -1 all pdsh@oleg446-client: oleg446-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg446-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg446-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg446-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg446-server: oleg446-server.virtnet: executing set_default_debug -1 all pdsh@oleg446-client: oleg446-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg446-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg446-client.virtnet: -o user_xattr,flock 192.168.204.146@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 775 (time 1788440953.08 total 10.00 last 77.49) - open/close 1564 (time 1788440963.09 total 20.01 last 78.87) - open/close 2317 (time 1788440973.12 total 30.04 last 75.03) - open/close 2970 (time 1788440983.13 total 40.05 last 65.25) - open/close 3560 (time 1788440993.13 total 50.05 last 58.99) - open/close 4198 (time 1788441003.14 total 60.06 last 63.75) total: 4500 open/close in 64.59 seconds: 69.67 ops/second - /unlink 1527 (time 1788441022.66 total 10.00 last 152.69) - /unlink 2871 (time 1788441032.67 total 20.01 last 134.33) - /unlink 4359 (time 1788441042.68 total 30.01 last 148.71) total: 4500 /unlink in 31.47 seconds: 142.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 795 (time 1788441059.47 total 10.02 last 79.38) - open/close 1569 (time 1788441069.49 total 20.04 last 77.22) - open/close 2420 (time 1788441079.49 total 30.04 last 85.08) - open/close 3161 (time 1788441089.49 total 40.04 last 74.09) - open/close 3938 (time 1788441099.51 total 50.06 last 77.56) total: 4500 open/close in 57.86 seconds: 77.78 ops/second - /unlink 1401 (time 1788441122.01 total 10.00 last 140.08) - /unlink 2795 (time 1788441132.01 total 20.01 last 139.31) - /unlink 4291 (time 1788441142.02 total 30.01 last 149.53) total: 4500 /unlink in 31.74 seconds: 141.76 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 742 (time 1788441159.05 total 10.01 last 74.13) - open/close 1481 (time 1788441169.06 total 20.02 last 73.82) - open/close 2323 (time 1788441179.07 total 30.03 last 84.11) - open/close 3079 (time 1788441189.09 total 40.05 last 75.47) - open/close 3822 (time 1788441199.10 total 50.06 last 74.21) total: 4500 open/close in 58.63 seconds: 76.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1324 (time 1788441222.22 total 10.01 last 132.31) - /unlink 2541 (time 1788441232.23 total 20.01 last 121.60) - /unlink 3880 (time 1788441242.23 total 30.01 last 133.90) total: 4500 /unlink in 33.86 seconds: 132.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 724 (time 1788441260.42 total 10.01 last 72.35) - open/close 1322 (time 1788441270.44 total 20.03 last 59.69) - open/close 1996 (time 1788441280.44 total 30.03 last 67.39) - open/close 2676 (time 1788441290.44 total 40.03 last 67.99) - open/close 3293 (time 1788441300.44 total 50.03 last 61.70) - open/close 3992 (time 1788441310.45 total 60.04 last 69.85) total: 4500 open/close in 67.56 seconds: 66.60 ops/second - /unlink 1447 (time 1788441333.09 total 10.00 last 144.68) - /unlink 2689 (time 1788441343.10 total 20.00 last 124.16) - /unlink 3885 (time 1788441353.11 total 30.02 last 119.44) total: 4500 /unlink in 33.87 seconds: 132.87 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 710 (time 1788441372.25 total 10.01 last 70.96) - open/close 1426 (time 1788441382.26 total 20.01 last 71.54) - open/close 2117 (time 1788441392.26 total 30.02 last 69.08) - open/close 2915 (time 1788441402.27 total 40.03 last 79.70) - open/close 3680 (time 1788441412.28 total 50.03 last 76.46) - open/close 4479 (time 1788441422.29 total 60.04 last 79.81) total: 4500 open/close in 60.33 seconds: 74.59 ops/second - /unlink 1417 (time 1788441437.90 total 10.02 last 141.40) - /unlink 2919 (time 1788441447.90 total 20.02 last 150.17) - /unlink 4268 (time 1788441457.90 total 30.02 last 134.89) total: 4500 /unlink in 31.82 seconds: 141.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 742 (time 1788441475.25 total 10.01 last 74.15) - open/close 1501 (time 1788441485.26 total 20.02 last 75.80) - open/close 2275 (time 1788441495.27 total 30.03 last 77.35) - open/close 3037 (time 1788441505.27 total 40.03 last 76.19) - open/close 3819 (time 1788441515.28 total 50.03 last 78.17) total: 4500 open/close in 58.44 seconds: 77.00 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1416 (time 1788441538.33 total 10.01 last 141.48) - /unlink 2996 (time 1788441548.34 total 20.01 last 157.98) total: 4500 /unlink in 29.73 seconds: 151.39 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 770 (time 1788441573.33 total 10.00 last 76.97) - open/close 1451 (time 1788441583.35 total 20.02 last 67.97) - open/close 2151 (time 1788441593.35 total 30.03 last 69.99) - open/close 2828 (time 1788441603.36 total 40.03 last 67.65) - open/close 3553 (time 1788441613.37 total 50.05 last 72.40) - open/close 4383 (time 1788441623.41 total 60.08 last 82.71) total: 4500 open/close in 61.71 seconds: 72.92 ops/second - /unlink 1450 (time 1788441639.25 total 10.01 last 144.83) - /unlink 2770 (time 1788441649.25 total 20.01 last 131.96) - /unlink 4307 (time 1788441659.25 total 30.02 last 153.66) total: 4500 /unlink in 31.44 seconds: 143.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 739 (time 1788441675.67 total 10.00 last 73.89) - open/close 1504 (time 1788441685.68 total 20.01 last 76.44) - open/close 2232 (time 1788441695.68 total 30.01 last 72.79) - open/close 3035 (time 1788441705.68 total 40.01 last 80.26) - open/close 3761 (time 1788441715.70 total 50.03 last 72.51) total: 4500 open/close in 59.34 seconds: 75.83 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1430 (time 1788441740.07 total 10.01 last 142.93) - /unlink 2744 (time 1788441750.07 total 20.01 last 131.36) - /unlink 4098 (time 1788441760.08 total 30.02 last 135.26) total: 4500 /unlink in 33.24 seconds: 135.40 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 788 (time 1788441779.91 total 10.03 last 78.60) - open/close 1544 (time 1788441789.91 total 20.03 last 75.57) - open/close 2332 (time 1788441799.93 total 30.05 last 78.68) - open/close 3155 (time 1788441809.93 total 40.05 last 82.30) - open/close 4000 (time 1788441819.93 total 50.05 last 84.49) total: 4500 open/close in 56.65 seconds: 79.44 ops/second - /unlink 1435 (time 1788441840.99 total 10.00 last 143.49) - /unlink 2902 (time 1788441850.99 total 20.00 last 146.66) - /unlink 4347 (time 1788441860.99 total 30.01 last 144.46) total: 4500 /unlink in 31.14 seconds: 144.52 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 860 (time 1788441876.85 total 10.00 last 85.98) - open/close 1632 (time 1788441886.85 total 20.01 last 77.17) - open/close 2435 (time 1788441896.86 total 30.01 last 80.24) - open/close 3167 (time 1788441906.87 total 40.02 last 73.15) - open/close 3924 (time 1788441916.87 total 50.02 last 75.68) total: 4500 open/close in 57.33 seconds: 78.50 ops/second - /unlink 1483 (time 1788441938.19 total 10.01 last 148.20) - /unlink 2908 (time 1788441948.19 total 20.01 last 142.48) - /unlink 4392 (time 1788441958.19 total 30.01 last 148.35) total: 4500 /unlink in 31.18 seconds: 144.33 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 729 (time 1788441975.12 total 10.00 last 72.88) - open/close 1480 (time 1788441985.15 total 20.03 last 74.86) - open/close 2209 (time 1788441995.16 total 30.04 last 72.84) - open/close 2978 (time 1788442005.16 total 40.04 last 76.90) - open/close 3813 (time 1788442015.18 total 50.06 last 83.33) total: 4500 open/close in 58.47 seconds: 76.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1538 (time 1788442038.14 total 10.00 last 153.75) - /unlink 3035 (time 1788442048.14 total 20.01 last 149.66) - /unlink 4339 (time 1788442058.14 total 30.01 last 130.40) total: 4500 /unlink in 31.55 seconds: 142.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 769 (time 1788442074.77 total 10.01 last 76.84) - open/close 1509 (time 1788442084.77 total 20.01 last 73.97) - open/close 2334 (time 1788442094.77 total 30.01 last 82.47) - open/close 3146 (time 1788442104.81 total 40.05 last 80.92) - open/close 3886 (time 1788442114.82 total 50.06 last 73.92) total: 4500 open/close in 58.14 seconds: 77.40 ops/second - /unlink 1372 (time 1788442137.72 total 10.00 last 137.16) - /unlink 2811 (time 1788442147.72 total 20.00 last 143.89) - /unlink 4245 (time 1788442157.73 total 30.01 last 143.38) total: 4500 /unlink in 31.88 seconds: 141.14 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 715 (time 1788442175.10 total 10.01 last 71.44) - open/close 1521 (time 1788442185.10 total 20.01 last 80.59) - open/close 2226 (time 1788442195.11 total 30.01 last 70.47) - open/close 2991 (time 1788442205.11 total 40.02 last 76.47) - open/close 3706 (time 1788442215.11 total 50.02 last 71.50) - open/close 4441 (time 1788442225.12 total 60.03 last 73.45) total: 4500 open/close in 60.99 seconds: 73.78 ops/second - /unlink 1427 (time 1788442241.11 total 10.02 last 142.46) - /unlink 2810 (time 1788442251.11 total 20.02 last 138.30) - /unlink 4234 (time 1788442261.11 total 30.02 last 142.38) total: 4500 /unlink in 32.07 seconds: 140.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 799 (time 1788442278.01 total 10.01 last 79.85) - open/close 1499 (time 1788442288.01 total 20.01 last 69.94) - open/close 2193 (time 1788442298.02 total 30.02 last 69.39) - open/close 2995 (time 1788442308.02 total 40.02 last 80.18) - open/close 3751 (time 1788442318.02 total 50.02 last 75.57) total: 4500 open/close in 59.79 seconds: 75.26 ops/second - /unlink 1283 (time 1788442342.45 total 10.00 last 128.29) - /unlink 2646 (time 1788442352.45 total 20.00 last 136.27) - /unlink 3922 (time 1788442362.47 total 30.02 last 127.40) total: 4500 /unlink in 33.84 seconds: 132.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 715 (time 1788442381.32 total 10.01 last 71.46) - open/close 1413 (time 1788442391.32 total 20.01 last 69.79) - open/close 2157 (time 1788442401.32 total 30.01 last 74.36) - open/close 2936 (time 1788442411.33 total 40.02 last 77.84) - open/close 3675 (time 1788442421.34 total 50.03 last 73.84) total: 4500 open/close in 59.97 seconds: 75.03 ops/second - /unlink 1470 (time 1788442445.68 total 10.00 last 146.96) - /unlink 2853 (time 1788442455.68 total 20.00 last 138.28) - /unlink 4251 (time 1788442465.68 total 30.00 last 139.79) total: 4500 /unlink in 31.99 seconds: 140.66 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 780 (time 1788442482.28 total 10.00 last 77.99) - open/close 1522 (time 1788442492.29 total 20.01 last 74.12) - open/close 2299 (time 1788442502.31 total 30.03 last 77.57) - open/close 3093 (time 1788442512.31 total 40.04 last 79.33) - open/close 3884 (time 1788442522.32 total 50.04 last 79.05) total: 4500 open/close in 57.63 seconds: 78.08 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1475 (time 1788442544.39 total 10.00 last 147.43) - /unlink 2887 (time 1788442554.39 total 20.01 last 141.14) - /unlink 4290 (time 1788442564.39 total 30.01 last 140.25) total: 4500 /unlink in 31.63 seconds: 142.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 833 (time 1788442581.19 total 10.02 last 83.17) - open/close 1602 (time 1788442591.19 total 20.02 last 76.86) - open/close 2415 (time 1788442601.21 total 30.04 last 81.18) - open/close 3211 (time 1788442611.21 total 40.04 last 79.58) - open/close 3985 (time 1788442621.22 total 50.05 last 77.34) total: 4500 open/close in 56.19 seconds: 80.08 ops/second - /unlink 1458 (time 1788442641.90 total 10.00 last 145.79) - /unlink 2885 (time 1788442651.90 total 20.01 last 142.61) - /unlink 4214 (time 1788442661.91 total 30.01 last 132.86) total: 4500 /unlink in 31.97 seconds: 140.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 775 (time 1788442678.16 total 10.01 last 77.46) - open/close 1642 (time 1788442688.16 total 20.01 last 86.68) - open/close 2380 (time 1788442698.17 total 30.02 last 73.74) - open/close 3131 (time 1788442708.17 total 40.02 last 75.09) - open/close 3903 (time 1788442718.18 total 50.02 last 77.15) total: 4500 open/close in 57.99 seconds: 77.60 ops/second - /unlink 1447 (time 1788442740.75 total 10.01 last 144.61) - /unlink 2809 (time 1788442750.76 total 20.02 last 136.05) - /unlink 4146 (time 1788442760.76 total 30.02 last 133.65) total: 4500 /unlink in 32.71 seconds: 137.57 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95308 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg446-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg446-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg446-server stop mds service on oleg446-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg446-server stop mds service on oleg446-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg446-server unloading modules via unload_modules_local on: 'oleg446-server' oleg446-server: oleg446-server.virtnet: executing unload_modules_local oleg446-server: kmemleak disabled modules unloaded.