== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 14:26:46 (1789583206) Stopping clients: oleg148-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg148-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg148-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg148-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg148-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg148-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg148-server oleg148-server: oleg148-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg148-server' oleg148-server: oleg148-server.virtnet: executing load_modules_local oleg148-server: kmemleak disabled oleg148-server: Loading modules from /home/green/git/lustre-release/lustre oleg148-server: detected 4 online CPUs by sysfs oleg148-server: MODOPTS_LIBCFS= oleg148-server: Force libcfs to create 2 CPU partitions oleg148-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg148-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg148-server: oleg148-server.virtnet: executing set_default_debug -1 all pdsh@oleg148-client: oleg148-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg148-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg148-server: oleg148-server.virtnet: executing set_default_debug -1 all pdsh@oleg148-client: oleg148-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg148-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg148-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg148-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg148-server: oleg148-server.virtnet: executing set_default_debug -1 all pdsh@oleg148-client: oleg148-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg148-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg148-client.virtnet: -o user_xattr,flock 192.168.201.148@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 618 (time 1789583420.19 total 10.00 last 61.79) - open/close 1303 (time 1789583430.20 total 20.00 last 68.49) - open/close 2076 (time 1789583440.20 total 30.01 last 77.25) - open/close 2855 (time 1789583450.21 total 40.02 last 77.83) - open/close 3706 (time 1789583460.21 total 50.02 last 85.10) total: 4500 open/close in 59.92 seconds: 75.10 ops/second - /unlink 1753 (time 1789583484.27 total 10.00 last 175.29) - /unlink 3147 (time 1789583494.27 total 20.00 last 139.34) - /unlink 3927 (time 1789583504.28 total 30.01 last 77.97) total: 4500 /unlink in 34.46 seconds: 130.58 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 668 (time 1789583523.86 total 10.02 last 66.68) - open/close 1298 (time 1789583533.87 total 20.03 last 62.92) - open/close 1830 (time 1789583543.88 total 30.04 last 53.16) - open/close 2517 (time 1789583553.89 total 40.05 last 68.64) - open/close 3367 (time 1789583563.89 total 50.05 last 84.96) - open/close 4228 (time 1789583573.90 total 60.05 last 86.06) total: 4500 open/close in 63.73 seconds: 70.61 ops/second - /unlink 1669 (time 1789583591.94 total 10.00 last 166.88) - /unlink 2934 (time 1789583601.94 total 20.00 last 126.47) - /unlink 4403 (time 1789583611.97 total 30.03 last 146.53) total: 4500 /unlink in 31.74 seconds: 141.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 693 (time 1789583630.50 total 10.01 last 69.24) - open/close 1393 (time 1789583640.51 total 20.01 last 69.98) - open/close 2175 (time 1789583650.53 total 30.03 last 78.05) - open/close 3054 (time 1789583660.53 total 40.04 last 87.84) - open/close 3842 (time 1789583670.54 total 50.04 last 78.77) total: 4500 open/close in 57.94 seconds: 77.67 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1538 (time 1789583693.70 total 10.02 last 153.51) - /unlink 2989 (time 1789583703.72 total 20.03 last 144.87) - /unlink 4270 (time 1789583713.73 total 30.04 last 128.04) total: 4500 /unlink in 32.17 seconds: 139.87 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 542 (time 1789583731.75 total 10.02 last 54.11) - open/close 1061 (time 1789583741.76 total 20.03 last 51.85) - open/close 1762 (time 1789583751.76 total 30.03 last 70.09) - open/close 2451 (time 1789583761.77 total 40.04 last 68.84) - open/close 3160 (time 1789583771.78 total 50.05 last 70.82) - open/close 3866 (time 1789583781.78 total 60.05 last 70.59) total: 4500 open/close in 69.87 seconds: 64.40 ops/second - /unlink 1356 (time 1789583806.75 total 10.00 last 135.54) - /unlink 2682 (time 1789583816.76 total 20.01 last 132.51) - /unlink 3886 (time 1789583826.76 total 30.01 last 120.39) total: 4500 /unlink in 35.90 seconds: 125.37 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 668 (time 1789583847.76 total 10.02 last 66.68) - open/close 1169 (time 1789583857.77 total 20.03 last 50.05) - open/close 1720 (time 1789583867.78 total 30.04 last 55.06) - open/close 2367 (time 1789583877.78 total 40.04 last 64.69) - open/close 2991 (time 1789583887.78 total 50.04 last 62.37) - open/close 3706 (time 1789583897.79 total 60.04 last 71.49) - open/close 4375 (time 1789583907.79 total 70.05 last 66.84) total: 4500 open/close in 72.15 seconds: 62.37 ops/second - /unlink 1401 (time 1789583925.11 total 10.02 last 139.77) - /unlink 2390 (time 1789583935.11 total 20.03 last 98.88) - /unlink 3311 (time 1789583945.12 total 30.04 last 91.99) - /unlink 4392 (time 1789583955.12 total 40.04 last 108.09) total: 4500 /unlink in 41.15 seconds: 109.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 482 (time 1789583972.48 total 10.05 last 47.98) - open/close 1038 (time 1789583982.49 total 20.05 last 55.56) - open/close 1688 (time 1789583992.50 total 30.07 last 64.92) - open/close 2346 (time 1789584002.51 total 40.07 last 65.78) - open/close 2951 (time 1789584012.51 total 50.08 last 60.45) - open/close 3701 (time 1789584022.53 total 60.09 last 74.91) - open/close 4422 (time 1789584032.53 total 70.09 last 72.08) total: 4500 open/close in 71.26 seconds: 63.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1379 (time 1789584048.60 total 10.02 last 137.68) - /unlink 2599 (time 1789584058.60 total 20.02 last 121.98) - /unlink 3978 (time 1789584068.60 total 30.02 last 137.81) total: 4500 /unlink in 33.34 seconds: 134.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 791 (time 1789584087.43 total 10.01 last 79.05) - open/close 1575 (time 1789584097.43 total 20.01 last 78.37) - open/close 2345 (time 1789584107.44 total 30.02 last 76.90) - open/close 3030 (time 1789584117.45 total 40.03 last 68.44) - open/close 3714 (time 1789584127.46 total 50.04 last 68.38) - open/close 4424 (time 1789584137.46 total 60.04 last 70.95) total: 4500 open/close in 60.88 seconds: 73.92 ops/second - /unlink 1428 (time 1789584152.58 total 10.01 last 142.70) - /unlink 2874 (time 1789584162.58 total 20.01 last 144.54) - /unlink 4298 (time 1789584172.59 total 30.02 last 142.24) total: 4500 /unlink in 31.44 seconds: 143.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 785 (time 1789584188.49 total 10.02 last 78.38) - open/close 1626 (time 1789584198.50 total 20.02 last 84.07) - open/close 2327 (time 1789584208.50 total 30.02 last 70.08) - open/close 3071 (time 1789584218.50 total 40.02 last 74.39) - open/close 3882 (time 1789584228.51 total 50.03 last 81.06) total: 4500 open/close in 58.32 seconds: 77.17 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1401 (time 1789584251.89 total 10.00 last 140.09) - /unlink 2792 (time 1789584261.90 total 20.01 last 138.99) - /unlink 4143 (time 1789584271.90 total 30.01 last 135.02) total: 4500 /unlink in 32.73 seconds: 137.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 677 (time 1789584288.63 total 10.02 last 67.59) - open/close 1323 (time 1789584298.65 total 20.03 last 64.49) - open/close 2150 (time 1789584308.66 total 30.04 last 82.61) - open/close 2842 (time 1789584318.66 total 40.05 last 69.15) - open/close 3506 (time 1789584328.68 total 50.06 last 66.31) - open/close 4077 (time 1789584338.69 total 60.07 last 57.05) total: 4500 open/close in 66.76 seconds: 67.41 ops/second - /unlink 1425 (time 1789584360.98 total 10.01 last 142.42) - /unlink 2836 (time 1789584370.98 total 20.01 last 141.03) - /unlink 4218 (time 1789584380.99 total 30.01 last 138.17) total: 4500 /unlink in 32.01 seconds: 140.59 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 760 (time 1789584398.41 total 10.01 last 75.96) - open/close 1521 (time 1789584408.42 total 20.01 last 76.04) - open/close 2312 (time 1789584418.45 total 30.04 last 78.90) - open/close 3046 (time 1789584428.46 total 40.05 last 73.32) - open/close 3778 (time 1789584438.47 total 50.06 last 73.12) total: 4500 open/close in 59.56 seconds: 75.55 ops/second - /unlink 1386 (time 1789584463.56 total 10.00 last 138.59) - /unlink 2755 (time 1789584473.56 total 20.00 last 136.87) - /unlink 3985 (time 1789584483.57 total 30.01 last 122.95) total: 4500 /unlink in 33.92 seconds: 132.67 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 649 (time 1789584502.19 total 10.01 last 64.82) - open/close 1293 (time 1789584512.19 total 20.01 last 64.39) - open/close 1974 (time 1789584522.22 total 30.04 last 67.90) - open/close 2731 (time 1789584532.22 total 40.05 last 75.68) - open/close 3491 (time 1789584542.23 total 50.05 last 75.96) - open/close 4229 (time 1789584552.24 total 60.07 last 73.68) total: 4500 open/close in 63.59 seconds: 70.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1325 (time 1789584570.87 total 10.00 last 132.45) - /unlink 2746 (time 1789584580.87 total 20.01 last 142.03) - /unlink 4210 (time 1789584590.89 total 30.02 last 146.18) total: 4500 /unlink in 32.17 seconds: 139.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 695 (time 1789584608.00 total 10.00 last 69.50) - open/close 1401 (time 1789584618.02 total 20.02 last 70.45) - open/close 2157 (time 1789584628.03 total 30.03 last 75.55) - open/close 2951 (time 1789584638.04 total 40.04 last 79.32) - open/close 3676 (time 1789584648.05 total 50.06 last 72.37) - open/close 4494 (time 1789584658.06 total 60.06 last 81.76) total: 4500 open/close in 60.13 seconds: 74.84 ops/second - /unlink 1422 (time 1789584672.80 total 10.00 last 142.19) - /unlink 2747 (time 1789584682.81 total 20.01 last 132.36) - /unlink 3856 (time 1789584692.83 total 30.03 last 110.68) total: 4500 /unlink in 36.16 seconds: 124.43 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 769 (time 1789584715.41 total 10.01 last 76.85) - open/close 1477 (time 1789584725.42 total 20.01 last 70.76) - open/close 2205 (time 1789584735.43 total 30.02 last 72.71) - open/close 2940 (time 1789584745.43 total 40.03 last 73.49) - open/close 3765 (time 1789584755.44 total 50.03 last 82.45) total: 4500 open/close in 59.13 seconds: 76.10 ops/second - /unlink 1110 (time 1789584779.45 total 10.01 last 110.91) - /unlink 2555 (time 1789584789.45 total 20.01 last 144.44) - /unlink 3870 (time 1789584799.45 total 30.02 last 131.46) total: 4500 /unlink in 34.14 seconds: 131.82 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 720 (time 1789584819.10 total 10.01 last 71.95) - open/close 1371 (time 1789584829.10 total 20.01 last 65.10) - open/close 2071 (time 1789584839.11 total 30.02 last 69.92) - open/close 2728 (time 1789584849.14 total 40.04 last 65.53) - open/close 3423 (time 1789584859.14 total 50.05 last 69.47) - open/close 4178 (time 1789584869.15 total 60.06 last 75.45) total: 4500 open/close in 65.17 seconds: 69.05 ops/second - /unlink 1182 (time 1789584889.16 total 10.00 last 118.17) - /unlink 2303 (time 1789584899.17 total 20.01 last 111.98) - /unlink 3531 (time 1789584909.18 total 30.02 last 122.71) total: 4500 /unlink in 38.30 seconds: 117.49 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 567 (time 1789584933.10 total 10.01 last 56.66) - open/close 1304 (time 1789584943.10 total 20.01 last 73.69) - open/close 2112 (time 1789584953.11 total 30.01 last 80.76) - open/close 2845 (time 1789584963.12 total 40.02 last 73.24) - open/close 3596 (time 1789584973.12 total 50.02 last 75.08) - open/close 4236 (time 1789584983.13 total 60.03 last 63.96) total: 4500 open/close in 65.51 seconds: 68.69 ops/second - /unlink 1256 (time 1789585005.42 total 10.01 last 125.51) - /unlink 2446 (time 1789585015.43 total 20.02 last 118.81) - /unlink 3691 (time 1789585025.43 total 30.02 last 124.49) total: 4500 /unlink in 37.18 seconds: 121.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 566 (time 1789585049.47 total 10.00 last 56.59) - open/close 1252 (time 1789585059.49 total 20.02 last 68.47) - open/close 2030 (time 1789585069.50 total 30.04 last 77.67) - open/close 2771 (time 1789585079.50 total 40.04 last 74.09) - open/close 3552 (time 1789585089.51 total 50.04 last 78.07) - open/close 4147 (time 1789585099.51 total 60.05 last 59.46) total: 4500 open/close in 65.43 seconds: 68.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1341 (time 1789585119.82 total 10.02 last 133.86) - /unlink 2648 (time 1789585129.84 total 20.03 last 130.53) - /unlink 4114 (time 1789585139.84 total 30.04 last 146.54) total: 4500 /unlink in 32.38 seconds: 138.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 755 (time 1789585157.04 total 10.00 last 75.49) - open/close 1567 (time 1789585167.04 total 20.00 last 81.18) - open/close 2385 (time 1789585177.05 total 30.02 last 81.69) - open/close 3120 (time 1789585187.06 total 40.02 last 73.46) - open/close 3914 (time 1789585197.06 total 50.03 last 79.35) total: 4500 open/close in 58.15 seconds: 77.38 ops/second - /unlink 1212 (time 1789585219.93 total 10.00 last 121.17) - /unlink 2313 (time 1789585229.94 total 20.01 last 109.98) - /unlink 3558 (time 1789585239.95 total 30.02 last 124.45) total: 4500 /unlink in 38.35 seconds: 117.33 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 700 (time 1789585264.49 total 10.00 last 69.97) - open/close 1463 (time 1789585274.50 total 20.01 last 76.25) - open/close 2241 (time 1789585284.51 total 30.02 last 77.69) - open/close 3002 (time 1789585294.51 total 40.03 last 76.09) - open/close 3835 (time 1789585304.52 total 50.03 last 83.24) total: 4500 open/close in 58.42 seconds: 77.03 ops/second - /unlink 1224 (time 1789585328.71 total 10.01 last 122.24) - /unlink 2385 (time 1789585338.71 total 20.01 last 116.09) - /unlink 3483 (time 1789585348.71 total 30.02 last 109.73) total: 4500 /unlink in 39.71 seconds: 113.32 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11350: 95373 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg148-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg148-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg148-server stop mds service on oleg148-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg148-server stop mds service on oleg148-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg148-server unloading modules via unload_modules_local on: 'oleg148-server' oleg148-server: oleg148-server.virtnet: executing unload_modules_local oleg148-server: kmemleak disabled modules unloaded.