== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 09:04:16 (1788527056) Stopping clients: oleg234-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg234-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg234-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg234-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg234-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg234-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg234-server oleg234-server: oleg234-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg234-server' oleg234-server: oleg234-server.virtnet: executing load_modules_local oleg234-server: kmemleak disabled oleg234-server: Loading modules from /home/green/git/lustre-release/lustre oleg234-server: detected 4 online CPUs by sysfs oleg234-server: MODOPTS_LIBCFS= oleg234-server: Force libcfs to create 2 CPU partitions oleg234-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg234-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg234-server: oleg234-server.virtnet: executing set_default_debug -1 all pdsh@oleg234-client: oleg234-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg234-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg234-server: oleg234-server.virtnet: executing set_default_debug -1 all pdsh@oleg234-client: oleg234-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg234-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg234-server: oleg234-server.virtnet: executing set_default_debug -1 all pdsh@oleg234-client: oleg234-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg234-client.virtnet: -o user_xattr,flock 192.168.202.134@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 853 (time 1788527264.23 total 10.00 last 85.28) - open/close 1706 (time 1788527274.23 total 20.00 last 85.29) - open/close 2492 (time 1788527284.23 total 30.01 last 78.58) - open/close 3459 (time 1788527294.23 total 40.01 last 96.64) - open/close 4333 (time 1788527304.24 total 50.02 last 87.32) total: 4500 open/close in 51.96 seconds: 86.60 ops/second - /unlink 1678 (time 1788527320.53 total 10.01 last 167.67) - /unlink 3306 (time 1788527330.53 total 20.01 last 162.78) total: 4500 /unlink in 27.17 seconds: 165.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 863 (time 1788527352.42 total 10.02 last 86.11) - open/close 1609 (time 1788527362.42 total 20.03 last 74.57) - open/close 2440 (time 1788527372.44 total 30.04 last 82.96) - open/close 3142 (time 1788527382.45 total 40.05 last 70.15) - open/close 4108 (time 1788527392.45 total 50.05 last 96.56) total: 4500 open/close in 54.51 seconds: 82.56 ops/second - /unlink 1957 (time 1788527411.10 total 10.00 last 195.64) - /unlink 3488 (time 1788527421.11 total 20.01 last 153.06) total: 4500 /unlink in 27.09 seconds: 166.08 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 805 (time 1788527443.12 total 10.01 last 80.44) - open/close 1526 (time 1788527453.13 total 20.02 last 72.01) - open/close 2343 (time 1788527463.14 total 30.03 last 81.64) - open/close 3160 (time 1788527473.16 total 40.05 last 81.54) - open/close 3948 (time 1788527483.16 total 50.05 last 78.79) total: 4500 open/close in 55.88 seconds: 80.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1635 (time 1788527503.92 total 10.00 last 163.45) - /unlink 3180 (time 1788527513.92 total 20.01 last 154.46) total: 4500 /unlink in 28.73 seconds: 156.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 723 (time 1788527537.68 total 10.00 last 72.28) - open/close 1405 (time 1788527547.68 total 20.01 last 68.18) - open/close 2234 (time 1788527557.69 total 30.01 last 82.88) - open/close 2959 (time 1788527567.69 total 40.01 last 72.49) - open/close 3699 (time 1788527577.69 total 50.01 last 73.96) total: 4500 open/close in 59.46 seconds: 75.69 ops/second - /unlink 1540 (time 1788527602.60 total 10.00 last 153.95) - /unlink 3065 (time 1788527612.61 total 20.01 last 152.40) total: 4500 /unlink in 29.24 seconds: 153.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 894 (time 1788527636.56 total 10.00 last 89.40) - open/close 1771 (time 1788527646.57 total 20.01 last 87.65) - open/close 2646 (time 1788527656.57 total 30.01 last 87.47) - open/close 3612 (time 1788527666.57 total 40.02 last 96.55) total: 4500 open/close in 48.80 seconds: 92.20 ops/second - /unlink 1754 (time 1788527689.96 total 10.00 last 175.35) - /unlink 3328 (time 1788527699.96 total 20.00 last 157.38) total: 4500 /unlink in 27.08 seconds: 166.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 789 (time 1788527722.14 total 10.00 last 78.88) - open/close 1586 (time 1788527732.15 total 20.01 last 79.65) - open/close 2462 (time 1788527742.16 total 30.02 last 87.50) - open/close 3339 (time 1788527752.17 total 40.03 last 87.64) - open/close 4200 (time 1788527762.17 total 50.03 last 86.08) total: 4500 open/close in 53.78 seconds: 83.68 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1571 (time 1788527781.02 total 10.00 last 157.07) - /unlink 3190 (time 1788527791.03 total 20.01 last 161.80) total: 4500 /unlink in 28.03 seconds: 160.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 761 (time 1788527814.59 total 10.00 last 76.08) - open/close 1530 (time 1788527824.59 total 20.00 last 76.90) - open/close 2345 (time 1788527834.61 total 30.02 last 81.37) - open/close 3281 (time 1788527844.62 total 40.03 last 93.50) - open/close 4188 (time 1788527854.65 total 50.06 last 90.40) total: 4500 open/close in 53.97 seconds: 83.38 ops/second - /unlink 1568 (time 1788527872.97 total 10.01 last 156.71) - /unlink 3041 (time 1788527882.98 total 20.01 last 147.26) total: 4500 /unlink in 28.81 seconds: 156.18 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 692 (time 1788527907.67 total 10.00 last 69.19) - open/close 1492 (time 1788527917.68 total 20.01 last 79.96) - open/close 2405 (time 1788527927.68 total 30.01 last 91.28) - open/close 3076 (time 1788527937.68 total 40.01 last 67.07) - open/close 3762 (time 1788527947.69 total 50.01 last 68.60) - open/close 4476 (time 1788527957.69 total 60.02 last 71.34) total: 4500 open/close in 60.59 seconds: 74.27 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1358 (time 1788527974.81 total 10.00 last 135.74) - /unlink 3001 (time 1788527984.81 total 20.00 last 164.29) total: 4500 /unlink in 29.62 seconds: 151.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 842 (time 1788528009.36 total 10.01 last 84.11) - open/close 1675 (time 1788528019.36 total 20.01 last 83.27) - open/close 2469 (time 1788528029.37 total 30.02 last 79.36) - open/close 3366 (time 1788528039.37 total 40.02 last 89.65) - open/close 4196 (time 1788528049.38 total 50.03 last 82.93) total: 4500 open/close in 53.24 seconds: 84.53 ops/second - /unlink 1834 (time 1788528066.79 total 10.00 last 183.32) - /unlink 3052 (time 1788528076.80 total 20.01 last 121.67) - /unlink 4253 (time 1788528086.80 total 30.02 last 120.05) total: 4500 /unlink in 32.06 seconds: 140.34 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 667 (time 1788528105.02 total 10.01 last 66.61) - open/close 1393 (time 1788528115.03 total 20.02 last 72.58) - open/close 2294 (time 1788528125.04 total 30.03 last 89.99) - open/close 3182 (time 1788528135.04 total 40.03 last 88.75) - open/close 4001 (time 1788528145.07 total 50.06 last 81.68) total: 4500 open/close in 56.57 seconds: 79.54 ops/second - /unlink 1687 (time 1788528166.14 total 10.01 last 168.59) - /unlink 3468 (time 1788528176.14 total 20.01 last 177.98) total: 4500 /unlink in 26.47 seconds: 170.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 793 (time 1788528196.89 total 10.01 last 79.25) - open/close 1573 (time 1788528206.89 total 20.01 last 77.99) - open/close 2392 (time 1788528216.90 total 30.01 last 81.86) - open/close 3211 (time 1788528226.91 total 40.02 last 81.84) - open/close 3914 (time 1788528236.91 total 50.02 last 70.28) total: 4500 open/close in 56.88 seconds: 79.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1647 (time 1788528258.94 total 10.04 last 164.00) - /unlink 2895 (time 1788528268.95 total 20.05 last 124.73) - /unlink 3711 (time 1788528278.96 total 30.06 last 81.48) total: 4500 /unlink in 37.88 seconds: 118.81 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 753 (time 1788528302.40 total 10.00 last 75.28) - open/close 1480 (time 1788528312.41 total 20.01 last 72.64) - open/close 2342 (time 1788528322.42 total 30.02 last 86.11) - open/close 3281 (time 1788528332.42 total 40.02 last 93.89) - open/close 4078 (time 1788528342.42 total 50.02 last 79.69) total: 4500 open/close in 55.62 seconds: 80.91 ops/second - /unlink 1650 (time 1788528361.68 total 10.00 last 164.93) - /unlink 3210 (time 1788528371.68 total 20.01 last 155.97) total: 4500 /unlink in 27.32 seconds: 164.74 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 805 (time 1788528394.50 total 10.00 last 80.47) - open/close 1587 (time 1788528404.51 total 20.01 last 78.14) - open/close 2446 (time 1788528414.53 total 30.03 last 85.73) - open/close 3300 (time 1788528424.54 total 40.04 last 85.33) - open/close 4155 (time 1788528434.55 total 50.06 last 85.36) total: 4500 open/close in 54.48 seconds: 82.60 ops/second - /unlink 1812 (time 1788528453.36 total 10.00 last 181.18) - /unlink 3310 (time 1788528463.36 total 20.00 last 149.79) total: 4500 /unlink in 27.32 seconds: 164.74 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 1009 (time 1788528485.56 total 10.00 last 100.88) - open/close 1874 (time 1788528495.56 total 20.01 last 86.47) - open/close 2682 (time 1788528505.57 total 30.01 last 80.72) - open/close 3437 (time 1788528515.58 total 40.02 last 75.44) - open/close 4272 (time 1788528525.58 total 50.02 last 83.49) total: 4500 open/close in 52.83 seconds: 85.18 ops/second - /unlink 1706 (time 1788528542.86 total 10.01 last 170.48) - /unlink 3429 (time 1788528552.87 total 20.01 last 172.23) total: 4500 /unlink in 27.42 seconds: 164.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 846 (time 1788528574.96 total 10.01 last 84.54) - open/close 1764 (time 1788528584.97 total 20.02 last 91.67) - open/close 2628 (time 1788528595.00 total 30.04 last 86.20) - open/close 3550 (time 1788528605.00 total 40.05 last 92.19) - open/close 4395 (time 1788528615.01 total 50.06 last 84.36) total: 4500 open/close in 51.18 seconds: 87.93 ops/second - /unlink 1665 (time 1788528630.00 total 10.00 last 166.47) - /unlink 3100 (time 1788528640.00 total 20.00 last 143.45) total: 4500 /unlink in 28.19 seconds: 159.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 937 (time 1788528663.08 total 10.01 last 93.65) - open/close 1894 (time 1788528673.10 total 20.02 last 95.56) - open/close 2680 (time 1788528683.12 total 30.04 last 78.44) - open/close 3615 (time 1788528693.12 total 40.04 last 93.48) total: 4500 open/close in 49.73 seconds: 90.49 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1547 (time 1788528717.33 total 10.01 last 154.59) - /unlink 3128 (time 1788528727.34 total 20.01 last 158.07) total: 4500 /unlink in 28.50 seconds: 157.87 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 815 (time 1788528751.00 total 10.01 last 81.41) - open/close 1532 (time 1788528761.00 total 20.02 last 71.65) - open/close 2295 (time 1788528771.01 total 30.02 last 76.28) - open/close 3047 (time 1788528781.02 total 40.03 last 75.11) - open/close 3745 (time 1788528791.03 total 50.05 last 69.70) - open/close 4459 (time 1788528801.03 total 60.05 last 71.39) total: 4500 open/close in 60.51 seconds: 74.36 ops/second - /unlink 1832 (time 1788528816.77 total 10.01 last 182.99) - /unlink 3689 (time 1788528826.78 total 20.02 last 185.62) total: 4500 /unlink in 24.17 seconds: 186.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 890 (time 1788528845.24 total 10.00 last 88.96) - open/close 1828 (time 1788528855.24 total 20.01 last 93.78) - open/close 2801 (time 1788528865.25 total 30.02 last 97.22) - open/close 3742 (time 1788528875.25 total 40.02 last 94.07) total: 4500 open/close in 48.00 seconds: 93.74 ops/second - /unlink 1510 (time 1788528897.04 total 10.00 last 150.96) - /unlink 3100 (time 1788528907.04 total 20.00 last 158.99) total: 4500 /unlink in 29.12 seconds: 154.52 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95237 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg234-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg234-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg234-server stop mds service on oleg234-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg234-server stop mds service on oleg234-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg234-server unloading modules via unload_modules_local on: 'oleg234-server' oleg234-server: oleg234-server.virtnet: executing unload_modules_local oleg234-server: kmemleak disabled modules unloaded.