== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 17:24:29 (1787865869) Stopping clients: oleg309-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg309-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg309-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg309-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg309-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg309-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg309-server oleg309-server: oleg309-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1049: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg309-server' oleg309-server: oleg309-server.virtnet: executing load_modules_local oleg309-server: Loading modules from /home/green/git/lustre-release/lustre oleg309-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1049: echo: write error: Device or resource busy oleg309-server: detected 4 online CPUs by sysfs oleg309-server: MODOPTS_LIBCFS= oleg309-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg309-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg309-server: oleg309-server.virtnet: executing set_default_debug -1 all pdsh@oleg309-client: oleg309-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg309-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg309-server: oleg309-server.virtnet: executing set_default_debug -1 all pdsh@oleg309-client: oleg309-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg309-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg309-server: oleg309-server.virtnet: executing set_default_debug -1 all pdsh@oleg309-client: oleg309-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg309-client.virtnet: -o user_xattr,flock 192.168.203.109@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 871 (time 1787866068.12 total 10.01 last 87.03) - open/close 1578 (time 1787866078.13 total 20.02 last 70.63) - open/close 2421 (time 1787866088.14 total 30.02 last 84.26) - open/close 3219 (time 1787866098.16 total 40.04 last 79.63) - open/close 4089 (time 1787866108.16 total 50.04 last 86.99) total: 4500 open/close in 54.70 seconds: 82.26 ops/second - /unlink 1447 (time 1787866126.72 total 10.00 last 144.69) - /unlink 3014 (time 1787866136.72 total 20.01 last 156.60) - /unlink 4397 (time 1787866146.73 total 30.01 last 138.25) total: 4500 /unlink in 30.75 seconds: 146.36 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 765 (time 1787866161.66 total 10.01 last 76.43) - open/close 1571 (time 1787866171.66 total 20.01 last 80.57) - open/close 2378 (time 1787866181.66 total 30.01 last 80.68) - open/close 3155 (time 1787866191.67 total 40.02 last 77.65) - open/close 4015 (time 1787866201.67 total 50.02 last 85.98) total: 4500 open/close in 55.69 seconds: 80.81 ops/second - /unlink 1453 (time 1787866221.23 total 10.01 last 145.12) - /unlink 3134 (time 1787866231.24 total 20.02 last 168.01) total: 4500 /unlink in 29.43 seconds: 152.92 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 813 (time 1787866254.99 total 10.01 last 81.25) - open/close 1602 (time 1787866264.99 total 20.01 last 78.85) - open/close 2368 (time 1787866274.99 total 30.01 last 76.59) - open/close 3231 (time 1787866285.01 total 40.02 last 86.20) - open/close 4078 (time 1787866295.01 total 50.03 last 84.69) total: 4500 open/close in 55.49 seconds: 81.10 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1592 (time 1787866314.86 total 10.01 last 159.08) - /unlink 3042 (time 1787866324.87 total 20.02 last 144.85) - /unlink 4390 (time 1787866334.88 total 30.02 last 134.75) total: 4500 /unlink in 30.86 seconds: 145.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 741 (time 1787866349.85 total 10.01 last 74.05) - open/close 1496 (time 1787866359.85 total 20.01 last 75.46) - open/close 2287 (time 1787866369.86 total 30.02 last 79.02) - open/close 3058 (time 1787866379.86 total 40.03 last 77.07) - open/close 3830 (time 1787866389.87 total 50.03 last 77.15) total: 4500 open/close in 58.87 seconds: 76.44 ops/second - /unlink 1607 (time 1787866412.44 total 10.00 last 160.67) - /unlink 3003 (time 1787866422.45 total 20.00 last 139.56) total: 4500 /unlink in 29.68 seconds: 151.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 837 (time 1787866446.44 total 10.00 last 83.67) - open/close 1644 (time 1787866456.44 total 20.01 last 80.69) - open/close 2430 (time 1787866466.45 total 30.02 last 78.52) - open/close 3280 (time 1787866476.45 total 40.02 last 84.97) - open/close 4094 (time 1787866486.45 total 50.02 last 81.37) total: 4500 open/close in 54.61 seconds: 82.40 ops/second - /unlink 1581 (time 1787866504.76 total 10.00 last 158.05) - /unlink 3024 (time 1787866514.76 total 20.00 last 144.28) total: 4500 /unlink in 29.90 seconds: 150.50 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 815 (time 1787866539.32 total 10.01 last 81.43) - open/close 1547 (time 1787866549.33 total 20.02 last 73.15) - open/close 2364 (time 1787866559.34 total 30.03 last 81.58) - open/close 3188 (time 1787866569.34 total 40.03 last 82.39) - open/close 4012 (time 1787866579.36 total 50.05 last 82.27) total: 4500 open/close in 55.97 seconds: 80.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1621 (time 1787866599.62 total 10.00 last 162.07) - /unlink 3176 (time 1787866609.63 total 20.01 last 155.44) total: 4500 /unlink in 29.01 seconds: 155.14 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 824 (time 1787866633.80 total 10.01 last 82.34) - open/close 1601 (time 1787866643.81 total 20.02 last 77.61) - open/close 2488 (time 1787866653.82 total 30.03 last 88.61) - open/close 3274 (time 1787866663.83 total 40.04 last 78.50) - open/close 4099 (time 1787866673.84 total 50.04 last 82.49) total: 4500 open/close in 55.30 seconds: 81.38 ops/second - /unlink 1599 (time 1787866693.35 total 10.01 last 159.78) - /unlink 3074 (time 1787866703.35 total 20.01 last 147.46) - /unlink 4328 (time 1787866713.36 total 30.02 last 125.31) total: 4500 /unlink in 31.68 seconds: 142.05 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 823 (time 1787866730.03 total 10.01 last 82.23) - open/close 1600 (time 1787866740.03 total 20.01 last 77.69) - open/close 2422 (time 1787866750.03 total 30.01 last 82.20) - open/close 3239 (time 1787866760.05 total 40.03 last 81.57) - open/close 4069 (time 1787866770.05 total 50.03 last 82.98) total: 4500 open/close in 55.43 seconds: 81.18 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1573 (time 1787866789.61 total 10.00 last 157.25) - /unlink 2982 (time 1787866799.61 total 20.01 last 140.82) - /unlink 4487 (time 1787866809.61 total 30.01 last 150.48) total: 4500 /unlink in 30.21 seconds: 148.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 787 (time 1787866823.78 total 10.01 last 78.66) - open/close 1631 (time 1787866833.81 total 20.03 last 84.15) - open/close 2463 (time 1787866843.82 total 30.04 last 83.16) - open/close 3244 (time 1787866853.82 total 40.05 last 78.04) - open/close 3977 (time 1787866863.83 total 50.05 last 73.29) total: 4500 open/close in 56.66 seconds: 79.42 ops/second - /unlink 1429 (time 1787866884.86 total 10.01 last 142.82) - /unlink 2883 (time 1787866894.86 total 20.01 last 145.40) - /unlink 4344 (time 1787866904.86 total 30.01 last 146.09) total: 4500 /unlink in 31.13 seconds: 144.58 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 777 (time 1787866919.81 total 10.01 last 77.66) - open/close 1587 (time 1787866929.81 total 20.01 last 80.95) - open/close 2410 (time 1787866939.82 total 30.02 last 82.25) - open/close 3196 (time 1787866949.83 total 40.02 last 78.54) - open/close 3982 (time 1787866959.83 total 50.03 last 78.59) total: 4500 open/close in 56.52 seconds: 79.62 ops/second - /unlink 1680 (time 1787866979.77 total 10.00 last 167.92) - /unlink 3132 (time 1787866989.78 total 20.01 last 145.13) total: 4500 /unlink in 29.18 seconds: 154.23 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 754 (time 1787867014.40 total 10.01 last 75.35) - open/close 1620 (time 1787867024.40 total 20.01 last 86.55) - open/close 2470 (time 1787867034.40 total 30.01 last 85.00) - open/close 3232 (time 1787867044.41 total 40.02 last 76.18) - open/close 4018 (time 1787867054.41 total 50.02 last 78.60) total: 4500 open/close in 55.67 seconds: 80.83 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1556 (time 1787867074.36 total 10.01 last 155.52) - /unlink 2948 (time 1787867084.36 total 20.01 last 139.14) - /unlink 4361 (time 1787867094.37 total 30.01 last 141.27) total: 4500 /unlink in 31.09 seconds: 144.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 752 (time 1787867109.91 total 10.00 last 75.20) - open/close 1482 (time 1787867119.93 total 20.01 last 72.90) - open/close 2250 (time 1787867129.93 total 30.02 last 76.75) - open/close 3075 (time 1787867139.95 total 40.04 last 82.37) - open/close 3911 (time 1787867149.95 total 50.04 last 83.59) total: 4500 open/close in 57.48 seconds: 78.29 ops/second - /unlink 1553 (time 1787867171.46 total 10.00 last 155.28) - /unlink 3058 (time 1787867181.46 total 20.00 last 150.47) total: 4500 /unlink in 28.84 seconds: 156.05 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 864 (time 1787867204.97 total 10.03 last 86.15) - open/close 1641 (time 1787867214.97 total 20.03 last 77.67) - open/close 2412 (time 1787867224.99 total 30.05 last 76.98) - open/close 3148 (time 1787867234.99 total 40.05 last 73.59) - open/close 4003 (time 1787867245.00 total 50.06 last 85.39) total: 4500 open/close in 56.21 seconds: 80.06 ops/second - /unlink 1452 (time 1787867265.77 total 10.00 last 145.18) - /unlink 2845 (time 1787867275.78 total 20.01 last 139.13) - /unlink 4295 (time 1787867285.78 total 30.01 last 144.99) total: 4500 /unlink in 31.32 seconds: 143.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 780 (time 1787867301.26 total 10.00 last 77.99) - open/close 1605 (time 1787867311.26 total 20.01 last 82.45) - open/close 2369 (time 1787867321.27 total 30.02 last 76.32) - open/close 3136 (time 1787867331.28 total 40.02 last 76.67) - open/close 3999 (time 1787867341.28 total 50.03 last 86.27) total: 4500 open/close in 56.02 seconds: 80.33 ops/second - /unlink 1576 (time 1787867361.02 total 10.01 last 157.45) - /unlink 2999 (time 1787867371.02 total 20.02 last 142.21) - /unlink 4351 (time 1787867381.03 total 30.02 last 135.16) total: 4500 /unlink in 31.32 seconds: 143.69 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 825 (time 1787867396.88 total 10.01 last 82.38) - open/close 1616 (time 1787867406.88 total 20.02 last 79.09) - open/close 2482 (time 1787867416.89 total 30.02 last 86.56) - open/close 3297 (time 1787867426.89 total 40.03 last 81.46) - open/close 4062 (time 1787867436.90 total 50.03 last 76.49) total: 4500 open/close in 56.06 seconds: 80.27 ops/second - /unlink 1365 (time 1787867457.41 total 10.01 last 136.38) - /unlink 2785 (time 1787867467.41 total 20.01 last 141.95) - /unlink 4271 (time 1787867477.42 total 30.02 last 148.51) total: 4500 /unlink in 31.44 seconds: 143.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 810 (time 1787867492.42 total 10.02 last 80.84) - open/close 1592 (time 1787867502.42 total 20.03 last 78.16) - open/close 2347 (time 1787867512.43 total 30.03 last 75.48) - open/close 3223 (time 1787867522.44 total 40.04 last 87.47) - open/close 3919 (time 1787867532.44 total 50.04 last 69.60) total: 4500 open/close in 57.29 seconds: 78.55 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1574 (time 1787867553.86 total 10.00 last 157.36) - /unlink 3112 (time 1787867563.87 total 20.01 last 153.73) total: 4500 /unlink in 29.50 seconds: 152.56 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 775 (time 1787867587.81 total 10.01 last 77.45) - open/close 1520 (time 1787867597.82 total 20.02 last 74.43) - open/close 2249 (time 1787867607.83 total 30.03 last 72.79) - open/close 2783 (time 1787867617.84 total 40.04 last 53.36) - open/close 3319 (time 1787867627.86 total 50.06 last 53.49) - open/close 3923 (time 1787867637.86 total 60.06 last 60.38) total: 4500 open/close in 66.52 seconds: 67.64 ops/second - /unlink 1510 (time 1787867657.85 total 10.00 last 150.98) - /unlink 3208 (time 1787867667.85 total 20.01 last 169.72) - /unlink 4483 (time 1787867677.86 total 30.01 last 127.48) total: 4500 /unlink in 30.25 seconds: 148.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 679 (time 1787867692.69 total 10.01 last 67.86) - open/close 1432 (time 1787867702.69 total 20.01 last 75.26) - open/close 2162 (time 1787867712.70 total 30.01 last 72.97) - open/close 2939 (time 1787867722.70 total 40.02 last 77.64) - open/close 3706 (time 1787867732.71 total 50.03 last 76.67) - open/close 4461 (time 1787867742.71 total 60.03 last 75.49) total: 4500 open/close in 60.65 seconds: 74.20 ops/second - /unlink 1337 (time 1787867757.66 total 10.00 last 133.69) - /unlink 2665 (time 1787867767.67 total 20.01 last 132.74) - /unlink 3995 (time 1787867777.67 total 30.01 last 132.96) total: 4500 /unlink in 34.02 seconds: 132.27 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11369: 95295 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg309-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg309-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg309-server stop mds service on oleg309-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg309-server stop mds service on oleg309-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg309-server unloading modules via unload_modules_local on: 'oleg309-server' oleg309-server: oleg309-server.virtnet: executing unload_modules_local oleg309-server: modules unloaded.