== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 20:25:49 (1787444749) Stopping clients: oleg153-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg153-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg153-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg153-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg153-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg153-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg153-server oleg153-server: oleg153-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing load_modules_local oleg153-server: Loading modules from /home/green/git/lustre-release/lustre oleg153-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg153-server: detected 4 online CPUs by sysfs oleg153-server: MODOPTS_LIBCFS= oleg153-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg153-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg153-server: oleg153-server.virtnet: executing set_default_debug -1 all pdsh@oleg153-client: oleg153-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg153-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg153-server: oleg153-server.virtnet: executing set_default_debug -1 all pdsh@oleg153-client: oleg153-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg153-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg153-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg153-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg153-server: oleg153-server.virtnet: executing set_default_debug -1 all pdsh@oleg153-client: oleg153-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg153-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg153-client.virtnet: -o user_xattr,flock 192.168.201.153@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 732 (time 1787444991.12 total 10.02 last 73.09) - open/close 1692 (time 1787445001.13 total 20.02 last 95.94) - open/close 2598 (time 1787445011.13 total 30.03 last 90.57) - open/close 3421 (time 1787445021.16 total 40.05 last 82.09) - open/close 4322 (time 1787445031.16 total 50.06 last 90.04) total: 4500 open/close in 52.16 seconds: 86.28 ops/second - /unlink 1536 (time 1787445048.03 total 10.00 last 153.60) - /unlink 3101 (time 1787445058.03 total 20.01 last 156.42) total: 4500 /unlink in 28.62 seconds: 157.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 761 (time 1787445082.36 total 10.00 last 76.06) - open/close 1623 (time 1787445092.37 total 20.01 last 86.14) - open/close 2326 (time 1787445102.38 total 30.02 last 70.20) - open/close 3098 (time 1787445112.40 total 40.04 last 77.07) - open/close 4029 (time 1787445122.41 total 50.05 last 93.00) total: 4500 open/close in 56.12 seconds: 80.18 ops/second - /unlink 1517 (time 1787445143.11 total 10.01 last 151.60) - /unlink 3198 (time 1787445153.11 total 20.01 last 168.10) total: 4500 /unlink in 29.83 seconds: 150.86 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 710 (time 1787445178.94 total 10.01 last 70.96) - open/close 1608 (time 1787445188.94 total 20.01 last 89.78) - open/close 2489 (time 1787445198.95 total 30.01 last 88.08) - open/close 3300 (time 1787445208.95 total 40.01 last 81.08) - open/close 4206 (time 1787445218.96 total 50.03 last 90.49) total: 4500 open/close in 52.93 seconds: 85.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1616 (time 1787445236.43 total 10.01 last 161.50) - /unlink 3213 (time 1787445246.44 total 20.01 last 159.62) - /unlink 4145 (time 1787445256.44 total 30.02 last 93.16) total: 4500 /unlink in 33.68 seconds: 133.63 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 542 (time 1787445277.65 total 10.06 last 53.87) - open/close 1380 (time 1787445287.67 total 20.08 last 83.62) - open/close 2022 (time 1787445297.67 total 30.08 last 64.19) - open/close 2802 (time 1787445307.67 total 40.08 last 78.00) - open/close 3593 (time 1787445317.67 total 50.09 last 79.09) - open/close 4428 (time 1787445327.67 total 60.09 last 83.50) total: 4500 open/close in 60.91 seconds: 73.88 ops/second - /unlink 1462 (time 1787445343.11 total 10.02 last 145.94) - /unlink 2830 (time 1787445353.11 total 20.02 last 136.80) - /unlink 4488 (time 1787445363.12 total 30.02 last 165.79) total: 4500 /unlink in 30.09 seconds: 149.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 786 (time 1787445378.38 total 10.01 last 78.54) - open/close 1587 (time 1787445388.39 total 20.02 last 80.01) - open/close 2466 (time 1787445398.40 total 30.03 last 87.83) - open/close 3409 (time 1787445408.40 total 40.03 last 94.30) - open/close 4205 (time 1787445418.40 total 50.03 last 79.60) total: 4500 open/close in 53.73 seconds: 83.75 ops/second - /unlink 1788 (time 1787445437.11 total 10.00 last 178.74) - /unlink 3093 (time 1787445447.12 total 20.01 last 130.45) - /unlink 4406 (time 1787445457.12 total 30.01 last 131.25) total: 4500 /unlink in 30.79 seconds: 146.14 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 870 (time 1787445472.33 total 10.01 last 86.93) - open/close 1715 (time 1787445482.33 total 20.01 last 84.46) - open/close 2529 (time 1787445492.34 total 30.02 last 81.35) - open/close 3300 (time 1787445502.35 total 40.02 last 77.06) - open/close 3996 (time 1787445512.38 total 50.06 last 69.37) total: 4500 open/close in 56.28 seconds: 79.95 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1561 (time 1787445533.01 total 10.01 last 156.01) - /unlink 3101 (time 1787445543.02 total 20.01 last 153.95) total: 4500 /unlink in 29.89 seconds: 150.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 879 (time 1787445567.86 total 10.01 last 87.85) - open/close 1788 (time 1787445577.86 total 20.01 last 90.88) - open/close 2543 (time 1787445587.87 total 30.01 last 75.45) - open/close 3250 (time 1787445597.88 total 40.02 last 70.62) - open/close 4008 (time 1787445607.88 total 50.03 last 75.80) total: 4500 open/close in 56.26 seconds: 79.99 ops/second - /unlink 1428 (time 1787445628.15 total 10.00 last 142.79) - /unlink 3013 (time 1787445638.16 total 20.00 last 158.43) - /unlink 4453 (time 1787445648.18 total 30.03 last 143.69) total: 4500 /unlink in 30.40 seconds: 148.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 743 (time 1787445665.40 total 10.01 last 74.21) - open/close 1523 (time 1787445675.41 total 20.01 last 77.98) - open/close 2421 (time 1787445685.41 total 30.02 last 89.79) - open/close 3160 (time 1787445695.41 total 40.02 last 73.86) - open/close 4028 (time 1787445705.42 total 50.02 last 86.79) total: 4500 open/close in 55.48 seconds: 81.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1622 (time 1787445725.45 total 10.01 last 162.07) - /unlink 3293 (time 1787445735.45 total 20.01 last 167.07) total: 4500 /unlink in 27.96 seconds: 160.93 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 891 (time 1787445758.69 total 10.00 last 89.07) - open/close 1695 (time 1787445768.71 total 20.02 last 80.22) - open/close 2540 (time 1787445778.72 total 30.03 last 84.42) - open/close 3318 (time 1787445788.72 total 40.04 last 77.78) - open/close 4158 (time 1787445798.73 total 50.04 last 83.97) total: 4500 open/close in 54.98 seconds: 81.85 ops/second - /unlink 1429 (time 1787445819.63 total 10.00 last 142.83) - /unlink 2928 (time 1787445829.63 total 20.00 last 149.90) - /unlink 4480 (time 1787445839.63 total 30.01 last 155.19) total: 4500 /unlink in 30.16 seconds: 149.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 779 (time 1787445854.36 total 10.00 last 77.88) - open/close 1430 (time 1787445864.37 total 20.01 last 65.06) - open/close 2201 (time 1787445874.37 total 30.01 last 77.09) - open/close 3073 (time 1787445884.38 total 40.02 last 87.13) - open/close 3793 (time 1787445894.38 total 50.02 last 71.97) - open/close 4429 (time 1787445904.40 total 60.04 last 63.52) total: 4500 open/close in 60.82 seconds: 73.98 ops/second - /unlink 1567 (time 1787445920.57 total 10.00 last 156.68) - /unlink 3193 (time 1787445930.58 total 20.00 last 162.56) total: 4500 /unlink in 28.58 seconds: 157.48 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 824 (time 1787445954.74 total 10.00 last 82.39) - open/close 1579 (time 1787445964.74 total 20.00 last 75.48) - open/close 2311 (time 1787445974.74 total 30.01 last 73.18) - open/close 3170 (time 1787445984.76 total 40.02 last 85.81) - open/close 4188 (time 1787445994.78 total 50.04 last 101.60) total: 4500 open/close in 54.32 seconds: 82.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1706 (time 1787446012.59 total 10.00 last 170.56) - /unlink 3368 (time 1787446022.60 total 20.01 last 166.14) total: 4500 /unlink in 27.01 seconds: 166.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 905 (time 1787446044.83 total 10.01 last 90.41) - open/close 1796 (time 1787446054.84 total 20.02 last 89.04) - open/close 2558 (time 1787446064.87 total 30.05 last 75.96) - open/close 3291 (time 1787446074.88 total 40.06 last 73.24) - open/close 4066 (time 1787446084.88 total 50.06 last 77.47) total: 4500 open/close in 54.43 seconds: 82.68 ops/second - /unlink 1889 (time 1787446103.96 total 10.00 last 188.89) - /unlink 3160 (time 1787446113.96 total 20.00 last 127.10) total: 4500 /unlink in 29.25 seconds: 153.86 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 774 (time 1787446137.67 total 10.00 last 77.39) - open/close 1582 (time 1787446147.67 total 20.00 last 80.79) - open/close 2385 (time 1787446157.68 total 30.01 last 80.24) - open/close 3362 (time 1787446167.70 total 40.03 last 97.52) - open/close 4329 (time 1787446177.70 total 50.03 last 96.66) total: 4500 open/close in 52.48 seconds: 85.76 ops/second - /unlink 1614 (time 1787446194.68 total 10.00 last 161.37) - /unlink 3271 (time 1787446204.69 total 20.01 last 165.64) total: 4500 /unlink in 28.57 seconds: 157.50 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 842 (time 1787446227.82 total 10.01 last 84.15) - open/close 1732 (time 1787446237.83 total 20.02 last 88.90) - open/close 2564 (time 1787446247.84 total 30.03 last 83.10) - open/close 3445 (time 1787446257.85 total 40.03 last 88.04) - open/close 4265 (time 1787446267.87 total 50.05 last 81.86) total: 4500 open/close in 53.03 seconds: 84.86 ops/second - /unlink 1397 (time 1787446284.81 total 10.00 last 139.66) - /unlink 2976 (time 1787446294.82 total 20.01 last 157.87) total: 4500 /unlink in 29.50 seconds: 152.55 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 856 (time 1787446318.77 total 10.01 last 85.55) - open/close 1667 (time 1787446328.78 total 20.01 last 81.07) - open/close 2492 (time 1787446338.78 total 30.02 last 82.43) - open/close 3314 (time 1787446348.79 total 40.02 last 82.18) - open/close 4166 (time 1787446358.79 total 50.02 last 85.16) total: 4500 open/close in 53.90 seconds: 83.49 ops/second - /unlink 1671 (time 1787446376.32 total 10.00 last 167.06) - /unlink 2967 (time 1787446386.33 total 20.01 last 129.50) total: 4500 /unlink in 29.74 seconds: 151.30 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 731 (time 1787446411.72 total 10.01 last 73.02) - open/close 1457 (time 1787446421.73 total 20.02 last 72.56) - open/close 2426 (time 1787446431.74 total 30.03 last 96.79) - open/close 3212 (time 1787446441.75 total 40.04 last 78.51) - open/close 4100 (time 1787446451.75 total 50.04 last 88.78) total: 4500 open/close in 55.39 seconds: 81.24 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1540 (time 1787446472.29 total 10.00 last 153.97) - /unlink 2897 (time 1787446482.29 total 20.01 last 135.64) - /unlink 4458 (time 1787446492.29 total 30.01 last 156.05) total: 4500 /unlink in 30.61 seconds: 147.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 875 (time 1787446507.70 total 10.00 last 87.50) - open/close 1723 (time 1787446517.70 total 20.00 last 84.76) - open/close 2620 (time 1787446527.71 total 30.01 last 89.69) - open/close 3586 (time 1787446537.71 total 40.01 last 96.57) total: 4500 open/close in 48.93 seconds: 91.98 ops/second - /unlink 1672 (time 1787446560.70 total 10.00 last 167.13) - /unlink 3542 (time 1787446570.70 total 20.01 last 186.99) total: 4500 /unlink in 26.06 seconds: 172.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 881 (time 1787446591.67 total 10.00 last 88.07) - open/close 1728 (time 1787446601.68 total 20.01 last 84.67) - open/close 2520 (time 1787446611.68 total 30.01 last 79.17) - open/close 3478 (time 1787446621.69 total 40.02 last 95.75) - open/close 4352 (time 1787446631.69 total 50.02 last 87.36) total: 4500 open/close in 51.64 seconds: 87.14 ops/second - /unlink 1719 (time 1787446647.20 total 10.00 last 171.81) - /unlink 3501 (time 1787446657.20 total 20.01 last 178.17) total: 4500 /unlink in 26.55 seconds: 169.49 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11316: 95311 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg153-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg153-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg153-server stop mds service on oleg153-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg153-server stop mds service on oleg153-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg153-server unloading modules via unload_modules_local on: 'oleg153-server' oleg153-server: oleg153-server.virtnet: executing unload_modules_local oleg153-server: modules unloaded.