== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 16:55:54 (1787864154) Stopping clients: oleg342-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg342-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg342-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg342-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg342-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg342-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg342-server oleg342-server: oleg342-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1049: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg342-server' oleg342-server: oleg342-server.virtnet: executing load_modules_local oleg342-server: Loading modules from /home/green/git/lustre-release/lustre oleg342-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1049: echo: write error: Device or resource busy oleg342-server: detected 4 online CPUs by sysfs oleg342-server: MODOPTS_LIBCFS= oleg342-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg342-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg342-server: oleg342-server.virtnet: executing set_default_debug -1 all pdsh@oleg342-client: oleg342-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg342-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg342-server: oleg342-server.virtnet: executing set_default_debug -1 all pdsh@oleg342-client: oleg342-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg342-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg342-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg342-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg342-server: oleg342-server.virtnet: executing set_default_debug -1 all pdsh@oleg342-client: oleg342-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg342-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg342-client.virtnet: -o user_xattr,flock 192.168.203.142@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 785 (time 1787864342.08 total 10.01 last 78.45) - open/close 1661 (time 1787864352.08 total 20.01 last 87.59) - open/close 2418 (time 1787864362.09 total 30.01 last 75.65) - open/close 3202 (time 1787864372.09 total 40.02 last 78.38) - open/close 4080 (time 1787864382.10 total 50.02 last 87.77) total: 4500 open/close in 55.09 seconds: 81.69 ops/second - /unlink 1608 (time 1787864401.04 total 10.00 last 160.78) - /unlink 3111 (time 1787864411.04 total 20.01 last 150.21) total: 4500 /unlink in 28.36 seconds: 158.67 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 862 (time 1787864433.71 total 10.00 last 86.18) - open/close 1658 (time 1787864443.72 total 20.01 last 79.51) - open/close 2349 (time 1787864453.72 total 30.02 last 69.08) - open/close 3200 (time 1787864463.72 total 40.02 last 85.09) - open/close 3939 (time 1787864473.74 total 50.03 last 73.79) total: 4500 open/close in 58.22 seconds: 77.29 ops/second - /unlink 1595 (time 1787864496.02 total 10.01 last 159.41) - /unlink 2838 (time 1787864506.03 total 20.02 last 124.17) total: 4500 /unlink in 29.92 seconds: 150.39 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 781 (time 1787864530.45 total 10.02 last 77.91) - open/close 1598 (time 1787864540.45 total 20.03 last 81.66) - open/close 2291 (time 1787864550.46 total 30.04 last 69.26) - open/close 3096 (time 1787864560.47 total 40.04 last 80.44) - open/close 3824 (time 1787864570.47 total 50.05 last 72.76) total: 4500 open/close in 58.90 seconds: 76.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1450 (time 1787864593.91 total 10.00 last 144.97) - /unlink 2747 (time 1787864603.92 total 20.01 last 129.57) - /unlink 4205 (time 1787864613.93 total 30.02 last 145.74) total: 4500 /unlink in 32.03 seconds: 140.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 824 (time 1787864630.09 total 10.01 last 82.34) - open/close 1662 (time 1787864640.09 total 20.01 last 83.78) - open/close 2554 (time 1787864650.10 total 30.02 last 89.11) - open/close 3386 (time 1787864660.11 total 40.03 last 83.14) - open/close 4107 (time 1787864670.11 total 50.03 last 72.07) total: 4500 open/close in 55.20 seconds: 81.52 ops/second - /unlink 1443 (time 1787864689.09 total 10.01 last 144.23) - /unlink 2980 (time 1787864699.09 total 20.01 last 153.65) total: 4500 /unlink in 29.24 seconds: 153.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 758 (time 1787864722.82 total 10.01 last 75.71) - open/close 1581 (time 1787864732.82 total 20.01 last 82.28) - open/close 2258 (time 1787864742.82 total 30.02 last 67.66) - open/close 2966 (time 1787864752.84 total 40.04 last 70.65) - open/close 3728 (time 1787864762.85 total 50.05 last 76.16) total: 4500 open/close in 59.24 seconds: 75.97 ops/second - /unlink 1632 (time 1787864786.01 total 10.00 last 163.14) - /unlink 2894 (time 1787864796.02 total 20.01 last 126.10) - /unlink 4268 (time 1787864806.02 total 30.02 last 137.29) total: 4500 /unlink in 32.27 seconds: 139.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 760 (time 1787864822.65 total 10.02 last 75.86) - open/close 1515 (time 1787864832.66 total 20.03 last 75.45) - open/close 2421 (time 1787864842.66 total 30.03 last 90.58) - open/close 3286 (time 1787864852.67 total 40.04 last 86.42) - open/close 4108 (time 1787864862.68 total 50.05 last 82.11) total: 4500 open/close in 54.68 seconds: 82.30 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1461 (time 1787864881.96 total 10.01 last 145.94) - /unlink 2809 (time 1787864891.97 total 20.02 last 134.73) - /unlink 4314 (time 1787864901.97 total 30.02 last 150.44) total: 4500 /unlink in 31.25 seconds: 144.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 815 (time 1787864917.44 total 10.00 last 81.47) - open/close 1621 (time 1787864927.44 total 20.01 last 80.57) - open/close 2461 (time 1787864937.44 total 30.01 last 84.00) - open/close 3251 (time 1787864947.44 total 40.01 last 78.99) - open/close 3886 (time 1787864957.45 total 50.01 last 63.48) total: 4500 open/close in 56.58 seconds: 79.54 ops/second - /unlink 1408 (time 1787864977.42 total 10.00 last 140.74) - /unlink 2777 (time 1787864987.42 total 20.01 last 136.87) - /unlink 4291 (time 1787864997.43 total 30.01 last 151.36) total: 4500 /unlink in 31.50 seconds: 142.87 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 788 (time 1787865013.74 total 10.01 last 78.69) - open/close 1611 (time 1787865023.75 total 20.02 last 82.23) - open/close 2445 (time 1787865033.76 total 30.03 last 83.29) - open/close 3229 (time 1787865043.76 total 40.04 last 78.39) - open/close 4016 (time 1787865053.77 total 50.04 last 78.63) total: 4500 open/close in 55.58 seconds: 80.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1467 (time 1787865074.12 total 10.01 last 146.61) - /unlink 2700 (time 1787865084.13 total 20.01 last 123.27) - /unlink 4015 (time 1787865094.13 total 30.01 last 131.42) total: 4500 /unlink in 34.43 seconds: 130.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 705 (time 1787865113.51 total 10.00 last 70.48) - open/close 1501 (time 1787865123.51 total 20.01 last 79.58) - open/close 2295 (time 1787865133.52 total 30.01 last 79.37) - open/close 3098 (time 1787865143.53 total 40.02 last 80.21) - open/close 3980 (time 1787865153.53 total 50.02 last 88.20) total: 4500 open/close in 56.81 seconds: 79.20 ops/second - /unlink 1297 (time 1787865175.48 total 10.00 last 129.70) - /unlink 2567 (time 1787865185.48 total 20.00 last 126.97) - /unlink 4048 (time 1787865195.48 total 30.01 last 148.05) total: 4500 /unlink in 33.07 seconds: 136.08 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 762 (time 1787865213.47 total 10.00 last 76.18) - open/close 1543 (time 1787865223.48 total 20.01 last 78.04) - open/close 2350 (time 1787865233.48 total 30.02 last 80.67) - open/close 2994 (time 1787865243.49 total 40.02 last 64.39) - open/close 3789 (time 1787865253.49 total 50.02 last 79.45) total: 4500 open/close in 59.23 seconds: 75.97 ops/second - /unlink 1507 (time 1787865276.97 total 10.00 last 150.67) - /unlink 3055 (time 1787865286.98 total 20.01 last 154.75) - /unlink 4434 (time 1787865296.98 total 30.01 last 137.86) total: 4500 /unlink in 30.57 seconds: 147.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 587 (time 1787865312.19 total 10.00 last 58.70) - open/close 1262 (time 1787865322.21 total 20.03 last 67.32) - open/close 1960 (time 1787865332.21 total 30.03 last 69.79) - open/close 2704 (time 1787865342.22 total 40.03 last 74.39) - open/close 3518 (time 1787865352.24 total 50.05 last 81.21) - open/close 4314 (time 1787865362.25 total 60.06 last 79.55) total: 4500 open/close in 62.45 seconds: 72.05 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1541 (time 1787865379.38 total 10.00 last 154.09) - /unlink 3138 (time 1787865389.38 total 20.00 last 159.65) total: 4500 /unlink in 28.77 seconds: 156.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 837 (time 1787865413.22 total 10.02 last 83.57) - open/close 1558 (time 1787865423.22 total 20.02 last 72.06) - open/close 2050 (time 1787865433.25 total 30.05 last 49.07) - open/close 2680 (time 1787865443.26 total 40.05 last 62.97) - open/close 3480 (time 1787865453.26 total 50.06 last 79.95) - open/close 4304 (time 1787865463.27 total 60.07 last 82.32) total: 4500 open/close in 62.66 seconds: 71.82 ops/second - /unlink 1447 (time 1787865480.67 total 10.00 last 144.67) - /unlink 3005 (time 1787865490.67 total 20.00 last 155.76) total: 4500 /unlink in 29.75 seconds: 151.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 811 (time 1787865514.47 total 10.00 last 81.06) - open/close 1633 (time 1787865524.48 total 20.02 last 82.10) - open/close 2443 (time 1787865534.49 total 30.03 last 80.89) - open/close 3187 (time 1787865544.50 total 40.03 last 74.38) - open/close 3969 (time 1787865554.51 total 50.05 last 78.06) total: 4500 open/close in 55.96 seconds: 80.41 ops/second - /unlink 1439 (time 1787865575.04 total 10.01 last 143.72) - /unlink 2941 (time 1787865585.05 total 20.02 last 150.08) - /unlink 4400 (time 1787865595.06 total 30.03 last 145.80) total: 4500 /unlink in 30.95 seconds: 145.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 877 (time 1787865611.67 total 10.00 last 87.69) - open/close 1549 (time 1787865621.67 total 20.00 last 67.20) - open/close 2411 (time 1787865631.68 total 30.00 last 86.18) - open/close 3245 (time 1787865641.68 total 40.01 last 83.32) - open/close 4129 (time 1787865651.71 total 50.04 last 88.18) total: 4500 open/close in 53.98 seconds: 83.36 ops/second - /unlink 1607 (time 1787865670.24 total 10.00 last 160.63) - /unlink 3100 (time 1787865680.25 total 20.01 last 149.19) total: 4500 /unlink in 28.68 seconds: 156.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 792 (time 1787865703.29 total 10.01 last 79.12) - open/close 1646 (time 1787865713.30 total 20.02 last 85.30) - open/close 2438 (time 1787865723.31 total 30.03 last 79.12) - open/close 3358 (time 1787865733.32 total 40.04 last 91.97) - open/close 4222 (time 1787865743.35 total 50.07 last 86.09) total: 4500 open/close in 54.96 seconds: 81.88 ops/second - /unlink 1642 (time 1787865763.10 total 10.00 last 164.15) - /unlink 3109 (time 1787865773.10 total 20.00 last 146.70) total: 4500 /unlink in 29.13 seconds: 154.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 825 (time 1787865797.48 total 10.01 last 82.46) - open/close 1637 (time 1787865807.49 total 20.02 last 81.10) - open/close 2399 (time 1787865817.50 total 30.03 last 76.14) - open/close 3219 (time 1787865827.50 total 40.03 last 81.97) - open/close 4018 (time 1787865837.53 total 50.06 last 79.69) total: 4500 open/close in 55.79 seconds: 80.66 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1408 (time 1787865857.86 total 10.00 last 140.78) - /unlink 2801 (time 1787865867.86 total 20.00 last 139.29) - /unlink 4370 (time 1787865877.86 total 30.00 last 156.86) total: 4500 /unlink in 30.96 seconds: 145.37 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 841 (time 1787865892.86 total 10.01 last 84.06) - open/close 1681 (time 1787865902.86 total 20.01 last 84.00) - open/close 2531 (time 1787865912.87 total 30.02 last 84.90) - open/close 3387 (time 1787865922.88 total 40.02 last 85.58) - open/close 4249 (time 1787865932.88 total 50.02 last 86.20) total: 4500 open/close in 52.82 seconds: 85.19 ops/second - /unlink 1689 (time 1787865949.77 total 10.01 last 168.76) - /unlink 3091 (time 1787865959.77 total 20.01 last 140.16) total: 4500 /unlink in 29.09 seconds: 154.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 799 (time 1787865983.61 total 10.00 last 79.87) - open/close 1585 (time 1787865993.62 total 20.01 last 78.52) - open/close 2364 (time 1787866003.62 total 30.02 last 77.87) - open/close 3123 (time 1787866013.66 total 40.06 last 75.58) - open/close 3964 (time 1787866023.66 total 50.06 last 84.07) total: 4500 open/close in 56.02 seconds: 80.33 ops/second - /unlink 1628 (time 1787866043.32 total 10.01 last 162.71) - /unlink 3067 (time 1787866053.32 total 20.01 last 143.88) total: 4500 /unlink in 29.55 seconds: 152.29 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11400: 95304 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg342-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg342-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg342-server stop mds service on oleg342-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg342-server stop mds service on oleg342-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg342-server unloading modules via unload_modules_local on: 'oleg342-server' oleg342-server: oleg342-server.virtnet: executing unload_modules_local oleg342-server: modules unloaded.