== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 22:48:22 (1787194102) Stopping clients: oleg624-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg624-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg624-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg624-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg624-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg624-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg624-server oleg624-server: oleg624-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg624-server' oleg624-server: oleg624-server.virtnet: executing load_modules_local oleg624-server: Loading modules from /home/green/git/lustre-release/lustre oleg624-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg624-server: detected 4 online CPUs by sysfs oleg624-server: MODOPTS_LIBCFS= oleg624-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg624-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg624-server: oleg624-server.virtnet: executing set_default_debug -1 all pdsh@oleg624-client: oleg624-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg624-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg624-server: oleg624-server.virtnet: executing set_default_debug -1 all pdsh@oleg624-client: oleg624-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg624-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg624-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg624-server: oleg624-server.virtnet: executing set_default_debug -1 all pdsh@oleg624-client: oleg624-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg624-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg624-client.virtnet: -o user_xattr,flock 192.168.206.124@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 755 (time 1787194286.33 total 10.00 last 75.46) - open/close 1509 (time 1787194296.33 total 20.01 last 75.39) - open/close 2332 (time 1787194306.34 total 30.01 last 82.25) - open/close 3114 (time 1787194316.34 total 40.01 last 78.18) - open/close 3909 (time 1787194326.36 total 50.03 last 79.34) total: 4500 open/close in 57.66 seconds: 78.05 ops/second - /unlink 1551 (time 1787194347.95 total 10.01 last 154.95) - /unlink 3223 (time 1787194357.95 total 20.01 last 167.19) total: 4500 /unlink in 28.22 seconds: 159.45 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 857 (time 1787194379.15 total 10.02 last 85.56) - open/close 1654 (time 1787194389.17 total 20.03 last 79.60) - open/close 2411 (time 1787194399.17 total 30.04 last 75.64) - open/close 3131 (time 1787194409.20 total 40.07 last 71.78) - open/close 3920 (time 1787194419.21 total 50.07 last 78.89) total: 4500 open/close in 57.15 seconds: 78.74 ops/second - /unlink 1493 (time 1787194439.75 total 10.01 last 149.21) - /unlink 2965 (time 1787194449.76 total 20.01 last 147.13) total: 4500 /unlink in 29.82 seconds: 150.92 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 833 (time 1787194472.56 total 10.01 last 83.19) - open/close 1648 (time 1787194482.57 total 20.03 last 81.40) - open/close 2471 (time 1787194492.58 total 30.03 last 82.23) - open/close 3260 (time 1787194502.58 total 40.04 last 78.85) - open/close 4015 (time 1787194512.59 total 50.05 last 75.45) total: 4500 open/close in 56.07 seconds: 80.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1391 (time 1787194532.32 total 10.00 last 139.07) - /unlink 2694 (time 1787194542.32 total 20.01 last 130.26) - /unlink 4095 (time 1787194552.32 total 30.01 last 140.09) total: 4500 /unlink in 33.09 seconds: 135.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 740 (time 1787194569.60 total 10.01 last 73.94) - open/close 1458 (time 1787194579.61 total 20.01 last 71.76) - open/close 2247 (time 1787194589.62 total 30.02 last 78.82) - open/close 3028 (time 1787194599.62 total 40.03 last 78.04) - open/close 3821 (time 1787194609.62 total 50.03 last 79.29) total: 4500 open/close in 57.92 seconds: 77.70 ops/second - /unlink 1616 (time 1787194630.97 total 10.00 last 161.57) - /unlink 3171 (time 1787194640.97 total 20.00 last 155.49) total: 4500 /unlink in 29.30 seconds: 153.58 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 680 (time 1787194664.96 total 10.01 last 67.97) - open/close 1470 (time 1787194674.98 total 20.02 last 78.87) - open/close 2108 (time 1787194685.00 total 30.04 last 63.68) - open/close 2804 (time 1787194695.00 total 40.04 last 69.58) - open/close 3465 (time 1787194705.01 total 50.05 last 66.05) - open/close 4164 (time 1787194715.01 total 60.05 last 69.89) total: 4500 open/close in 64.33 seconds: 69.96 ops/second - /unlink 1887 (time 1787194732.33 total 10.00 last 188.70) - /unlink 3608 (time 1787194742.34 total 20.00 last 172.09) total: 4500 /unlink in 25.25 seconds: 178.23 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 779 (time 1787194760.55 total 10.01 last 77.81) - open/close 1556 (time 1787194770.57 total 20.03 last 77.57) - open/close 2283 (time 1787194780.58 total 30.04 last 72.64) - open/close 3087 (time 1787194790.58 total 40.04 last 80.39) - open/close 3878 (time 1787194800.59 total 50.05 last 79.00) total: 4500 open/close in 58.02 seconds: 77.56 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1503 (time 1787194822.19 total 10.00 last 150.27) - /unlink 2973 (time 1787194832.19 total 20.00 last 146.98) - /unlink 4425 (time 1787194842.19 total 30.01 last 145.17) total: 4500 /unlink in 30.46 seconds: 147.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 747 (time 1787194856.17 total 10.00 last 74.68) - open/close 1543 (time 1787194866.18 total 20.01 last 79.55) - open/close 2283 (time 1787194876.19 total 30.02 last 73.88) - open/close 3073 (time 1787194886.20 total 40.04 last 78.91) - open/close 3822 (time 1787194896.20 total 50.04 last 74.89) total: 4500 open/close in 58.27 seconds: 77.22 ops/second - /unlink 1805 (time 1787194917.64 total 10.01 last 180.38) - /unlink 3409 (time 1787194927.64 total 20.01 last 160.40) total: 4500 /unlink in 27.34 seconds: 164.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 789 (time 1787194948.73 total 10.01 last 78.81) - open/close 1594 (time 1787194958.74 total 20.03 last 80.39) - open/close 2396 (time 1787194968.74 total 30.03 last 80.18) - open/close 3196 (time 1787194978.75 total 40.04 last 79.92) - open/close 4039 (time 1787194988.76 total 50.04 last 84.29) total: 4500 open/close in 55.80 seconds: 80.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1553 (time 1787195008.11 total 10.00 last 155.24) - /unlink 2972 (time 1787195018.13 total 20.02 last 141.69) - /unlink 4331 (time 1787195028.13 total 30.02 last 135.85) total: 4500 /unlink in 31.18 seconds: 144.30 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 720 (time 1787195043.15 total 10.00 last 71.98) - open/close 1451 (time 1787195053.16 total 20.01 last 73.07) - open/close 2229 (time 1787195063.16 total 30.01 last 77.78) - open/close 3002 (time 1787195073.16 total 40.01 last 77.28) - open/close 3779 (time 1787195083.17 total 50.02 last 77.66) total: 4500 open/close in 58.91 seconds: 76.39 ops/second - /unlink 1474 (time 1787195105.98 total 10.00 last 147.33) - /unlink 2906 (time 1787195115.98 total 20.01 last 143.16) - /unlink 4273 (time 1787195125.98 total 30.01 last 136.66) total: 4500 /unlink in 31.99 seconds: 140.67 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 646 (time 1787195143.08 total 10.01 last 64.57) - open/close 1302 (time 1787195153.09 total 20.01 last 65.54) - open/close 2116 (time 1787195163.09 total 30.02 last 81.40) - open/close 2889 (time 1787195173.10 total 40.03 last 77.22) - open/close 3669 (time 1787195183.10 total 50.03 last 77.99) - open/close 4442 (time 1787195193.10 total 60.03 last 77.25) total: 4500 open/close in 60.67 seconds: 74.17 ops/second - /unlink 1421 (time 1787195207.22 total 10.00 last 142.07) - /unlink 2865 (time 1787195217.23 total 20.01 last 144.31) - /unlink 4263 (time 1787195227.23 total 30.01 last 139.76) total: 4500 /unlink in 31.58 seconds: 142.50 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 784 (time 1787195242.79 total 10.01 last 78.34) - open/close 1473 (time 1787195252.81 total 20.03 last 68.75) - open/close 2201 (time 1787195262.82 total 30.04 last 72.73) - open/close 2910 (time 1787195272.83 total 40.05 last 70.82) - open/close 3649 (time 1787195282.83 total 50.05 last 73.90) - open/close 4425 (time 1787195292.84 total 60.05 last 77.57) total: 4500 open/close in 60.93 seconds: 73.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1463 (time 1787195307.08 total 10.00 last 146.28) - /unlink 3016 (time 1787195317.08 total 20.01 last 155.20) total: 4500 /unlink in 29.83 seconds: 150.87 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 821 (time 1787195340.66 total 10.01 last 82.06) - open/close 1652 (time 1787195350.68 total 20.03 last 82.90) - open/close 2469 (time 1787195360.69 total 30.03 last 81.68) - open/close 3246 (time 1787195370.69 total 40.03 last 77.68) - open/close 4064 (time 1787195380.71 total 50.06 last 81.62) total: 4500 open/close in 55.22 seconds: 81.49 ops/second - /unlink 1496 (time 1787195400.02 total 10.01 last 149.46) - /unlink 2821 (time 1787195410.02 total 20.01 last 132.49) - /unlink 4263 (time 1787195420.02 total 30.01 last 144.19) total: 4500 /unlink in 31.92 seconds: 140.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 815 (time 1787195435.98 total 10.00 last 81.47) - open/close 1603 (time 1787195445.98 total 20.01 last 78.78) - open/close 2420 (time 1787195455.99 total 30.01 last 81.67) - open/close 3141 (time 1787195466.02 total 40.04 last 71.89) - open/close 3898 (time 1787195476.02 total 50.04 last 75.68) total: 4500 open/close in 57.74 seconds: 77.94 ops/second - /unlink 1290 (time 1787195498.08 total 10.01 last 128.93) - /unlink 2717 (time 1787195508.08 total 20.01 last 142.67) - /unlink 4076 (time 1787195518.09 total 30.01 last 135.86) total: 4500 /unlink in 33.36 seconds: 134.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 623 (time 1787195537.16 total 10.00 last 62.28) - open/close 1336 (time 1787195547.17 total 20.01 last 71.27) - open/close 2105 (time 1787195557.18 total 30.02 last 76.83) - open/close 2790 (time 1787195567.18 total 40.02 last 68.49) - open/close 3584 (time 1787195577.18 total 50.02 last 79.39) - open/close 4347 (time 1787195587.20 total 60.04 last 76.16) total: 4500 open/close in 61.79 seconds: 72.83 ops/second - /unlink 1542 (time 1787195602.44 total 10.00 last 154.17) - /unlink 3039 (time 1787195612.44 total 20.00 last 149.69) total: 4500 /unlink in 28.79 seconds: 156.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 803 (time 1787195634.76 total 10.00 last 80.26) - open/close 1576 (time 1787195644.77 total 20.01 last 77.24) - open/close 2375 (time 1787195654.78 total 30.02 last 79.86) - open/close 3185 (time 1787195664.78 total 40.02 last 80.99) - open/close 3984 (time 1787195674.78 total 50.02 last 79.88) total: 4500 open/close in 56.87 seconds: 79.13 ops/second - /unlink 1470 (time 1787195696.18 total 10.00 last 146.99) - /unlink 2860 (time 1787195706.18 total 20.00 last 139.00) - /unlink 4261 (time 1787195716.19 total 30.01 last 140.01) total: 4500 /unlink in 31.77 seconds: 141.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 706 (time 1787195733.45 total 10.00 last 70.57) - open/close 1514 (time 1787195743.46 total 20.01 last 80.73) - open/close 2228 (time 1787195753.47 total 30.02 last 71.39) - open/close 3041 (time 1787195763.47 total 40.02 last 81.26) - open/close 3811 (time 1787195773.47 total 50.02 last 76.98) total: 4500 open/close in 59.49 seconds: 75.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1485 (time 1787195797.83 total 10.00 last 148.44) - /unlink 2833 (time 1787195807.83 total 20.00 last 134.79) - /unlink 4313 (time 1787195817.84 total 30.01 last 147.90) total: 4500 /unlink in 31.41 seconds: 143.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 735 (time 1787195833.22 total 10.01 last 73.45) - open/close 1477 (time 1787195843.22 total 20.01 last 74.14) - open/close 2228 (time 1787195853.24 total 30.03 last 75.00) - open/close 2998 (time 1787195863.26 total 40.05 last 76.87) - open/close 3767 (time 1787195873.30 total 50.08 last 76.60) total: 4500 open/close in 59.49 seconds: 75.64 ops/second - /unlink 1399 (time 1787195896.88 total 10.00 last 139.87) - /unlink 2812 (time 1787195906.88 total 20.01 last 141.22) - /unlink 4294 (time 1787195916.89 total 30.02 last 148.04) total: 4500 /unlink in 31.62 seconds: 142.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 831 (time 1787195932.67 total 10.00 last 83.09) - open/close 1587 (time 1787195942.68 total 20.01 last 75.54) - open/close 2363 (time 1787195952.69 total 30.03 last 77.45) - open/close 3079 (time 1787195962.70 total 40.03 last 71.58) - open/close 3822 (time 1787195972.70 total 50.04 last 74.26) total: 4500 open/close in 59.24 seconds: 75.96 ops/second - /unlink 1284 (time 1787195996.75 total 10.00 last 128.37) - /unlink 2675 (time 1787196006.75 total 20.00 last 139.08) - /unlink 4053 (time 1787196016.75 total 30.01 last 137.77) total: 4500 /unlink in 33.27 seconds: 135.26 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11316: 95188 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg624-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg624-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg624-server stop mds service on oleg624-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg624-server stop mds service on oleg624-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg624-server unloading modules via unload_modules_local on: 'oleg624-server' oleg624-server: oleg624-server.virtnet: executing unload_modules_local oleg624-server: modules unloaded.