== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 22:46:39 (1788921999) Stopping clients: oleg110-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg110-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg110-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg110-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg110-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg110-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg110-server oleg110-server: oleg110-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg110-server' oleg110-server: oleg110-server.virtnet: executing load_modules_local oleg110-server: kmemleak disabled oleg110-server: Loading modules from /home/green/git/lustre-release/lustre oleg110-server: detected 4 online CPUs by sysfs oleg110-server: MODOPTS_LIBCFS= oleg110-server: Force libcfs to create 2 CPU partitions oleg110-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg110-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg110-server: oleg110-server.virtnet: executing set_default_debug -1 all pdsh@oleg110-client: oleg110-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg110-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg110-server: oleg110-server.virtnet: executing set_default_debug -1 all pdsh@oleg110-client: oleg110-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg110-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg110-server: oleg110-server.virtnet: executing set_default_debug -1 all pdsh@oleg110-client: oleg110-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 759 (time 1788922259.82 total 10.00 last 75.88) - open/close 1664 (time 1788922269.83 total 20.00 last 90.47) - open/close 2527 (time 1788922279.83 total 30.01 last 86.29) - open/close 3380 (time 1788922289.83 total 40.01 last 85.29) - open/close 4203 (time 1788922299.83 total 50.01 last 82.27) total: 4500 open/close in 55.25 seconds: 81.45 ops/second - /unlink 1116 (time 1788922321.23 total 10.01 last 111.45) - /unlink 2294 (time 1788922331.23 total 20.02 last 117.75) - /unlink 3442 (time 1788922341.24 total 30.02 last 114.73) total: 4500 /unlink in 36.77 seconds: 122.37 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 885 (time 1788922363.97 total 10.04 last 88.11) - open/close 1750 (time 1788922373.97 total 20.04 last 86.49) - open/close 2608 (time 1788922383.98 total 30.06 last 85.69) - open/close 3409 (time 1788922393.99 total 40.06 last 80.05) - open/close 4198 (time 1788922404.01 total 50.08 last 78.77) total: 4500 open/close in 53.77 seconds: 83.69 ops/second - /unlink 1484 (time 1788922422.94 total 10.00 last 148.37) - /unlink 3100 (time 1788922432.95 total 20.01 last 161.51) total: 4500 /unlink in 28.31 seconds: 158.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 695 (time 1788922457.31 total 10.02 last 69.37) - open/close 1403 (time 1788922467.34 total 20.05 last 70.59) - open/close 2058 (time 1788922477.34 total 30.05 last 65.49) - open/close 2617 (time 1788922487.36 total 40.06 last 55.83) - open/close 3341 (time 1788922497.37 total 50.08 last 72.31) - open/close 4070 (time 1788922507.41 total 60.11 last 72.64) total: 4500 open/close in 65.62 seconds: 68.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1509 (time 1788922527.96 total 10.01 last 150.73) - /unlink 2852 (time 1788922537.96 total 20.01 last 134.26) - /unlink 4222 (time 1788922547.96 total 30.02 last 136.95) total: 4500 /unlink in 32.72 seconds: 137.55 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 598 (time 1788922567.11 total 10.00 last 59.78) - open/close 1388 (time 1788922577.12 total 20.01 last 78.93) - open/close 2045 (time 1788922587.13 total 30.02 last 65.65) - open/close 2676 (time 1788922597.13 total 40.03 last 63.06) - open/close 3255 (time 1788922607.14 total 50.03 last 57.89) - open/close 3887 (time 1788922617.16 total 60.05 last 63.04) - open/close 4344 (time 1788922627.19 total 70.08 last 45.59) total: 4500 open/close in 72.75 seconds: 61.85 ops/second - /unlink 1226 (time 1788922645.31 total 10.00 last 122.57) - /unlink 2228 (time 1788922655.31 total 20.01 last 100.13) - /unlink 3290 (time 1788922665.32 total 30.01 last 106.19) total: 4500 /unlink in 39.06 seconds: 115.21 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 579 (time 1788922691.55 total 10.00 last 57.88) - open/close 1287 (time 1788922701.56 total 20.01 last 70.76) - open/close 1901 (time 1788922711.56 total 30.01 last 61.38) - open/close 2481 (time 1788922721.57 total 40.03 last 57.92) - open/close 3049 (time 1788922731.58 total 50.03 last 56.76) - open/close 3824 (time 1788922741.59 total 60.04 last 77.45) - open/close 4439 (time 1788922751.59 total 70.05 last 61.46) total: 4500 open/close in 70.54 seconds: 63.79 ops/second - /unlink 1530 (time 1788922767.05 total 10.00 last 152.95) - /unlink 3016 (time 1788922777.06 total 20.01 last 148.55) - /unlink 3760 (time 1788922787.06 total 30.01 last 74.36) total: 4500 /unlink in 37.16 seconds: 121.09 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 692 (time 1788922810.00 total 10.00 last 69.17) - open/close 1579 (time 1788922820.01 total 20.01 last 88.66) - open/close 2309 (time 1788922830.01 total 30.01 last 73.00) - open/close 2979 (time 1788922840.02 total 40.02 last 66.92) - open/close 3531 (time 1788922850.03 total 50.03 last 55.17) - open/close 4265 (time 1788922860.04 total 60.04 last 73.31) total: 4500 open/close in 64.50 seconds: 69.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1298 (time 1788922879.62 total 10.00 last 129.77) - /unlink 2605 (time 1788922889.63 total 20.01 last 130.60) - /unlink 3460 (time 1788922899.64 total 30.02 last 85.42) - /unlink 4432 (time 1788922909.64 total 40.02 last 97.17) total: 4500 /unlink in 40.88 seconds: 110.07 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 778 (time 1788922925.59 total 10.02 last 77.67) - open/close 1562 (time 1788922935.62 total 20.05 last 78.15) - open/close 2425 (time 1788922945.63 total 30.05 last 86.27) - open/close 3257 (time 1788922955.64 total 40.06 last 83.13) - open/close 4109 (time 1788922965.66 total 50.08 last 85.00) total: 4500 open/close in 54.54 seconds: 82.51 ops/second - /unlink 1365 (time 1788922985.24 total 10.00 last 136.48) - /unlink 2787 (time 1788922995.24 total 20.00 last 142.17) - /unlink 4104 (time 1788923005.24 total 30.01 last 131.66) total: 4500 /unlink in 32.99 seconds: 136.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 573 (time 1788923024.36 total 10.03 last 57.12) - open/close 1135 (time 1788923034.37 total 20.04 last 56.16) - open/close 1790 (time 1788923044.37 total 30.05 last 65.44) - open/close 2534 (time 1788923054.37 total 40.05 last 74.40) - open/close 3409 (time 1788923064.39 total 50.06 last 87.39) - open/close 4268 (time 1788923074.39 total 60.07 last 85.85) total: 4500 open/close in 62.74 seconds: 71.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1358 (time 1788923091.33 total 10.02 last 135.57) - /unlink 2825 (time 1788923101.33 total 20.02 last 146.70) - /unlink 4346 (time 1788923111.35 total 30.03 last 151.83) total: 4500 /unlink in 31.60 seconds: 142.39 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 819 (time 1788923127.89 total 10.00 last 81.89) - open/close 1647 (time 1788923137.90 total 20.01 last 82.74) - open/close 2429 (time 1788923147.91 total 30.02 last 78.09) - open/close 3223 (time 1788923157.92 total 40.03 last 79.32) - open/close 4085 (time 1788923167.94 total 50.05 last 86.03) total: 4500 open/close in 55.37 seconds: 81.27 ops/second - /unlink 1357 (time 1788923189.35 total 10.01 last 135.58) - /unlink 2758 (time 1788923199.35 total 20.01 last 140.09) - /unlink 3994 (time 1788923209.35 total 30.01 last 123.57) total: 4500 /unlink in 33.26 seconds: 135.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 581 (time 1788923230.20 total 10.00 last 58.08) - open/close 1358 (time 1788923240.21 total 20.01 last 77.62) - open/close 2167 (time 1788923250.21 total 30.02 last 80.87) - open/close 2861 (time 1788923260.22 total 40.02 last 69.36) - open/close 3517 (time 1788923270.22 total 50.02 last 65.59) - open/close 4377 (time 1788923280.22 total 60.03 last 85.96) total: 4500 open/close in 61.48 seconds: 73.19 ops/second - /unlink 1323 (time 1788923297.30 total 10.04 last 131.84) - /unlink 2670 (time 1788923307.30 total 20.04 last 134.62) - /unlink 3863 (time 1788923317.31 total 30.04 last 119.28) total: 4500 /unlink in 33.93 seconds: 132.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 662 (time 1788923339.13 total 10.01 last 66.16) - open/close 1190 (time 1788923349.13 total 20.01 last 52.80) - open/close 1805 (time 1788923359.13 total 30.01 last 61.48) - open/close 2464 (time 1788923369.14 total 40.01 last 65.86) - open/close 3214 (time 1788923379.15 total 50.03 last 74.91) - open/close 3780 (time 1788923389.17 total 60.04 last 56.51) - open/close 4251 (time 1788923399.18 total 70.06 last 47.04) total: 4500 open/close in 73.63 seconds: 61.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1447 (time 1788923419.87 total 10.00 last 144.69) - /unlink 2668 (time 1788923429.87 total 20.00 last 122.07) - /unlink 3708 (time 1788923439.88 total 30.01 last 103.95) total: 4500 /unlink in 37.41 seconds: 120.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 616 (time 1788923464.05 total 10.01 last 61.53) - open/close 1101 (time 1788923474.06 total 20.02 last 48.46) - open/close 1901 (time 1788923484.09 total 30.05 last 79.78) - open/close 2638 (time 1788923494.09 total 40.05 last 73.65) - open/close 3301 (time 1788923504.10 total 50.06 last 66.27) - open/close 4015 (time 1788923514.11 total 60.07 last 71.35) total: 4500 open/close in 66.00 seconds: 68.18 ops/second - /unlink 1268 (time 1788923535.10 total 10.00 last 126.74) - /unlink 2626 (time 1788923545.10 total 20.01 last 135.79) - /unlink 4149 (time 1788923555.10 total 30.01 last 152.28) total: 4500 /unlink in 32.19 seconds: 139.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 912 (time 1788923572.96 total 10.00 last 91.18) - open/close 1767 (time 1788923582.97 total 20.01 last 85.42) - open/close 2548 (time 1788923592.97 total 30.01 last 78.08) - open/close 3346 (time 1788923602.99 total 40.02 last 79.72) - open/close 4020 (time 1788923613.00 total 50.04 last 67.30) total: 4500 open/close in 57.58 seconds: 78.16 ops/second - /unlink 1180 (time 1788923636.93 total 10.00 last 117.97) - /unlink 2193 (time 1788923646.93 total 20.00 last 101.30) - /unlink 3188 (time 1788923656.93 total 30.01 last 99.47) total: 4500 /unlink in 39.97 seconds: 112.59 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 911 (time 1788923682.72 total 10.00 last 91.07) - open/close 1738 (time 1788923692.72 total 20.01 last 82.66) - open/close 2476 (time 1788923702.73 total 30.01 last 73.75) - open/close 3309 (time 1788923712.75 total 40.03 last 83.14) - open/close 3959 (time 1788923722.75 total 50.04 last 64.98) total: 4500 open/close in 58.75 seconds: 76.60 ops/second - /unlink 953 (time 1788923747.49 total 10.00 last 95.27) - /unlink 2210 (time 1788923757.50 total 20.01 last 125.66) - /unlink 3198 (time 1788923767.50 total 30.01 last 98.73) total: 4500 /unlink in 38.66 seconds: 116.39 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 990 (time 1788923791.04 total 10.00 last 98.96) - open/close 1821 (time 1788923801.04 total 20.01 last 83.04) - open/close 2726 (time 1788923811.04 total 30.01 last 90.49) - open/close 3531 (time 1788923821.05 total 40.02 last 80.43) - open/close 4164 (time 1788923831.05 total 50.02 last 63.29) total: 4500 open/close in 53.37 seconds: 84.32 ops/second - /unlink 1565 (time 1788923849.88 total 10.00 last 156.44) - /unlink 3021 (time 1788923859.88 total 20.01 last 145.53) - /unlink 4236 (time 1788923869.88 total 30.01 last 121.45) total: 4500 /unlink in 32.77 seconds: 137.33 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 579 (time 1788923889.34 total 10.00 last 57.89) - open/close 1200 (time 1788923899.34 total 20.00 last 62.08) - open/close 1847 (time 1788923909.35 total 30.01 last 64.67) - open/close 2728 (time 1788923919.36 total 40.02 last 88.00) - open/close 3424 (time 1788923929.37 total 50.03 last 69.55) - open/close 4221 (time 1788923939.37 total 60.04 last 79.64) total: 4500 open/close in 63.91 seconds: 70.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1394 (time 1788923958.44 total 10.02 last 139.15) - /unlink 2756 (time 1788923968.44 total 20.02 last 136.19) - /unlink 3980 (time 1788923978.44 total 30.03 last 122.31) total: 4500 /unlink in 35.09 seconds: 128.24 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 581 (time 1788923999.46 total 10.01 last 58.06) - open/close 1270 (time 1788924009.47 total 20.01 last 68.85) - open/close 1969 (time 1788924019.47 total 30.01 last 69.90) - open/close 2781 (time 1788924029.47 total 40.02 last 81.17) - open/close 3447 (time 1788924039.49 total 50.04 last 66.46) - open/close 4082 (time 1788924049.51 total 60.05 last 63.41) total: 4500 open/close in 66.13 seconds: 68.05 ops/second - /unlink 1322 (time 1788924071.16 total 10.00 last 132.20) - /unlink 2832 (time 1788924081.16 total 20.00 last 150.99) - /unlink 4257 (time 1788924091.17 total 30.01 last 142.40) total: 4500 /unlink in 32.17 seconds: 139.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 764 (time 1788924110.32 total 10.00 last 76.37) - open/close 1431 (time 1788924120.34 total 20.02 last 66.60) - open/close 1992 (time 1788924130.34 total 30.02 last 56.08) - open/close 2750 (time 1788924140.35 total 40.03 last 75.76) - open/close 3630 (time 1788924150.35 total 50.03 last 87.99) - open/close 4412 (time 1788924160.35 total 60.03 last 78.18) total: 4500 open/close in 60.97 seconds: 73.81 ops/second - /unlink 1782 (time 1788924176.10 total 10.00 last 178.20) - /unlink 3275 (time 1788924186.10 total 20.00 last 149.28) total: 4500 /unlink in 27.41 seconds: 164.15 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95044 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg110-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg110-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg110-server stop mds service on oleg110-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg110-server stop mds service on oleg110-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg110-server unloading modules via unload_modules_local on: 'oleg110-server' oleg110-server: oleg110-server.virtnet: executing unload_modules_local oleg110-server: kmemleak disabled modules unloaded.