== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 09:24:42 (1788873882) Stopping clients: oleg216-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg216-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg216-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg216-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg216-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg216-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg216-server oleg216-server: oleg216-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg216-server' oleg216-server: oleg216-server.virtnet: executing load_modules_local oleg216-server: kmemleak disabled oleg216-server: Loading modules from /home/green/git/lustre-release/lustre oleg216-server: detected 4 online CPUs by sysfs oleg216-server: MODOPTS_LIBCFS= oleg216-server: Force libcfs to create 2 CPU partitions oleg216-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg216-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg216-server: oleg216-server.virtnet: executing set_default_debug -1 all pdsh@oleg216-client: oleg216-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg216-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg216-server: oleg216-server.virtnet: executing set_default_debug -1 all pdsh@oleg216-client: oleg216-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg216-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg216-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg216-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg216-server: oleg216-server.virtnet: executing set_default_debug -1 all pdsh@oleg216-client: oleg216-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg216-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg216-client.virtnet: -o user_xattr,flock 192.168.202.116@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 876 (time 1788874089.79 total 10.00 last 87.58) - open/close 1688 (time 1788874099.80 total 20.02 last 81.08) - open/close 2349 (time 1788874109.81 total 30.02 last 66.05) - open/close 2997 (time 1788874119.82 total 40.04 last 64.72) - open/close 3594 (time 1788874129.83 total 50.04 last 59.68) - open/close 4360 (time 1788874139.86 total 60.07 last 76.37) total: 4500 open/close in 62.82 seconds: 71.63 ops/second - /unlink 1151 (time 1788874157.68 total 10.01 last 114.99) - /unlink 2632 (time 1788874167.69 total 20.01 last 148.06) - /unlink 4304 (time 1788874177.69 total 30.01 last 167.19) total: 4500 /unlink in 31.35 seconds: 143.52 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 909 (time 1788874193.31 total 10.01 last 90.82) - open/close 1663 (time 1788874203.31 total 20.01 last 75.39) - open/close 2343 (time 1788874213.33 total 30.03 last 67.88) - open/close 3059 (time 1788874223.33 total 40.03 last 71.59) - open/close 3869 (time 1788874233.34 total 50.04 last 80.93) total: 4500 open/close in 58.20 seconds: 77.32 ops/second - /unlink 1587 (time 1788874255.80 total 10.00 last 158.68) - /unlink 3251 (time 1788874265.80 total 20.00 last 166.36) total: 4500 /unlink in 28.12 seconds: 160.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 825 (time 1788874287.90 total 10.00 last 82.48) - open/close 1576 (time 1788874297.91 total 20.01 last 75.02) - open/close 2405 (time 1788874307.91 total 30.02 last 82.88) - open/close 3210 (time 1788874317.93 total 40.03 last 80.36) - open/close 4010 (time 1788874327.93 total 50.04 last 79.98) total: 4500 open/close in 56.14 seconds: 80.16 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1478 (time 1788874348.57 total 10.00 last 147.73) - /unlink 2941 (time 1788874358.57 total 20.01 last 146.28) - /unlink 4326 (time 1788874368.58 total 30.02 last 138.33) total: 4500 /unlink in 31.61 seconds: 142.38 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 734 (time 1788874385.32 total 10.02 last 73.29) - open/close 1578 (time 1788874395.32 total 20.02 last 84.33) - open/close 2451 (time 1788874405.35 total 30.05 last 87.08) - open/close 3088 (time 1788874415.35 total 40.05 last 63.69) - open/close 3715 (time 1788874425.37 total 50.07 last 62.59) - open/close 4280 (time 1788874435.37 total 60.07 last 56.50) total: 4500 open/close in 63.12 seconds: 71.29 ops/second - /unlink 1519 (time 1788874453.15 total 10.00 last 151.87) - /unlink 2827 (time 1788874463.15 total 20.00 last 130.78) - /unlink 4060 (time 1788874473.15 total 30.00 last 123.29) total: 4500 /unlink in 33.37 seconds: 134.83 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 691 (time 1788874493.03 total 10.00 last 69.08) - open/close 1435 (time 1788874503.04 total 20.01 last 74.34) - open/close 2102 (time 1788874513.05 total 30.02 last 66.65) - open/close 2655 (time 1788874523.05 total 40.02 last 55.29) - open/close 3414 (time 1788874533.05 total 50.02 last 75.89) - open/close 4113 (time 1788874543.06 total 60.03 last 69.86) total: 4500 open/close in 66.53 seconds: 67.64 ops/second - /unlink 1349 (time 1788874564.06 total 10.00 last 134.90) - /unlink 2458 (time 1788874574.06 total 20.00 last 110.90) - /unlink 3655 (time 1788874584.07 total 30.01 last 119.59) total: 4500 /unlink in 38.07 seconds: 118.21 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 675 (time 1788874606.99 total 10.01 last 67.44) - open/close 1324 (time 1788874617.01 total 20.03 last 64.78) - open/close 2099 (time 1788874627.02 total 30.04 last 77.43) - open/close 2810 (time 1788874637.03 total 40.04 last 71.05) - open/close 3483 (time 1788874647.04 total 50.05 last 67.24) - open/close 4239 (time 1788874657.04 total 60.05 last 75.60) total: 4500 open/close in 64.78 seconds: 69.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1277 (time 1788874677.85 total 10.00 last 127.70) - /unlink 2354 (time 1788874687.85 total 20.00 last 107.69) - /unlink 3696 (time 1788874697.86 total 30.01 last 134.14) total: 4500 /unlink in 36.13 seconds: 124.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 668 (time 1788874720.02 total 10.02 last 66.68) - open/close 1420 (time 1788874730.03 total 20.02 last 75.15) - open/close 2069 (time 1788874740.03 total 30.03 last 64.88) - open/close 2652 (time 1788874750.04 total 40.04 last 58.22) - open/close 3285 (time 1788874760.06 total 50.05 last 63.22) - open/close 3828 (time 1788874770.06 total 60.06 last 54.26) - open/close 4442 (time 1788874780.08 total 70.08 last 61.29) total: 4500 open/close in 70.79 seconds: 63.57 ops/second - /unlink 821 (time 1788874796.86 total 10.03 last 81.87) - /unlink 1874 (time 1788874806.86 total 20.03 last 105.29) - /unlink 3184 (time 1788874816.86 total 30.03 last 130.98) total: 4500 /unlink in 39.40 seconds: 114.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 608 (time 1788874841.67 total 10.00 last 60.78) - open/close 1292 (time 1788874851.68 total 20.01 last 68.33) - open/close 1976 (time 1788874861.68 total 30.01 last 68.40) - open/close 2557 (time 1788874871.70 total 40.03 last 58.03) - open/close 3093 (time 1788874881.70 total 50.03 last 53.58) - open/close 3739 (time 1788874891.70 total 60.03 last 64.57) - open/close 4284 (time 1788874901.71 total 70.04 last 54.46) total: 4500 open/close in 73.96 seconds: 60.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1523 (time 1788874921.36 total 10.01 last 152.16) - /unlink 2767 (time 1788874931.37 total 20.02 last 124.29) - /unlink 4124 (time 1788874941.37 total 30.02 last 135.63) total: 4500 /unlink in 33.05 seconds: 136.15 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 653 (time 1788874960.03 total 10.00 last 65.28) - open/close 1356 (time 1788874970.03 total 20.01 last 70.26) - open/close 2178 (time 1788874980.04 total 30.01 last 82.17) - open/close 2964 (time 1788874990.05 total 40.03 last 78.48) - open/close 3616 (time 1788875000.06 total 50.04 last 65.12) - open/close 4268 (time 1788875010.07 total 60.05 last 65.16) total: 4500 open/close in 63.37 seconds: 71.01 ops/second - /unlink 1483 (time 1788875028.76 total 10.00 last 148.28) - /unlink 2856 (time 1788875038.76 total 20.00 last 137.28) - /unlink 4343 (time 1788875048.77 total 30.01 last 148.64) total: 4500 /unlink in 31.49 seconds: 142.89 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 794 (time 1788875064.72 total 10.01 last 79.33) - open/close 1545 (time 1788875074.74 total 20.03 last 74.98) - open/close 2294 (time 1788875084.76 total 30.04 last 74.76) - open/close 3105 (time 1788875094.77 total 40.05 last 81.05) - open/close 3881 (time 1788875104.78 total 50.06 last 77.53) total: 4500 open/close in 57.86 seconds: 77.78 ops/second - /unlink 1306 (time 1788875128.06 total 10.00 last 130.57) - /unlink 2640 (time 1788875138.06 total 20.01 last 133.34) - /unlink 3853 (time 1788875148.06 total 30.01 last 121.28) total: 4500 /unlink in 34.88 seconds: 129.00 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 677 (time 1788875168.20 total 10.00 last 67.68) - open/close 1335 (time 1788875178.21 total 20.01 last 65.76) - open/close 2186 (time 1788875188.23 total 30.03 last 84.96) - open/close 2897 (time 1788875198.24 total 40.04 last 71.02) - open/close 3557 (time 1788875208.24 total 50.04 last 65.98) - open/close 4119 (time 1788875218.25 total 60.05 last 56.16) total: 4500 open/close in 67.81 seconds: 66.36 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1433 (time 1788875241.04 total 10.01 last 143.10) - /unlink 2789 (time 1788875251.05 total 20.03 last 135.45) - /unlink 4113 (time 1788875261.06 total 30.03 last 132.29) total: 4500 /unlink in 32.76 seconds: 137.37 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 770 (time 1788875279.04 total 10.02 last 76.81) - open/close 1450 (time 1788875289.05 total 20.03 last 67.98) - open/close 2202 (time 1788875299.07 total 30.05 last 75.03) - open/close 2937 (time 1788875309.08 total 40.06 last 73.41) - open/close 3632 (time 1788875319.08 total 50.06 last 69.49) - open/close 4339 (time 1788875329.13 total 60.12 last 70.33) total: 4500 open/close in 62.21 seconds: 72.34 ops/second - /unlink 1500 (time 1788875346.75 total 10.00 last 149.98) - /unlink 2893 (time 1788875356.76 total 20.01 last 139.21) - /unlink 4367 (time 1788875366.76 total 30.01 last 147.35) total: 4500 /unlink in 30.83 seconds: 145.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 695 (time 1788875382.19 total 10.00 last 69.49) - open/close 1543 (time 1788875392.19 total 20.01 last 84.75) - open/close 2360 (time 1788875402.20 total 30.02 last 81.62) - open/close 3335 (time 1788875412.21 total 40.02 last 97.46) - open/close 4218 (time 1788875422.22 total 50.03 last 88.19) total: 4500 open/close in 52.98 seconds: 84.94 ops/second - /unlink 1683 (time 1788875438.84 total 10.00 last 168.30) - /unlink 3187 (time 1788875448.85 total 20.01 last 150.29) total: 4500 /unlink in 28.31 seconds: 158.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 867 (time 1788875470.82 total 10.00 last 86.69) - open/close 1758 (time 1788875480.82 total 20.01 last 89.06) - open/close 2689 (time 1788875490.82 total 30.01 last 93.07) - open/close 3556 (time 1788875500.83 total 40.01 last 86.68) - open/close 4434 (time 1788875510.83 total 50.02 last 87.76) total: 4500 open/close in 50.65 seconds: 88.85 ops/second - /unlink 1541 (time 1788875525.78 total 10.01 last 153.90) - /unlink 3192 (time 1788875535.78 total 20.01 last 165.09) total: 4500 /unlink in 27.85 seconds: 161.58 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 949 (time 1788875557.92 total 10.01 last 94.84) - open/close 1894 (time 1788875567.93 total 20.01 last 94.45) - open/close 2798 (time 1788875577.93 total 30.01 last 90.39) - open/close 3637 (time 1788875587.93 total 40.02 last 83.87) - open/close 4493 (time 1788875597.94 total 50.02 last 85.56) total: 4500 open/close in 50.07 seconds: 89.87 ops/second - /unlink 1632 (time 1788875611.65 total 10.01 last 163.10) - /unlink 3403 (time 1788875621.65 total 20.01 last 177.09) total: 4500 /unlink in 25.31 seconds: 177.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 974 (time 1788875640.30 total 10.00 last 97.36) - open/close 1940 (time 1788875650.30 total 20.01 last 96.54) - open/close 2970 (time 1788875660.31 total 30.02 last 102.92) - open/close 4100 (time 1788875670.31 total 40.02 last 112.96) total: 4500 open/close in 44.02 seconds: 102.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1806 (time 1788875687.71 total 10.01 last 180.40) - /unlink 3768 (time 1788875697.72 total 20.01 last 196.15) total: 4500 /unlink in 24.38 seconds: 184.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 946 (time 1788875715.34 total 10.00 last 94.56) - open/close 1998 (time 1788875725.35 total 20.01 last 105.13) - open/close 2924 (time 1788875735.36 total 30.02 last 92.50) - open/close 3928 (time 1788875745.39 total 40.06 last 100.06) total: 4500 open/close in 46.28 seconds: 97.24 ops/second - /unlink 2010 (time 1788875764.90 total 10.00 last 200.95) - /unlink 4183 (time 1788875774.90 total 20.00 last 217.29) total: 4500 /unlink in 21.78 seconds: 206.59 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 1033 (time 1788875789.87 total 10.01 last 103.24) - open/close 2014 (time 1788875799.87 total 20.01 last 98.05) - open/close 3002 (time 1788875809.88 total 30.02 last 98.73) - open/close 3949 (time 1788875819.90 total 40.03 last 94.56) total: 4500 open/close in 45.33 seconds: 99.27 ops/second - /unlink 2039 (time 1788875838.10 total 10.00 last 203.83) - /unlink 3901 (time 1788875848.11 total 20.00 last 186.17) total: 4500 /unlink in 23.52 seconds: 191.37 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95296 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg216-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg216-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg216-server stop mds service on oleg216-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg216-server stop mds service on oleg216-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg216-server unloading modules via unload_modules_local on: 'oleg216-server' oleg216-server: oleg216-server.virtnet: executing unload_modules_local oleg216-server: kmemleak disabled modules unloaded.