== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 12:11:48 (1788538308) Stopping clients: oleg332-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg332-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg332-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg332-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg332-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg332-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg332-server oleg332-server: oleg332-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg332-server' oleg332-server: oleg332-server.virtnet: executing load_modules_local oleg332-server: kmemleak disabled oleg332-server: Loading modules from /home/green/git/lustre-release/lustre oleg332-server: detected 4 online CPUs by sysfs oleg332-server: MODOPTS_LIBCFS= oleg332-server: Force libcfs to create 2 CPU partitions oleg332-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg332-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg332-server: oleg332-server.virtnet: executing set_default_debug -1 all pdsh@oleg332-client: oleg332-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg332-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg332-server: oleg332-server.virtnet: executing set_default_debug -1 all pdsh@oleg332-client: oleg332-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg332-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg332-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg332-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg332-server: oleg332-server.virtnet: executing set_default_debug -1 all pdsh@oleg332-client: oleg332-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg332-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg332-client.virtnet: -o user_xattr,flock 192.168.203.132@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_loc=0x1312 fail_val=5 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 859 (time 1788538513.92 total 10.01 last 85.78) - open/close 1715 (time 1788538523.93 total 20.03 last 85.49) - open/close 2484 (time 1788538533.94 total 30.03 last 76.86) - open/close 3335 (time 1788538543.95 total 40.05 last 84.95) - open/close 4156 (time 1788538553.97 total 50.07 last 81.95) total: 4500 open/close in 54.33 seconds: 82.83 ops/second - /unlink 1373 (time 1788538573.44 total 10.00 last 137.29) - /unlink 2864 (time 1788538583.45 total 20.01 last 149.03) - /unlink 4334 (time 1788538593.45 total 30.01 last 147.00) total: 4500 /unlink in 31.32 seconds: 143.66 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 773 (time 1788538610.23 total 10.01 last 77.25) - open/close 1593 (time 1788538620.23 total 20.01 last 81.99) - open/close 2445 (time 1788538630.24 total 30.02 last 85.12) - open/close 3257 (time 1788538640.24 total 40.02 last 81.17) - open/close 4003 (time 1788538650.25 total 50.03 last 74.54) total: 4500 open/close in 56.46 seconds: 79.70 ops/second - /unlink 1365 (time 1788538671.83 total 10.00 last 136.49) - /unlink 2525 (time 1788538681.84 total 20.01 last 115.89) - /unlink 3869 (time 1788538691.84 total 30.01 last 134.35) total: 4500 /unlink in 34.52 seconds: 130.37 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 690 (time 1788538710.57 total 10.02 last 68.84) - open/close 1280 (time 1788538720.59 total 20.04 last 58.90) - open/close 1954 (time 1788538730.60 total 30.05 last 67.35) - open/close 2605 (time 1788538740.60 total 40.05 last 65.08) - open/close 3339 (time 1788538750.60 total 50.06 last 73.36) - open/close 4136 (time 1788538760.60 total 60.06 last 79.70) total: 4500 open/close in 64.45 seconds: 69.82 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1133 (time 1788538779.45 total 10.01 last 113.23) - /unlink 2438 (time 1788538789.45 total 20.01 last 130.50) - /unlink 3659 (time 1788538799.46 total 30.01 last 122.03) total: 4500 /unlink in 35.88 seconds: 125.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 830 (time 1788538819.85 total 10.02 last 82.86) - open/close 1615 (time 1788538829.87 total 20.03 last 78.37) - open/close 2423 (time 1788538839.87 total 30.04 last 80.76) - open/close 3245 (time 1788538849.89 total 40.06 last 82.07) - open/close 3951 (time 1788538859.90 total 50.06 last 70.56) total: 4500 open/close in 56.60 seconds: 79.51 ops/second - /unlink 1339 (time 1788538880.21 total 10.00 last 133.89) - /unlink 2735 (time 1788538890.21 total 20.00 last 139.58) - /unlink 4249 (time 1788538900.21 total 30.00 last 151.38) total: 4500 /unlink in 31.85 seconds: 141.30 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 757 (time 1788538916.41 total 10.01 last 75.59) - open/close 1542 (time 1788538926.41 total 20.02 last 78.45) - open/close 2289 (time 1788538936.43 total 30.04 last 74.58) - open/close 3037 (time 1788538946.44 total 40.05 last 74.70) - open/close 3810 (time 1788538956.47 total 50.07 last 77.12) - open/close 4317 (time 1788538966.47 total 60.08 last 50.69) total: 4500 open/close in 62.60 seconds: 71.89 ops/second - /unlink 1187 (time 1788538984.18 total 10.00 last 118.68) - /unlink 2567 (time 1788538994.19 total 20.00 last 137.96) - /unlink 4040 (time 1788539004.19 total 30.01 last 147.23) total: 4500 /unlink in 32.79 seconds: 137.23 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 711 (time 1788539020.95 total 10.00 last 71.08) - open/close 1526 (time 1788539030.97 total 20.03 last 81.28) - open/close 2314 (time 1788539040.97 total 30.03 last 78.79) - open/close 3149 (time 1788539050.99 total 40.04 last 83.40) - open/close 4001 (time 1788539060.99 total 50.05 last 85.16) total: 4500 open/close in 56.43 seconds: 79.74 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1246 (time 1788539081.49 total 10.01 last 124.51) - /unlink 2681 (time 1788539091.49 total 20.01 last 143.50) - /unlink 4203 (time 1788539101.50 total 30.01 last 152.13) total: 4500 /unlink in 32.54 seconds: 138.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 888 (time 1788539118.16 total 10.02 last 88.65) - open/close 1676 (time 1788539128.17 total 20.02 last 78.75) - open/close 2567 (time 1788539138.17 total 30.03 last 89.06) - open/close 3319 (time 1788539148.20 total 40.06 last 74.98) - open/close 4095 (time 1788539158.20 total 50.06 last 77.59) total: 4500 open/close in 56.19 seconds: 80.09 ops/second - /unlink 1550 (time 1788539178.58 total 10.00 last 154.99) - /unlink 3133 (time 1788539188.59 total 20.00 last 158.26) total: 4500 /unlink in 29.12 seconds: 154.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 863 (time 1788539212.59 total 10.01 last 86.24) - open/close 1698 (time 1788539222.60 total 20.02 last 83.42) - open/close 2543 (time 1788539232.60 total 30.02 last 84.46) - open/close 3412 (time 1788539242.61 total 40.03 last 86.84) - open/close 4244 (time 1788539252.62 total 50.03 last 83.15) total: 4500 open/close in 53.19 seconds: 84.60 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1596 (time 1788539269.80 total 10.00 last 159.57) - /unlink 3060 (time 1788539279.81 total 20.01 last 146.35) - /unlink 4349 (time 1788539289.81 total 30.01 last 128.89) total: 4500 /unlink in 31.36 seconds: 143.48 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 673 (time 1788539307.06 total 10.00 last 67.28) - open/close 1335 (time 1788539317.08 total 20.02 last 66.06) - open/close 2140 (time 1788539327.09 total 30.03 last 80.43) - open/close 2863 (time 1788539337.10 total 40.05 last 72.20) - open/close 3577 (time 1788539347.11 total 50.06 last 71.31) - open/close 4396 (time 1788539357.12 total 60.07 last 81.80) total: 4500 open/close in 61.61 seconds: 73.04 ops/second - /unlink 1392 (time 1788539373.38 total 10.00 last 139.19) - /unlink 2878 (time 1788539383.39 total 20.01 last 148.40) - /unlink 4329 (time 1788539393.39 total 30.02 last 145.05) total: 4500 /unlink in 31.48 seconds: 142.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 738 (time 1788539409.11 total 10.00 last 73.78) - open/close 1558 (time 1788539419.12 total 20.02 last 81.88) - open/close 2319 (time 1788539429.13 total 30.02 last 76.09) - open/close 3049 (time 1788539439.14 total 40.03 last 72.92) - open/close 3801 (time 1788539449.14 total 50.04 last 75.16) - open/close 4394 (time 1788539459.15 total 60.04 last 59.28) total: 4500 open/close in 61.67 seconds: 72.97 ops/second - /unlink 1463 (time 1788539475.25 total 10.01 last 146.09) - /unlink 2813 (time 1788539485.27 total 20.04 last 134.68) - /unlink 4105 (time 1788539495.27 total 30.04 last 129.15) total: 4500 /unlink in 33.09 seconds: 135.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 724 (time 1788539513.59 total 10.01 last 72.32) - open/close 1432 (time 1788539523.60 total 20.02 last 70.75) - open/close 2265 (time 1788539533.60 total 30.02 last 83.27) - open/close 2993 (time 1788539543.62 total 40.04 last 72.63) - open/close 3736 (time 1788539553.63 total 50.05 last 74.23) total: 4500 open/close in 59.69 seconds: 75.40 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1409 (time 1788539578.01 total 10.00 last 140.85) - /unlink 2812 (time 1788539588.02 total 20.01 last 140.25) - /unlink 4307 (time 1788539598.02 total 30.01 last 149.44) total: 4500 /unlink in 31.53 seconds: 142.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 769 (time 1788539614.43 total 10.01 last 76.79) - open/close 1529 (time 1788539624.44 total 20.02 last 75.94) - open/close 2339 (time 1788539634.45 total 30.03 last 80.94) - open/close 3147 (time 1788539644.47 total 40.05 last 80.62) - open/close 3896 (time 1788539654.48 total 50.06 last 74.83) total: 4500 open/close in 59.64 seconds: 75.45 ops/second - /unlink 1449 (time 1788539678.49 total 10.01 last 144.80) - /unlink 3044 (time 1788539688.50 total 20.01 last 159.45) total: 4500 /unlink in 29.58 seconds: 152.14 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 738 (time 1788539712.40 total 10.01 last 73.75) - open/close 1490 (time 1788539722.41 total 20.02 last 75.12) - open/close 2219 (time 1788539732.44 total 30.04 last 72.72) - open/close 2974 (time 1788539742.44 total 40.04 last 75.49) - open/close 3611 (time 1788539752.45 total 50.06 last 63.62) - open/close 4309 (time 1788539762.47 total 60.07 last 69.70) total: 4500 open/close in 63.22 seconds: 71.18 ops/second - /unlink 1637 (time 1788539780.30 total 10.00 last 163.63) - /unlink 3153 (time 1788539790.30 total 20.01 last 151.55) total: 4500 /unlink in 28.70 seconds: 156.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 852 (time 1788539814.46 total 10.00 last 85.18) - open/close 1779 (time 1788539824.46 total 20.01 last 92.62) - open/close 2685 (time 1788539834.47 total 30.02 last 90.55) - open/close 3561 (time 1788539844.49 total 40.04 last 87.43) - open/close 4435 (time 1788539854.50 total 50.05 last 87.30) total: 4500 open/close in 50.84 seconds: 88.51 ops/second - /unlink 1450 (time 1788539868.83 total 10.00 last 144.95) - /unlink 3021 (time 1788539878.84 total 20.01 last 157.03) total: 4500 /unlink in 29.87 seconds: 150.66 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 854 (time 1788539903.11 total 10.00 last 85.36) - open/close 1616 (time 1788539913.13 total 20.02 last 76.10) - open/close 2354 (time 1788539923.14 total 30.03 last 73.74) - open/close 3229 (time 1788539933.14 total 40.03 last 87.46) - open/close 4020 (time 1788539943.16 total 50.05 last 78.93) total: 4500 open/close in 56.64 seconds: 79.45 ops/second - /unlink 1547 (time 1788539964.59 total 10.00 last 154.70) - /unlink 2973 (time 1788539974.59 total 20.00 last 142.60) - /unlink 4482 (time 1788539984.59 total 30.00 last 150.85) total: 4500 /unlink in 30.07 seconds: 149.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 772 (time 1788539999.67 total 10.00 last 77.16) - open/close 1543 (time 1788540009.70 total 20.03 last 76.93) - open/close 2324 (time 1788540019.72 total 30.05 last 77.94) - open/close 3077 (time 1788540029.73 total 40.06 last 75.23) - open/close 3845 (time 1788540039.75 total 50.08 last 76.61) - open/close 4412 (time 1788540049.78 total 60.11 last 56.55) total: 4500 open/close in 61.77 seconds: 72.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1416 (time 1788540068.14 total 10.02 last 141.32) - /unlink 2566 (time 1788540078.15 total 20.03 last 114.87) - /unlink 3829 (time 1788540088.16 total 30.04 last 126.19) total: 4500 /unlink in 34.60 seconds: 130.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 832 (time 1788540107.58 total 10.01 last 83.16) - open/close 1669 (time 1788540117.59 total 20.02 last 83.57) - open/close 2472 (time 1788540127.60 total 30.02 last 80.28) - open/close 3314 (time 1788540137.60 total 40.03 last 84.16) - open/close 4071 (time 1788540147.60 total 50.03 last 75.69) total: 4500 open/close in 55.62 seconds: 80.91 ops/second - /unlink 1610 (time 1788540167.50 total 10.00 last 160.98) - /unlink 3058 (time 1788540177.51 total 20.01 last 144.69) total: 4500 /unlink in 29.03 seconds: 155.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 802 (time 1788540200.64 total 10.00 last 80.19) - open/close 1629 (time 1788540210.65 total 20.01 last 82.66) - open/close 2409 (time 1788540220.65 total 30.01 last 77.98) - open/close 3269 (time 1788540230.66 total 40.02 last 85.95) - open/close 4131 (time 1788540240.66 total 50.02 last 86.16) total: 4500 open/close in 54.89 seconds: 81.98 ops/second - /unlink 1562 (time 1788540259.86 total 10.01 last 155.97) - /unlink 3194 (time 1788540269.86 total 20.02 last 163.17) total: 4500 /unlink in 28.32 seconds: 158.93 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95228 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg332-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg332-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg332-server stop mds service on oleg332-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg332-server stop mds service on oleg332-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg332-server unloading modules via unload_modules_local on: 'oleg332-server' oleg332-server: oleg332-server.virtnet: executing unload_modules_local oleg332-server: kmemleak disabled modules unloaded.