== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 20:25:54 (1784939154) Stopping clients: oleg651-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg651-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg651-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg651-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg651-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg651-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg651-server oleg651-server: oleg651-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg651-server' oleg651-server: oleg651-server.virtnet: executing load_modules_local oleg651-server: Loading modules from /home/green/git/lustre-release/lustre oleg651-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg651-server: detected 4 online CPUs by sysfs oleg651-server: MODOPTS_LIBCFS= oleg651-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg651-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg651-server: oleg651-server.virtnet: executing set_default_debug -1 all pdsh@oleg651-client: oleg651-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg651-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg651-server: oleg651-server.virtnet: executing set_default_debug -1 all pdsh@oleg651-client: oleg651-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg651-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg651-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg651-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg651-server: oleg651-server.virtnet: executing set_default_debug -1 all pdsh@oleg651-client: oleg651-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg651-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg651-client.virtnet: -o user_xattr,flock 192.168.206.151@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H fnv_1a_64 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 716 (time 1784939345.15 total 10.00 last 71.59) - open/close 1548 (time 1784939355.15 total 20.00 last 83.20) - open/close 2316 (time 1784939365.16 total 30.00 last 76.78) - open/close 3104 (time 1784939375.17 total 40.01 last 78.73) - open/close 3900 (time 1784939385.18 total 50.03 last 79.48) total: 4500 open/close in 58.09 seconds: 77.46 ops/second - /unlink 1253 (time 1784939407.41 total 10.00 last 125.27) - /unlink 2703 (time 1784939417.42 total 20.01 last 144.96) - /unlink 4138 (time 1784939427.42 total 30.01 last 143.43) total: 4500 /unlink in 32.46 seconds: 138.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 827 (time 1784939444.61 total 10.02 last 82.57) - open/close 1717 (time 1784939454.63 total 20.03 last 88.84) - open/close 2546 (time 1784939464.64 total 30.04 last 82.85) - open/close 3446 (time 1784939474.64 total 40.05 last 89.94) - open/close 4264 (time 1784939484.64 total 50.05 last 81.78) total: 4500 open/close in 52.99 seconds: 84.92 ops/second - /unlink 1489 (time 1784939501.64 total 10.00 last 148.89) - /unlink 2840 (time 1784939511.64 total 20.01 last 135.01) - /unlink 4332 (time 1784939521.64 total 30.01 last 149.19) total: 4500 /unlink in 31.34 seconds: 143.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 745 (time 1784939536.99 total 10.01 last 74.46) - open/close 1554 (time 1784939547.00 total 20.01 last 80.86) - open/close 2340 (time 1784939557.01 total 30.02 last 78.54) - open/close 3140 (time 1784939567.01 total 40.02 last 79.97) - open/close 3990 (time 1784939577.01 total 50.02 last 84.99) total: 4500 open/close in 56.92 seconds: 79.06 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1425 (time 1784939598.53 total 10.00 last 142.44) - /unlink 2771 (time 1784939608.53 total 20.01 last 134.54) - /unlink 4287 (time 1784939618.53 total 30.01 last 151.59) total: 4500 /unlink in 31.52 seconds: 142.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 851 (time 1784939633.38 total 10.00 last 85.10) - open/close 1705 (time 1784939643.39 total 20.01 last 85.31) - open/close 2530 (time 1784939653.39 total 30.01 last 82.47) - open/close 3329 (time 1784939663.39 total 40.02 last 79.88) - open/close 4188 (time 1784939673.41 total 50.03 last 85.78) total: 4500 open/close in 53.80 seconds: 83.64 ops/second - /unlink 1472 (time 1784939690.96 total 10.00 last 147.18) - /unlink 2984 (time 1784939700.96 total 20.00 last 151.16) - /unlink 4483 (time 1784939710.96 total 30.00 last 149.90) total: 4500 /unlink in 30.16 seconds: 149.22 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 860 (time 1784939725.82 total 10.00 last 85.99) - open/close 1710 (time 1784939735.83 total 20.01 last 84.95) - open/close 2518 (time 1784939745.84 total 30.02 last 80.68) - open/close 3402 (time 1784939755.85 total 40.03 last 88.36) - open/close 4202 (time 1784939765.86 total 50.04 last 79.90) total: 4500 open/close in 53.94 seconds: 83.43 ops/second - /unlink 1370 (time 1784939784.27 total 10.00 last 136.99) - /unlink 2951 (time 1784939794.28 total 20.01 last 157.96) - /unlink 4488 (time 1784939804.29 total 30.01 last 153.65) total: 4500 /unlink in 30.05 seconds: 149.73 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 757 (time 1784939818.69 total 10.00 last 75.68) - open/close 1530 (time 1784939828.69 total 20.00 last 77.30) - open/close 2406 (time 1784939838.70 total 30.01 last 87.56) - open/close 3250 (time 1784939848.70 total 40.01 last 84.37) - open/close 4104 (time 1784939858.70 total 50.01 last 85.37) total: 4500 open/close in 54.45 seconds: 82.65 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1637 (time 1784939877.10 total 10.00 last 163.65) - /unlink 3220 (time 1784939887.10 total 20.00 last 158.30) total: 4500 /unlink in 28.17 seconds: 159.76 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 813 (time 1784939910.06 total 10.01 last 81.23) - open/close 1663 (time 1784939920.07 total 20.01 last 84.96) - open/close 2428 (time 1784939930.07 total 30.02 last 76.47) - open/close 3279 (time 1784939940.07 total 40.02 last 85.09) - open/close 4134 (time 1784939950.08 total 50.03 last 85.41) total: 4500 open/close in 53.99 seconds: 83.35 ops/second - /unlink 1598 (time 1784939967.65 total 10.00 last 159.75) - /unlink 3006 (time 1784939977.65 total 20.00 last 140.80) - /unlink 4411 (time 1784939987.65 total 30.00 last 140.49) total: 4500 /unlink in 30.69 seconds: 146.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 841 (time 1784940003.59 total 10.01 last 84.02) - open/close 1690 (time 1784940013.59 total 20.01 last 84.89) - open/close 2490 (time 1784940023.60 total 30.02 last 79.90) - open/close 3200 (time 1784940033.62 total 40.04 last 70.90) - open/close 3925 (time 1784940043.63 total 50.05 last 72.40) total: 4500 open/close in 56.24 seconds: 80.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1563 (time 1784940063.68 total 10.00 last 156.28) - /unlink 3259 (time 1784940073.68 total 20.01 last 169.50) total: 4500 /unlink in 27.95 seconds: 161.01 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 873 (time 1784940096.03 total 10.01 last 87.26) - open/close 1705 (time 1784940106.03 total 20.01 last 83.17) - open/close 2536 (time 1784940116.04 total 30.01 last 83.05) - open/close 3313 (time 1784940126.04 total 40.02 last 77.66) - open/close 4100 (time 1784940136.05 total 50.02 last 78.65) total: 4500 open/close in 55.33 seconds: 81.33 ops/second - /unlink 1425 (time 1784940155.34 total 10.01 last 142.42) - /unlink 2735 (time 1784940165.34 total 20.01 last 130.95) - /unlink 4108 (time 1784940175.34 total 30.01 last 137.29) total: 4500 /unlink in 32.55 seconds: 138.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 808 (time 1784940192.62 total 10.00 last 80.76) - open/close 1653 (time 1784940202.63 total 20.02 last 84.41) - open/close 2489 (time 1784940212.64 total 30.02 last 83.56) - open/close 3397 (time 1784940222.65 total 40.03 last 90.71) - open/close 4282 (time 1784940232.65 total 50.03 last 88.50) total: 4500 open/close in 52.41 seconds: 85.86 ops/second - /unlink 1509 (time 1784940248.76 total 10.00 last 150.85) - /unlink 3022 (time 1784940258.76 total 20.00 last 151.30) total: 4500 /unlink in 29.84 seconds: 150.79 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 883 (time 1784940282.72 total 10.01 last 88.19) - open/close 1762 (time 1784940292.73 total 20.02 last 87.81) - open/close 2601 (time 1784940302.74 total 30.03 last 83.85) - open/close 3346 (time 1784940312.75 total 40.04 last 74.41) - open/close 4239 (time 1784940322.76 total 50.05 last 89.26) total: 4500 open/close in 53.25 seconds: 84.51 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1535 (time 1784940339.94 total 10.00 last 153.46) - /unlink 3006 (time 1784940349.95 total 20.00 last 147.08) - /unlink 4456 (time 1784940359.95 total 30.00 last 145.00) total: 4500 /unlink in 30.44 seconds: 147.85 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 685 (time 1784940375.12 total 10.01 last 68.44) - open/close 1520 (time 1784940385.13 total 20.02 last 83.43) - open/close 2376 (time 1784940395.13 total 30.02 last 85.60) - open/close 3236 (time 1784940405.14 total 40.03 last 85.89) - open/close 4104 (time 1784940415.16 total 50.04 last 86.70) total: 4500 open/close in 54.48 seconds: 82.60 ops/second - /unlink 1543 (time 1784940433.71 total 10.00 last 154.29) - /unlink 3198 (time 1784940443.71 total 20.00 last 165.46) total: 4500 /unlink in 27.95 seconds: 160.98 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 845 (time 1784940465.93 total 10.00 last 84.48) - open/close 1726 (time 1784940475.95 total 20.02 last 87.97) - open/close 2562 (time 1784940485.95 total 30.02 last 83.56) - open/close 3215 (time 1784940495.95 total 40.02 last 65.29) - open/close 4095 (time 1784940505.96 total 50.03 last 87.95) total: 4500 open/close in 54.29 seconds: 82.89 ops/second - /unlink 1340 (time 1784940524.91 total 10.00 last 134.00) - /unlink 2862 (time 1784940534.92 total 20.00 last 152.13) - /unlink 4423 (time 1784940544.92 total 30.01 last 156.03) total: 4500 /unlink in 30.85 seconds: 145.84 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 870 (time 1784940560.13 total 10.00 last 86.97) - open/close 1752 (time 1784940570.13 total 20.01 last 88.18) - open/close 2604 (time 1784940580.13 total 30.01 last 85.18) - open/close 3525 (time 1784940590.14 total 40.02 last 92.02) - open/close 4396 (time 1784940600.14 total 50.02 last 87.06) total: 4500 open/close in 51.02 seconds: 88.20 ops/second - /unlink 1515 (time 1784940615.23 total 10.00 last 151.43) - /unlink 3045 (time 1784940625.23 total 20.01 last 152.95) total: 4500 /unlink in 29.29 seconds: 153.64 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 886 (time 1784940649.06 total 10.00 last 88.56) - open/close 1715 (time 1784940659.06 total 20.01 last 82.88) - open/close 2583 (time 1784940669.07 total 30.02 last 86.73) - open/close 3402 (time 1784940679.07 total 40.02 last 81.85) - open/close 4294 (time 1784940689.08 total 50.03 last 89.17) total: 4500 open/close in 52.64 seconds: 85.48 ops/second - /unlink 1652 (time 1784940705.53 total 10.01 last 165.00) - /unlink 3239 (time 1784940715.54 total 20.02 last 158.60) total: 4500 /unlink in 29.20 seconds: 154.13 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 612 (time 1784940741.20 total 10.01 last 61.14) - open/close 1386 (time 1784940751.21 total 20.02 last 77.33) - open/close 2262 (time 1784940761.22 total 30.03 last 87.53) - open/close 3025 (time 1784940771.22 total 40.03 last 76.27) - open/close 3857 (time 1784940781.23 total 50.04 last 83.15) total: 4500 open/close in 57.46 seconds: 78.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1439 (time 1784940802.82 total 10.01 last 143.80) - /unlink 2772 (time 1784940812.82 total 20.01 last 133.29) - /unlink 4302 (time 1784940822.82 total 30.01 last 152.92) total: 4500 /unlink in 31.30 seconds: 143.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 852 (time 1784940838.17 total 10.00 last 85.20) - open/close 1667 (time 1784940848.17 total 20.00 last 81.50) - open/close 2524 (time 1784940858.17 total 30.01 last 85.65) - open/close 3317 (time 1784940868.19 total 40.02 last 79.19) - open/close 4097 (time 1784940878.20 total 50.03 last 77.92) total: 4500 open/close in 54.54 seconds: 82.50 ops/second - /unlink 1594 (time 1784940896.04 total 10.00 last 159.40) - /unlink 3121 (time 1784940906.04 total 20.00 last 152.68) total: 4500 /unlink in 29.57 seconds: 152.18 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 815 (time 1784940929.63 total 10.01 last 81.45) - open/close 1588 (time 1784940939.63 total 20.01 last 77.26) - open/close 2474 (time 1784940949.63 total 30.01 last 88.57) - open/close 3329 (time 1784940959.64 total 40.02 last 85.44) - open/close 4153 (time 1784940969.65 total 50.03 last 82.36) total: 4500 open/close in 53.92 seconds: 83.45 ops/second - /unlink 1565 (time 1784940987.38 total 10.00 last 156.44) - /unlink 3019 (time 1784940997.39 total 20.01 last 145.34) total: 4500 /unlink in 29.94 seconds: 150.29 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11297: 95180 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg651-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg651-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg651-server stop mds service on oleg651-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg651-server stop mds service on oleg651-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg651-server unloading modules via unload_modules_local on: 'oleg651-server' oleg651-server: oleg651-server.virtnet: executing unload_modules_local oleg651-server: modules unloaded.