== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 07:29:12 (1786447752) Stopping clients: oleg138-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg138-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg138-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg138-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg138-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg138-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg138-server oleg138-server: oleg138-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg138-server' oleg138-server: oleg138-server.virtnet: executing load_modules_local oleg138-server: Loading modules from /home/green/git/lustre-release/lustre oleg138-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg138-server: detected 4 online CPUs by sysfs oleg138-server: MODOPTS_LIBCFS= oleg138-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg138-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg138-server: oleg138-server.virtnet: executing set_default_debug -1 all pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg138-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg138-server: oleg138-server.virtnet: executing set_default_debug -1 all pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg138-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg138-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg138-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg138-server: oleg138-server.virtnet: executing set_default_debug -1 all pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg138-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg138-client.virtnet: -o user_xattr,flock 192.168.201.138@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 906 (time 1786447898.88 total 10.01 last 90.51) - open/close 1888 (time 1786447908.88 total 20.02 last 98.12) - open/close 2842 (time 1786447918.89 total 30.03 last 95.32) - open/close 3719 (time 1786447928.90 total 40.03 last 87.64) total: 4500 open/close in 49.38 seconds: 91.12 ops/second - /unlink 1576 (time 1786447951.84 total 10.00 last 157.54) - /unlink 3214 (time 1786447961.85 total 20.01 last 163.71) total: 4500 /unlink in 27.06 seconds: 166.27 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 878 (time 1786447982.79 total 10.01 last 87.74) - open/close 1817 (time 1786447992.79 total 20.01 last 93.87) - open/close 2709 (time 1786448002.81 total 30.03 last 89.05) - open/close 3622 (time 1786448012.81 total 40.03 last 91.30) total: 4500 open/close in 49.25 seconds: 91.36 ops/second - /unlink 1796 (time 1786448035.39 total 10.01 last 179.50) - /unlink 3395 (time 1786448045.40 total 20.01 last 159.79) total: 4500 /unlink in 26.81 seconds: 167.83 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 874 (time 1786448065.70 total 10.01 last 87.31) - open/close 1849 (time 1786448075.71 total 20.02 last 97.42) - open/close 2863 (time 1786448085.71 total 30.03 last 101.31) - open/close 3909 (time 1786448095.72 total 40.04 last 104.52) total: 4500 open/close in 46.21 seconds: 97.38 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1778 (time 1786448114.72 total 10.00 last 177.76) - /unlink 3557 (time 1786448124.73 total 20.01 last 177.82) total: 4500 /unlink in 25.35 seconds: 177.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 950 (time 1786448143.38 total 10.01 last 94.93) - open/close 1866 (time 1786448153.38 total 20.01 last 91.58) - open/close 2811 (time 1786448163.38 total 30.01 last 94.48) - open/close 3805 (time 1786448173.38 total 40.01 last 99.39) total: 4500 open/close in 47.19 seconds: 95.36 ops/second - /unlink 1983 (time 1786448193.89 total 10.00 last 198.28) - /unlink 3939 (time 1786448203.89 total 20.00 last 195.54) total: 4500 /unlink in 23.09 seconds: 194.91 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 1007 (time 1786448220.23 total 10.01 last 100.63) - open/close 2070 (time 1786448230.23 total 20.01 last 106.30) - open/close 3054 (time 1786448240.24 total 30.01 last 98.33) - open/close 4162 (time 1786448250.26 total 40.03 last 110.61) total: 4500 open/close in 43.31 seconds: 103.91 ops/second - /unlink 2157 (time 1786448266.09 total 10.00 last 215.68) - /unlink 4016 (time 1786448276.10 total 20.01 last 185.79) total: 4500 /unlink in 23.27 seconds: 193.37 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 945 (time 1786448292.94 total 10.01 last 94.43) - open/close 1841 (time 1786448302.95 total 20.01 last 89.58) - open/close 2851 (time 1786448312.95 total 30.01 last 100.99) - open/close 3812 (time 1786448322.96 total 40.03 last 95.96) total: 4500 open/close in 46.78 seconds: 96.20 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1699 (time 1786448342.77 total 10.01 last 169.74) - /unlink 3480 (time 1786448352.78 total 20.02 last 178.00) total: 4500 /unlink in 25.77 seconds: 174.59 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 1160 (time 1786448371.52 total 10.00 last 115.95) - open/close 2161 (time 1786448381.53 total 20.01 last 100.04) - open/close 3213 (time 1786448391.53 total 30.01 last 105.15) - open/close 4272 (time 1786448401.54 total 40.02 last 105.86) total: 4500 open/close in 42.07 seconds: 106.96 ops/second - /unlink 1970 (time 1786448416.12 total 10.00 last 196.99) - /unlink 4140 (time 1786448426.12 total 20.00 last 217.00) total: 4500 /unlink in 21.74 seconds: 206.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 1008 (time 1786448440.36 total 10.00 last 100.80) - open/close 2065 (time 1786448450.36 total 20.00 last 105.65) - open/close 3225 (time 1786448460.36 total 30.01 last 115.99) total: 4500 open/close in 39.35 seconds: 114.35 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 2438 (time 1786448481.57 total 10.00 last 243.74) - /unlink 4480 (time 1786448491.57 total 20.01 last 204.12) total: 4500 /unlink in 20.10 seconds: 223.86 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 1238 (time 1786448504.69 total 10.01 last 123.67) - open/close 2412 (time 1786448514.70 total 20.01 last 117.35) - open/close 3554 (time 1786448524.70 total 30.02 last 114.19) total: 4500 open/close in 37.73 seconds: 119.27 ops/second - /unlink 2104 (time 1786448544.66 total 10.00 last 210.38) - /unlink 4178 (time 1786448554.66 total 20.00 last 207.35) total: 4500 /unlink in 21.50 seconds: 209.28 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 1190 (time 1786448568.61 total 10.00 last 119.00) - open/close 2483 (time 1786448578.62 total 20.01 last 129.23) - open/close 3663 (time 1786448588.62 total 30.01 last 117.95) total: 4500 open/close in 35.32 seconds: 127.42 ops/second - /unlink 2705 (time 1786448606.02 total 10.00 last 270.41) total: 4500 /unlink in 18.22 seconds: 247.05 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 1062 (time 1786448627.04 total 10.00 last 106.19) - open/close 2293 (time 1786448637.05 total 20.00 last 123.07) - open/close 3377 (time 1786448647.05 total 30.00 last 108.39) - open/close 4475 (time 1786448657.05 total 40.01 last 109.79) total: 4500 open/close in 40.19 seconds: 111.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 2442 (time 1786448669.54 total 10.00 last 244.17) total: 4500 /unlink in 19.32 seconds: 232.87 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 1192 (time 1786448690.99 total 10.01 last 119.13) - open/close 2374 (time 1786448700.99 total 20.01 last 118.19) - open/close 3556 (time 1786448711.00 total 30.01 last 118.15) total: 4500 open/close in 38.51 seconds: 116.86 ops/second - /unlink 2079 (time 1786448732.05 total 10.00 last 207.89) - /unlink 4097 (time 1786448742.05 total 20.00 last 201.79) total: 4500 /unlink in 21.85 seconds: 205.94 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 1242 (time 1786448756.56 total 10.01 last 124.11) - open/close 2699 (time 1786448766.57 total 20.02 last 145.55) - open/close 3895 (time 1786448776.58 total 30.03 last 119.46) total: 4500 open/close in 35.21 seconds: 127.80 ops/second - /unlink 2374 (time 1786448794.19 total 10.00 last 237.36) total: 4500 /unlink in 17.68 seconds: 254.46 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 1321 (time 1786448813.89 total 10.00 last 132.05) - open/close 2584 (time 1786448823.89 total 20.01 last 126.22) - open/close 3682 (time 1786448833.90 total 30.02 last 109.73) total: 4500 open/close in 36.70 seconds: 122.62 ops/second - /unlink 2042 (time 1786448852.82 total 10.00 last 204.20) - /unlink 3957 (time 1786448862.83 total 20.01 last 191.38) total: 4500 /unlink in 22.90 seconds: 196.50 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 1152 (time 1786448878.31 total 10.00 last 115.18) - open/close 2377 (time 1786448888.32 total 20.01 last 122.37) - open/close 3543 (time 1786448898.32 total 30.02 last 116.54) total: 4500 open/close in 38.93 seconds: 115.59 ops/second - /unlink 2215 (time 1786448919.99 total 10.00 last 221.47) total: 4500 /unlink in 19.76 seconds: 227.71 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 1070 (time 1786448942.61 total 10.00 last 106.97) - open/close 2030 (time 1786448952.62 total 20.02 last 95.88) - open/close 2997 (time 1786448962.63 total 30.02 last 96.64) - open/close 4137 (time 1786448972.64 total 40.04 last 113.84) total: 4500 open/close in 43.40 seconds: 103.68 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 2453 (time 1786448988.48 total 10.01 last 244.97) total: 4500 /unlink in 17.83 seconds: 252.34 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 1321 (time 1786449008.37 total 10.00 last 132.09) - open/close 2662 (time 1786449018.37 total 20.01 last 133.99) - open/close 3933 (time 1786449028.38 total 30.02 last 127.02) total: 4500 open/close in 34.49 seconds: 130.46 ops/second - /unlink 2693 (time 1786449044.94 total 10.00 last 269.23) total: 4500 /unlink in 16.38 seconds: 274.68 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 1418 (time 1786449063.11 total 10.00 last 141.80) - open/close 3031 (time 1786449073.11 total 20.00 last 161.29) - open/close 4297 (time 1786449083.12 total 30.00 last 126.54) total: 4500 open/close in 32.01 seconds: 140.56 ops/second - /unlink 2319 (time 1786449097.44 total 10.00 last 231.85) total: 4500 /unlink in 18.27 seconds: 246.31 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11316: 95139 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg138-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg138-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg138-server stop mds service on oleg138-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg138-server stop mds service on oleg138-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg138-server unloading modules via unload_modules_local on: 'oleg138-server' oleg138-server: oleg138-server.virtnet: executing unload_modules_local oleg138-server: modules unloaded.