== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 10:15:50 (1753366550) Stopping clients: oleg123-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg123-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg123-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg123-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg123-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg123-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg123-server oleg123-server: oleg123-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1024: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1025: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg123-server' oleg123-server: oleg123-server.virtnet: executing load_modules_local oleg123-server: Loading modules from /home/green/git/lustre-release/lustre oleg123-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1024: echo: write error: Device or resource busy oleg123-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1025: echo: write error: Device or resource busy oleg123-server: detected 4 online CPUs by sysfs oleg123-server: MODOPTS_LIBCFS= oleg123-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg123-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg123-server: oleg123-server.virtnet: executing set_default_debug -1 all pdsh@oleg123-client: oleg123-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg123-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg123-server: oleg123-server.virtnet: executing set_default_debug -1 all pdsh@oleg123-client: oleg123-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg123-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg123-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg123-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg123-server: oleg123-server.virtnet: executing set_default_debug -1 all pdsh@oleg123-client: oleg123-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg123-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg123-client.virtnet: -o user_xattr,flock 192.168.201.123@tcp:/lustre /mnt/lustre oleg123-client: fail_loc=0x1312 oleg123-client: fail_val=5 oleg123-server: fail_loc=0x1312 oleg123-server: fail_val=5 striped dir -i0 -c1 -H all_char /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 1336 (time 1753366637.69 total 10.01 last 133.49) - open/close 2337 (time 1753366647.69 total 20.01 last 100.07) - open/close 3272 (time 1753366657.71 total 30.02 last 93.39) - open/close 4224 (time 1753366667.71 total 40.03 last 95.14) total: 4500 open/close in 43.18 seconds: 104.21 ops/second - /unlink 2107 (time 1753366684.95 total 10.01 last 210.52) - /unlink 4490 (time 1753366694.96 total 20.02 last 238.09) total: 4500 /unlink in 20.11 seconds: 223.72 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 1266 (time 1753366707.12 total 10.01 last 126.51) - open/close 2885 (time 1753366717.12 total 20.01 last 161.88) - open/close 4160 (time 1753366727.12 total 30.01 last 127.43) total: 4500 open/close in 32.66 seconds: 137.80 ops/second - /unlink 2806 (time 1753366742.11 total 10.00 last 280.58) total: 4500 /unlink in 16.28 seconds: 276.49 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 1280 (time 1753366760.49 total 10.00 last 127.99) - open/close 2833 (time 1753366770.49 total 20.00 last 155.27) - open/close 4352 (time 1753366780.49 total 30.00 last 151.87) total: 4500 open/close in 30.97 seconds: 145.31 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 2825 (time 1753366793.31 total 10.00 last 282.49) total: 4500 /unlink in 16.22 seconds: 277.43 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 1501 (time 1753366811.54 total 10.01 last 149.99) - open/close 3317 (time 1753366821.54 total 20.01 last 181.60) total: 4500 open/close in 28.03 seconds: 160.57 ops/second - /unlink 3621 (time 1753366841.29 total 10.00 last 361.99) total: 4500 /unlink in 13.05 seconds: 344.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 1668 (time 1753366856.19 total 10.00 last 166.79) - open/close 2916 (time 1753366866.19 total 20.00 last 124.76) - open/close 4411 (time 1753366876.20 total 30.01 last 149.37) total: 4500 open/close in 30.73 seconds: 146.42 ops/second - /unlink 3030 (time 1753366889.04 total 10.00 last 302.92) total: 4500 /unlink in 14.86 seconds: 302.74 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 1697 (time 1753366906.31 total 10.00 last 169.70) - open/close 3115 (time 1753366916.32 total 20.01 last 141.73) total: 4500 open/close in 27.89 seconds: 161.32 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 3228 (time 1753366936.06 total 10.00 last 322.78) total: 4500 /unlink in 14.78 seconds: 304.41 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 1280 (time 1753366953.19 total 10.01 last 127.88) - open/close 2621 (time 1753366963.20 total 20.02 last 134.01) - open/close 3793 (time 1753366973.20 total 30.02 last 117.17) total: 4500 open/close in 36.20 seconds: 124.32 ops/second - /unlink 3455 (time 1753366991.35 total 10.00 last 345.38) total: 4500 /unlink in 13.90 seconds: 323.76 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 1548 (time 1753367007.11 total 10.01 last 154.71) - open/close 3047 (time 1753367017.11 total 20.01 last 149.80) total: 4500 open/close in 29.49 seconds: 152.57 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 2873 (time 1753367038.42 total 10.00 last 287.27) total: 4500 /unlink in 16.26 seconds: 276.75 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 1803 (time 1753367056.47 total 10.00 last 180.28) - open/close 4275 (time 1753367066.47 total 20.00 last 247.16) total: 4500 open/close in 20.93 seconds: 215.05 ops/second - /unlink 4379 (time 1753367078.89 total 10.00 last 437.83) total: 4500 /unlink in 10.28 seconds: 437.70 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 2382 (time 1753367090.79 total 10.00 last 238.14) total: 4500 open/close in 18.94 seconds: 237.56 ops/second total: 4500 /unlink in 6.77 seconds: 664.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 2640 (time 1753367119.33 total 10.00 last 263.94) total: 4500 open/close in 14.81 seconds: 303.78 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 total: 4500 /unlink in 7.54 seconds: 596.69 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 2858 (time 1753367144.47 total 10.00 last 285.77) total: 4500 open/close in 16.85 seconds: 267.11 ops/second total: 4500 /unlink in 7.58 seconds: 594.02 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 3156 (time 1753367171.84 total 10.00 last 315.60) total: 4500 open/close in 13.84 seconds: 325.15 ops/second total: 4500 /unlink in 8.76 seconds: 513.53 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 2327 (time 1753367197.46 total 10.01 last 232.56) total: 4500 open/close in 17.17 seconds: 262.02 ops/second total: 4500 /unlink in 7.11 seconds: 633.04 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 3812 (time 1753367224.78 total 10.00 last 381.12) total: 4500 open/close in 11.84 seconds: 379.97 ops/second total: 4500 /unlink in 7.87 seconds: 571.61 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 3234 (time 1753367247.34 total 10.00 last 323.36) total: 4500 open/close in 13.32 seconds: 337.90 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 total: 4500 /unlink in 7.10 seconds: 633.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 2406 (time 1753367270.54 total 10.00 last 240.57) total: 4500 open/close in 19.03 seconds: 236.46 ops/second - /unlink 4335 (time 1753367291.20 total 10.00 last 433.42) total: 4500 /unlink in 10.41 seconds: 432.21 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 2187 (time 1753367303.56 total 10.00 last 218.68) - open/close 4186 (time 1753367313.56 total 20.01 last 199.81) total: 4500 open/close in 21.87 seconds: 205.76 ops/second total: 4500 /unlink in 9.99 seconds: 450.26 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11149: 93216 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg123-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg123-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg123-server stop mds service on oleg123-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg123-server stop mds service on oleg123-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg123-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg123-server' oleg123-server: oleg123-server.virtnet: executing unload_modules_local oleg123-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg123-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded.