== conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 22:11:06 (1787796666) Stopping clients: oleg323-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg323-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg323-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg323-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg323-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg323-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg323-server oleg323-server: oleg323-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg323-server' oleg323-server: oleg323-server.virtnet: executing load_modules_local oleg323-server: kmemleak disabled oleg323-server: Loading modules from /home/green/git/lustre-release/lustre oleg323-server: detected 4 online CPUs by sysfs oleg323-server: MODOPTS_LIBCFS= oleg323-server: Force libcfs to create 2 CPU partitions oleg323-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg323-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg323-server: oleg323-server.virtnet: executing set_default_debug -1 all pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg323-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg323-server: oleg323-server.virtnet: executing set_default_debug -1 all pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg323-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg323-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg323-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg323-server: oleg323-server.virtnet: executing set_default_debug -1 all pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg323-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg323-client.virtnet: -o user_xattr,flock 192.168.203.123@tcp:/lustre /mnt/lustre fail_loc=0x1312 fail_val=5 fail_loc=0x1312 fail_val=5 striped dir -i0 -c1 -H crush2 /mnt/lustre/d135.conf-sanity seq.cli-lustre-OST0000-super.width=0x1ffffff mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Wrap arround changelog catalog lustre-MDT0000: clear the changelog for cl1 to record #12998 - open/close 709 (time 1787796898.31 total 10.00 last 70.89) - open/close 1408 (time 1787796908.32 total 20.01 last 69.86) - open/close 2030 (time 1787796918.33 total 30.02 last 62.12) - open/close 2660 (time 1787796928.35 total 40.04 last 62.89) - open/close 3361 (time 1787796938.35 total 50.04 last 70.06) - open/close 3899 (time 1787796948.36 total 60.05 last 53.77) total: 4500 open/close in 68.73 seconds: 65.48 ops/second - /unlink 1308 (time 1787796971.99 total 10.00 last 130.78) - /unlink 2660 (time 1787796982.01 total 20.02 last 135.00) - /unlink 3999 (time 1787796992.01 total 30.02 last 133.81) total: 4500 /unlink in 34.07 seconds: 132.08 ops/second lustre-MDT0000: clear the changelog for cl1 to record #25998 - open/close 717 (time 1787797010.20 total 10.01 last 71.65) - open/close 1477 (time 1787797020.21 total 20.02 last 75.89) - open/close 2149 (time 1787797030.22 total 30.03 last 67.15) - open/close 2781 (time 1787797040.22 total 40.03 last 63.18) - open/close 3444 (time 1787797050.22 total 50.03 last 66.29) - open/close 4016 (time 1787797060.23 total 60.04 last 57.20) total: 4500 open/close in 68.10 seconds: 66.08 ops/second - /unlink 1395 (time 1787797083.22 total 10.00 last 139.44) - /unlink 2678 (time 1787797093.22 total 20.01 last 128.24) - /unlink 3978 (time 1787797103.23 total 30.01 last 129.93) total: 4500 /unlink in 34.07 seconds: 132.08 ops/second lustre-MDT0000: clear the changelog for cl1 to record #38998 - open/close 701 (time 1787797122.27 total 10.01 last 70.04) - open/close 1383 (time 1787797132.27 total 20.02 last 68.16) - open/close 2113 (time 1787797142.28 total 30.03 last 72.92) - open/close 2747 (time 1787797152.28 total 40.03 last 63.39) - open/close 3383 (time 1787797162.31 total 50.05 last 63.47) - open/close 4002 (time 1787797172.31 total 60.05 last 61.86) total: 4500 open/close in 67.06 seconds: 67.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #51998 - /unlink 1381 (time 1787797193.85 total 10.00 last 138.09) - /unlink 2632 (time 1787797203.86 total 20.00 last 125.07) - /unlink 3906 (time 1787797213.86 total 30.01 last 127.31) total: 4500 /unlink in 34.81 seconds: 129.29 ops/second lustre-MDT0000: clear the changelog for cl1 to record #64998 - open/close 636 (time 1787797233.13 total 10.01 last 63.57) - open/close 1295 (time 1787797243.14 total 20.01 last 65.88) - open/close 1931 (time 1787797253.14 total 30.01 last 63.58) - open/close 2564 (time 1787797263.15 total 40.02 last 63.25) - open/close 3264 (time 1787797273.16 total 50.03 last 69.91) - open/close 3915 (time 1787797283.18 total 60.05 last 64.97) total: 4500 open/close in 69.19 seconds: 65.04 ops/second - /unlink 1222 (time 1787797306.94 total 10.00 last 122.19) - /unlink 2509 (time 1787797316.94 total 20.01 last 128.61) - /unlink 3861 (time 1787797326.95 total 30.01 last 135.19) total: 4500 /unlink in 35.44 seconds: 126.99 ops/second lustre-MDT0000: clear the changelog for cl1 to record #77998 - open/close 693 (time 1787797347.44 total 10.00 last 69.30) - open/close 1320 (time 1787797357.44 total 20.00 last 62.69) - open/close 2017 (time 1787797367.46 total 30.01 last 69.62) - open/close 2656 (time 1787797377.46 total 40.02 last 63.87) - open/close 3351 (time 1787797387.47 total 50.03 last 69.44) - open/close 4053 (time 1787797397.47 total 60.03 last 70.17) total: 4500 open/close in 66.72 seconds: 67.44 ops/second - /unlink 1208 (time 1787797418.16 total 10.01 last 120.70) - /unlink 2318 (time 1787797428.17 total 20.01 last 110.95) - /unlink 3519 (time 1787797438.18 total 30.02 last 119.98) total: 4500 /unlink in 38.15 seconds: 117.97 ops/second lustre-MDT0000: clear the changelog for cl1 to record #90998 - open/close 659 (time 1787797461.08 total 10.01 last 65.83) - open/close 1295 (time 1787797471.10 total 20.03 last 63.45) - open/close 1894 (time 1787797481.11 total 30.04 last 59.85) - open/close 2602 (time 1787797491.12 total 40.05 last 70.71) - open/close 3268 (time 1787797501.13 total 50.06 last 66.54) - open/close 3981 (time 1787797511.13 total 60.06 last 71.29) total: 4500 open/close in 67.75 seconds: 66.42 ops/second lustre-MDT0000: clear the changelog for cl1 to record #103998 - /unlink 1215 (time 1787797533.66 total 10.00 last 121.46) - /unlink 2521 (time 1787797543.67 total 20.01 last 130.57) - /unlink 3807 (time 1787797553.67 total 30.01 last 128.57) total: 4500 /unlink in 35.49 seconds: 126.80 ops/second lustre-MDT0000: clear the changelog for cl1 to record #116998 - open/close 668 (time 1787797573.33 total 10.01 last 66.76) - open/close 1294 (time 1787797583.33 total 20.01 last 62.60) - open/close 1964 (time 1787797593.34 total 30.01 last 66.98) - open/close 2665 (time 1787797603.34 total 40.01 last 70.09) - open/close 3394 (time 1787797613.36 total 50.03 last 72.74) - open/close 4043 (time 1787797623.37 total 60.04 last 64.86) total: 4500 open/close in 66.82 seconds: 67.34 ops/second - /unlink 1384 (time 1787797644.30 total 10.01 last 138.31) - /unlink 2732 (time 1787797654.31 total 20.02 last 134.69) - /unlink 4059 (time 1787797664.32 total 30.02 last 132.58) total: 4500 /unlink in 33.46 seconds: 134.47 ops/second lustre-MDT0000: clear the changelog for cl1 to record #129998 - open/close 595 (time 1787797683.00 total 10.00 last 59.47) - open/close 1222 (time 1787797693.00 total 20.01 last 62.68) - open/close 1872 (time 1787797703.01 total 30.02 last 64.94) - open/close 2546 (time 1787797713.02 total 40.02 last 67.37) - open/close 3217 (time 1787797723.02 total 50.02 last 67.09) - open/close 3949 (time 1787797733.03 total 60.03 last 73.14) total: 4500 open/close in 68.07 seconds: 66.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #142998 - /unlink 1288 (time 1787797756.31 total 10.00 last 128.77) - /unlink 2624 (time 1787797766.31 total 20.00 last 133.59) - /unlink 3873 (time 1787797776.31 total 30.01 last 124.85) total: 4500 /unlink in 34.85 seconds: 129.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #155998 - open/close 616 (time 1787797795.62 total 10.01 last 61.55) - open/close 1269 (time 1787797805.63 total 20.01 last 65.27) - open/close 1932 (time 1787797815.66 total 30.04 last 66.11) - open/close 2620 (time 1787797825.66 total 40.05 last 68.75) - open/close 3314 (time 1787797835.66 total 50.05 last 69.39) - open/close 4000 (time 1787797845.68 total 60.06 last 68.53) total: 4500 open/close in 67.48 seconds: 66.68 ops/second - /unlink 1378 (time 1787797866.52 total 10.01 last 137.64) - /unlink 2702 (time 1787797876.52 total 20.02 last 132.35) - /unlink 3823 (time 1787797886.53 total 30.03 last 112.00) total: 4500 /unlink in 35.13 seconds: 128.11 ops/second lustre-MDT0000: clear the changelog for cl1 to record #168998 - open/close 721 (time 1787797906.46 total 10.00 last 72.07) - open/close 1379 (time 1787797916.47 total 20.01 last 65.76) - open/close 2098 (time 1787797926.49 total 30.03 last 71.79) - open/close 2818 (time 1787797936.49 total 40.03 last 71.95) - open/close 3506 (time 1787797946.50 total 50.04 last 68.75) - open/close 4078 (time 1787797956.50 total 60.04 last 57.19) total: 4500 open/close in 66.09 seconds: 68.08 ops/second - /unlink 1252 (time 1787797977.25 total 10.01 last 125.13) - /unlink 2501 (time 1787797987.26 total 20.01 last 124.80) - /unlink 3783 (time 1787797997.27 total 30.03 last 128.03) total: 4500 /unlink in 35.91 seconds: 125.33 ops/second lustre-MDT0000: clear the changelog for cl1 to record #181998 - open/close 728 (time 1787798017.29 total 10.01 last 72.70) - open/close 1359 (time 1787798027.30 total 20.02 last 63.03) - open/close 2044 (time 1787798037.31 total 30.04 last 68.39) - open/close 2653 (time 1787798047.33 total 40.05 last 60.82) - open/close 3304 (time 1787798057.33 total 50.05 last 65.10) - open/close 3950 (time 1787798067.33 total 60.06 last 64.58) total: 4500 open/close in 67.68 seconds: 66.49 ops/second lustre-MDT0000: clear the changelog for cl1 to record #194998 - /unlink 1308 (time 1787798089.54 total 10.00 last 130.75) - /unlink 2678 (time 1787798099.55 total 20.01 last 136.91) - /unlink 4120 (time 1787798109.55 total 30.02 last 144.12) total: 4500 /unlink in 32.87 seconds: 136.88 ops/second lustre-MDT0000: clear the changelog for cl1 to record #207998 - open/close 733 (time 1787798127.32 total 10.01 last 73.25) - open/close 1420 (time 1787798137.34 total 20.02 last 68.61) - open/close 2156 (time 1787798147.34 total 30.02 last 73.59) - open/close 2899 (time 1787798157.34 total 40.02 last 74.28) - open/close 3648 (time 1787798167.36 total 50.04 last 74.79) - open/close 4326 (time 1787798177.36 total 60.04 last 67.77) total: 4500 open/close in 62.52 seconds: 71.98 ops/second - /unlink 1252 (time 1787798195.47 total 10.00 last 125.15) - /unlink 2518 (time 1787798205.49 total 20.02 last 126.38) - /unlink 3800 (time 1787798215.49 total 30.02 last 128.17) total: 4500 /unlink in 35.78 seconds: 125.78 ops/second lustre-MDT0000: clear the changelog for cl1 to record #220998 - open/close 671 (time 1787798235.10 total 10.01 last 67.03) - open/close 1367 (time 1787798245.11 total 20.02 last 69.53) - open/close 1923 (time 1787798255.13 total 30.04 last 55.48) - open/close 2531 (time 1787798265.14 total 40.05 last 60.75) - open/close 3161 (time 1787798275.14 total 50.05 last 62.98) - open/close 3810 (time 1787798285.15 total 60.06 last 64.84) - open/close 4401 (time 1787798295.16 total 70.07 last 59.05) total: 4500 open/close in 71.47 seconds: 62.97 ops/second - /unlink 1422 (time 1787798310.92 total 10.01 last 142.06) - /unlink 2610 (time 1787798320.93 total 20.02 last 118.73) - /unlink 3737 (time 1787798330.93 total 30.02 last 112.69) total: 4500 /unlink in 37.74 seconds: 119.25 ops/second lustre-MDT0000: clear the changelog for cl1 to record #233998 lustre-MDT0000: clear the changelog for cl1 to record #246998 - open/close 628 (time 1787798353.83 total 10.02 last 62.69) - open/close 1209 (time 1787798363.84 total 20.03 last 58.04) - open/close 1705 (time 1787798373.85 total 30.04 last 49.55) - open/close 2303 (time 1787798383.86 total 40.04 last 59.75) - open/close 2885 (time 1787798393.86 total 50.05 last 58.18) - open/close 3505 (time 1787798403.87 total 60.06 last 61.95) - open/close 4188 (time 1787798413.88 total 70.07 last 68.24) total: 4500 open/close in 75.05 seconds: 59.96 ops/second - /unlink 1288 (time 1787798433.17 total 10.00 last 128.79) - /unlink 2459 (time 1787798443.17 total 20.01 last 117.01) - /unlink 3614 (time 1787798453.18 total 30.01 last 115.47) total: 4500 /unlink in 37.26 seconds: 120.77 ops/second lustre-MDT0000: clear the changelog for cl1 to record #259998 - open/close 654 (time 1787798475.35 total 10.00 last 65.37) - open/close 1323 (time 1787798485.35 total 20.01 last 66.88) - open/close 1981 (time 1787798495.36 total 30.01 last 65.78) - open/close 2687 (time 1787798505.39 total 40.05 last 70.33) - open/close 3365 (time 1787798515.40 total 50.06 last 67.74) - open/close 4046 (time 1787798525.41 total 60.06 last 68.06) total: 4500 open/close in 67.06 seconds: 67.10 ops/second - /unlink 1275 (time 1787798547.09 total 10.00 last 127.45) - /unlink 2460 (time 1787798557.09 total 20.01 last 118.46) - /unlink 3704 (time 1787798567.10 total 30.02 last 124.29) total: 4500 /unlink in 36.89 seconds: 122.00 ops/second lustre-MDT0000: clear the changelog for cl1 to record #272998 - open/close 666 (time 1787798589.49 total 10.01 last 66.56) - open/close 1322 (time 1787798599.49 total 20.01 last 65.58) - open/close 1936 (time 1787798609.49 total 30.01 last 61.40) - open/close 2595 (time 1787798619.50 total 40.01 last 65.87) - open/close 3235 (time 1787798629.51 total 50.02 last 63.95) - open/close 3864 (time 1787798639.51 total 60.02 last 62.89) - open/close 4409 (time 1787798649.52 total 70.03 last 54.46) total: 4500 open/close in 71.75 seconds: 62.72 ops/second lustre-MDT0000: clear the changelog for cl1 to record #285998 - /unlink 1118 (time 1787798666.61 total 10.01 last 111.74) - /unlink 2222 (time 1787798676.62 total 20.01 last 110.33) - /unlink 3454 (time 1787798686.62 total 30.01 last 123.20) total: 4500 /unlink in 38.72 seconds: 116.23 ops/second lustre-MDT0000: clear the changelog for cl1 to record #298998 - open/close 593 (time 1787798710.13 total 10.01 last 59.25) - open/close 1194 (time 1787798720.14 total 20.02 last 60.01) - open/close 1884 (time 1787798730.15 total 30.03 last 68.96) - open/close 2520 (time 1787798740.16 total 40.04 last 63.54) - open/close 3156 (time 1787798750.16 total 50.04 last 63.58) - open/close 3869 (time 1787798760.18 total 60.06 last 71.16) total: 4500 open/close in 69.03 seconds: 65.19 ops/second - /unlink 1388 (time 1787798783.50 total 10.00 last 138.78) - /unlink 2633 (time 1787798793.50 total 20.01 last 124.44) - /unlink 3925 (time 1787798803.51 total 30.01 last 129.12) total: 4500 /unlink in 34.73 seconds: 129.56 ops/second lustre-MDT0000: clear the changelog for cl1 to record #311998 - open/close 607 (time 1787798823.03 total 10.01 last 60.65) - open/close 1312 (time 1787798833.04 total 20.02 last 70.42) - open/close 1951 (time 1787798843.05 total 30.03 last 63.82) - open/close 2614 (time 1787798853.06 total 40.04 last 66.22) - open/close 3220 (time 1787798863.06 total 50.05 last 60.59) - open/close 3895 (time 1787798873.07 total 60.06 last 67.44) total: 4500 open/close in 69.15 seconds: 65.08 ops/second - /unlink 1184 (time 1787798896.74 total 10.00 last 118.38) - /unlink 2485 (time 1787798906.74 total 20.00 last 130.08) - /unlink 3658 (time 1787798916.76 total 30.01 last 117.17) total: 4500 /unlink in 36.99 seconds: 121.65 ops/second kill changelog reader /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 11344: 95247 Terminated coproc COPROC $LFS changelog --follow $service (wd: ~) lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 Cleanup test_135 umount lustre on /mnt/lustre..... Stopping client oleg323-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg323-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg323-server stop mds service on oleg323-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg323-server stop mds service on oleg323-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg323-server unloading modules via unload_modules_local on: 'oleg323-server' oleg323-server: oleg323-server.virtnet: executing unload_modules_local oleg323-server: kmemleak disabled modules unloaded.