== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 18:21:44 (1715206904) start mds service on oleg230-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg230-server' oleg230-server: oleg230-server.virtnet: executing load_modules_local oleg230-server: Loading modules from /home/green/git/lustre-release/lustre oleg230-server: detected 4 online CPUs by sysfs oleg230-server: Force libcfs to create 2 CPU partitions oleg230-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg230-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg230-server: oleg230-server.virtnet: executing set_default_debug -1 all pdsh@oleg230-client: oleg230-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg230-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg230-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg230-server: oleg230-server.virtnet: executing set_default_debug -1 all pdsh@oleg230-client: oleg230-server: ssh exited with exit code 1 Started lustre-OST0000 oleg230-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg230-client.virtnet: -o user_xattr,flock oleg230-server@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 464 (time 1715206942.54 total 10.50 last 44.19) - open/close 801 (time 1715206953.54 total 21.50 last 30.64) - open/close 1252 (time 1715206963.87 total 31.83 last 43.67) - open/close 1929 (time 1715206974.81 total 42.77 last 61.88) - open/close 2257 (time 1715206984.91 total 52.87 last 32.47) - open/close 2547 (time 1715206994.93 total 62.89 last 28.93) - open/close 2842 (time 1715207005.80 total 73.76 last 27.15) - open/close 3177 (time 1715207015.91 total 83.87 last 33.13) - open/close 3424 (time 1715207026.71 total 94.67 last 22.88) - open/close 3895 (time 1715207036.99 total 104.95 last 45.81) - open/close 4222 (time 1715207047.05 total 115.01 last 32.51) - open/close 4416 (time 1715207057.81 total 125.77 last 18.02) - open/close 4923 (time 1715207068.58 total 136.54 last 47.09) - open/close 5268 (time 1715207078.79 total 146.75 last 33.79) - open/close 5685 (time 1715207089.16 total 157.12 last 40.21) - open/close 6109 (time 1715207099.57 total 167.53 last 40.71) - open/close 6465 (time 1715207109.88 total 177.84 last 34.55) - open/close 6883 (time 1715207120.37 total 188.33 last 39.83) - open/close 7232 (time 1715207130.63 total 198.59 last 34.03) - open/close 7491 (time 1715207140.63 total 208.59 last 25.90) - open/close 7554 (time 1715207150.92 total 218.88 last 6.12) - open/close 7599 (time 1715207161.14 total 229.10 last 4.40) - open/close 7917 (time 1715207171.39 total 239.35 last 31.04) - open/close 8160 (time 1715207182.34 total 250.30 last 22.18) - open/close 8596 (time 1715207192.82 total 260.78 last 41.61) - open/close 8848 (time 1715207202.82 total 270.78 last 25.19) - open/close 9165 (time 1715207212.96 total 280.92 last 31.26) - open/close 9444 (time 1715207223.85 total 291.81 last 25.62) - open/close 9820 (time 1715207234.26 total 302.22 last 36.12) - open/close 10000 (time 1715207237.99 total 305.95 last 48.27) - open/close 10485 (time 1715207248.64 total 316.60 last 45.53) - open/close 10722 (time 1715207259.60 total 327.56 last 21.63) - open/close 13991 (time 1715207269.60 total 337.56 last 326.85) - open/close 16584 (time 1715207279.60 total 347.56 last 259.30) - open/close 19171 (time 1715207289.60 total 357.56 last 258.66) - open/close 20000 (time 1715207292.93 total 360.89 last 249.02) - open/close 22520 (time 1715207302.93 total 370.90 last 251.92) - open/close 25239 (time 1715207312.94 total 380.90 last 271.89) - open/close 29056 (time 1715207322.94 total 390.90 last 381.70) - open/close 32357 (time 1715207332.94 total 400.90 last 330.10) - open/close 34901 (time 1715207342.94 total 410.90 last 254.35) - open/close 37691 (time 1715207352.94 total 420.90 last 278.96) - open/close 40000 (time 1715207362.07 total 430.03 last 252.99) - open/close 42691 (time 1715207372.07 total 440.03 last 269.01) - open/close 45105 (time 1715207382.07 total 450.03 last 241.36) - open/close 47539 (time 1715207392.07 total 460.04 last 243.31) - open/close 49992 (time 1715207402.07 total 470.04 last 245.30) total: 50000 open/close in 470.06 seconds: 106.37 ops/second - unlinked 0 (time 1715207403 ; total 0 ; last 0) - unlinked 10000 (time 1715207418 ; total 15 ; last 15) - unlinked 20000 (time 1715207437 ; total 34 ; last 19) - unlinked 30000 (time 1715207458 ; total 55 ; last 21) - unlinked 40000 (time 1715207479 ; total 76 ; last 21) total: 50000 unlinks in 97 seconds: 515.463928 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg230-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg230-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg230-server stop mds service on oleg230-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg230-server unloading modules on: 'oleg230-server' oleg230-server: oleg230-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg230-client: oleg230-client: ssh exited with exit code 2 pdsh@oleg230-client: oleg230-server: ssh exited with exit code 2 pdsh@oleg230-client: oleg230-client: ssh exited with exit code 2 pdsh@oleg230-client: oleg230-server: ssh exited with exit code 2