== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 21:50:22 (1749606622) start mds service on oleg136-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg136-server' oleg136-server: oleg136-server.virtnet: executing load_modules_local oleg136-server: Loading modules from /home/green/git/lustre-release/lustre oleg136-server: detected 4 online CPUs by sysfs oleg136-server: MODOPTS_LIBCFS= oleg136-server: Force libcfs to create 2 CPU partitions oleg136-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory oleg136-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg136-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg136-server: oleg136-server.virtnet: executing set_default_debug -1 all pdsh@oleg136-client: oleg136-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg136-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg136-server: oleg136-server.virtnet: executing set_default_debug -1 all pdsh@oleg136-client: oleg136-server: ssh exited with exit code 1 Started lustre-OST0000 oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg136-client.virtnet: -o user_xattr,flock 192.168.201.136@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 302 (time 1749606664.27 total 10.00 last 30.19) - open/close 559 (time 1749606674.28 total 20.01 last 25.69) - open/close 758 (time 1749606685.05 total 30.78 last 18.46) - open/close 1001 (time 1749606695.92 total 41.65 last 22.37) - open/close 1355 (time 1749606706.26 total 51.99 last 34.23) - open/close 1663 (time 1749606716.45 total 62.18 last 30.21) - open/close 2028 (time 1749606726.75 total 72.48 last 35.45) - open/close 2225 (time 1749606737.56 total 83.30 last 18.22) - open/close 2454 (time 1749606748.27 total 94.00 last 21.39) - open/close 2709 (time 1749606758.27 total 104.00 last 25.50) - open/close 2960 (time 1749606768.27 total 114.00 last 25.10) - open/close 3160 (time 1749606778.91 total 124.64 last 18.80) - open/close 3470 (time 1749606788.97 total 134.70 last 30.80) - open/close 3864 (time 1749606799.03 total 144.76 last 39.18) - open/close 4577 (time 1749606809.94 total 155.67 last 65.33) - open/close 4880 (time 1749606819.94 total 165.67 last 30.30) - open/close 5412 (time 1749606829.94 total 175.67 last 53.20) - open/close 5918 (time 1749606840.00 total 185.73 last 50.33) - open/close 6381 (time 1749606850.03 total 195.76 last 46.16) - open/close 6936 (time 1749606860.05 total 205.78 last 55.39) - open/close 7496 (time 1749606870.08 total 215.81 last 55.81) - open/close 8051 (time 1749606880.14 total 225.87 last 55.16) - open/close 8695 (time 1749606890.32 total 236.05 last 63.27) - open/close 9310 (time 1749606900.46 total 246.19 last 60.66) - open/close 9912 (time 1749606910.62 total 256.35 last 59.23) - open/close 10000 (time 1749606912.86 total 258.59 last 39.30) - open/close 10391 (time 1749606922.97 total 268.70 last 38.68) - open/close 10944 (time 1749606933.50 total 279.23 last 52.50) - open/close 13410 (time 1749606943.50 total 289.23 last 246.56) - open/close 18316 (time 1749606953.50 total 299.24 last 490.58) - open/close 20000 (time 1749606957.67 total 303.40 last 404.35) - open/close 24005 (time 1749606967.67 total 313.40 last 400.50) - open/close 27891 (time 1749606977.67 total 323.40 last 388.54) - open/close 30000 (time 1749606983.81 total 329.54 last 343.54) - open/close 34033 (time 1749606993.81 total 339.54 last 403.23) - open/close 38718 (time 1749607003.81 total 349.54 last 468.47) - open/close 40000 (time 1749607007.03 total 352.76 last 398.44) - open/close 44637 (time 1749607017.03 total 362.76 last 463.59) - open/close 47367 (time 1749607027.03 total 372.76 last 272.97) total: 50000 open/close in 380.99 seconds: 131.24 ops/second - unlinked 0 (time 1749607036 ; total 0 ; last 0) - unlinked 10000 (time 1749607054 ; total 18 ; last 18) - unlinked 20000 (time 1749607068 ; total 32 ; last 14) - unlinked 30000 (time 1749607085 ; total 49 ; last 17) - unlinked 40000 (time 1749607104 ; total 68 ; last 19) total: 50000 unlinks in 82 seconds: 609.756104 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg136-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg136-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg136-server stop mds service on oleg136-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg136-server unloading modules via unload_modules_local on: 'oleg136-server' oleg136-server: oleg136-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg136-client: oleg136-server: ssh exited with exit code 2 pdsh@oleg136-client: oleg136-client: ssh exited with exit code 2 pdsh@oleg136-client: oleg136-server: ssh exited with exit code 2 pdsh@oleg136-client: oleg136-client: ssh exited with exit code 2