== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 01:32:16 (1775885536) start mds service on oleg353-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg353-server' oleg353-server: oleg353-server.virtnet: executing load_modules_local oleg353-server: Loading modules from /home/green/git/lustre-release/lustre oleg353-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy oleg353-server: detected 4 online CPUs by sysfs oleg353-server: MODOPTS_LIBCFS= oleg353-server: Force libcfs to create 2 CPU partitions oleg353-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg353-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg353-server: mdt/mdt options: 'mdt_enable_flr_ec=1' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg353-server: oleg353-server.virtnet: executing set_default_debug -1 all pdsh@oleg353-client: oleg353-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg353-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg353-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg353-server: oleg353-server.virtnet: executing set_default_debug -1 all pdsh@oleg353-client: oleg353-server: ssh exited with exit code 1 Started lustre-OST0000 oleg353-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg353-client.virtnet: -o user_xattr,flock 192.168.203.153@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 490 (time 1775885577.18 total 10.34 last 47.38) - open/close 1113 (time 1775885588.00 total 21.16 last 57.60) - open/close 1711 (time 1775885598.88 total 32.04 last 54.96) - open/close 2347 (time 1775885609.89 total 43.05 last 57.77) - open/close 2905 (time 1775885620.70 total 53.86 last 51.59) - open/close 3475 (time 1775885631.52 total 64.68 last 52.71) - open/close 4043 (time 1775885642.40 total 75.56 last 52.21) - open/close 4557 (time 1775885652.96 total 86.12 last 48.65) - open/close 5077 (time 1775885663.45 total 96.61 last 49.56) - open/close 5611 (time 1775885674.08 total 107.24 last 50.25) - open/close 6297 (time 1775885684.08 total 117.24 last 68.59) - open/close 7125 (time 1775885694.56 total 127.72 last 79.03) - open/close 7913 (time 1775885704.93 total 138.09 last 76.00) - open/close 8691 (time 1775885715.30 total 148.46 last 75.03) - open/close 9517 (time 1775885725.66 total 158.83 last 79.67) - open/close 10000 (time 1775885734.24 total 167.41 last 56.29) - open/close 10585 (time 1775885744.93 total 178.09 last 54.76) - open/close 11151 (time 1775885755.68 total 188.84 last 52.64) - open/close 11651 (time 1775885766.11 total 199.27 last 47.93) - open/close 12193 (time 1775885776.80 total 209.96 last 50.72) - open/close 12712 (time 1775885787.36 total 220.52 last 49.13) - open/close 13265 (time 1775885798.11 total 231.27 last 51.45) - open/close 13817 (time 1775885808.86 total 242.03 last 51.33) - open/close 14446 (time 1775885818.87 total 252.03 last 62.89) - open/close 15063 (time 1775885829.66 total 262.82 last 57.15) - open/close 15349 (time 1775885840.67 total 273.83 last 25.98) - open/close 15676 (time 1775885850.78 total 283.94 last 32.34) - open/close 16319 (time 1775885861.54 total 294.70 last 59.79) - open/close 18639 (time 1775885871.54 total 304.70 last 231.98) - open/close 20000 (time 1775885874.43 total 307.59 last 470.14) - open/close 25226 (time 1775885884.43 total 317.59 last 522.54) - open/close 30000 (time 1775885894.16 total 327.32 last 490.87) - open/close 34863 (time 1775885904.16 total 337.32 last 486.23) - open/close 40000 (time 1775885914.14 total 347.30 last 514.64) - open/close 45264 (time 1775885924.14 total 357.30 last 526.39) total: 50000 open/close in 366.59 seconds: 136.39 ops/second - unlinked 0 (time 1775885934 ; total 0 ; last 0) - unlinked 10000 (time 1775885947 ; total 13 ; last 13) - unlinked 20000 (time 1775885961 ; total 27 ; last 14) - unlinked 30000 (time 1775885977 ; total 43 ; last 16) - unlinked 40000 (time 1775885992 ; total 58 ; last 15) total: 50000 unlinks in 74 seconds: 675.675659 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg353-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg353-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg353-server stop mds service on oleg353-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg353-server unloading modules via unload_modules_local on: 'oleg353-server' oleg353-server: oleg353-server.virtnet: executing unload_modules_local oleg353-server: modules unloaded. pdsh@oleg353-client: oleg353-client: ssh exited with exit code 2 pdsh@oleg353-client: oleg353-server: ssh exited with exit code 2 pdsh@oleg353-client: oleg353-client: ssh exited with exit code 2 pdsh@oleg353-client: oleg353-server: ssh exited with exit code 2