== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 20:13:14 (1777594394) start mds service on oleg256-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg256-server' oleg256-server: oleg256-server.virtnet: executing load_modules_local oleg256-server: Loading modules from /home/green/git/lustre-release/lustre oleg256-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy oleg256-server: detected 4 online CPUs by sysfs oleg256-server: MODOPTS_LIBCFS= oleg256-server: Force libcfs to create 2 CPU partitions oleg256-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg256-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg256-server: mdt/mdt options: 'mdt_enable_flr_ec=1' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg256-server: oleg256-server.virtnet: executing set_default_debug -1 all pdsh@oleg256-client: oleg256-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg256-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg256-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg256-server: oleg256-server.virtnet: executing set_default_debug -1 all pdsh@oleg256-client: oleg256-server: ssh exited with exit code 1 Started lustre-OST0000 oleg256-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg256-client.virtnet: -o user_xattr,flock 192.168.202.156@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 545 (time 1777594436.31 total 10.53 last 51.78) - open/close 1092 (time 1777594446.80 total 21.02 last 52.11) - open/close 1746 (time 1777594457.75 total 31.97 last 59.75) - open/close 2390 (time 1777594467.75 total 41.97 last 64.39) - open/close 2971 (time 1777594477.84 total 52.06 last 57.56) - open/close 3677 (time 1777594488.79 total 63.01 last 64.49) - open/close 4349 (time 1777594498.79 total 73.01 last 67.18) - open/close 4867 (time 1777594509.33 total 83.55 last 49.13) - open/close 5520 (time 1777594520.21 total 94.43 last 60.03) - open/close 6197 (time 1777594531.03 total 105.25 last 62.59) - open/close 6709 (time 1777594541.59 total 115.81 last 48.49) - open/close 7265 (time 1777594552.21 total 126.43 last 52.32) - open/close 7812 (time 1777594562.77 total 136.99 last 51.80) - open/close 8383 (time 1777594573.39 total 147.61 last 53.76) - open/close 8951 (time 1777594584.09 total 158.30 last 53.13) - open/close 9646 (time 1777594594.09 total 168.31 last 69.49) - open/close 10000 (time 1777594598.05 total 172.27 last 89.36) - open/close 10722 (time 1777594608.05 total 182.27 last 72.20) - open/close 11417 (time 1777594618.13 total 192.35 last 68.93) - open/close 12103 (time 1777594628.24 total 202.46 last 67.83) - open/close 12797 (time 1777594638.29 total 212.51 last 69.06) - open/close 13535 (time 1777594648.29 total 222.51 last 73.80) - open/close 14256 (time 1777594658.39 total 232.61 last 71.40) - open/close 14992 (time 1777594668.76 total 242.98 last 70.99) - open/close 15722 (time 1777594678.76 total 252.98 last 72.98) - open/close 16297 (time 1777594689.57 total 263.79 last 53.22) - open/close 17036 (time 1777594699.57 total 273.79 last 73.90) - open/close 17618 (time 1777594710.42 total 284.64 last 53.61) - open/close 18301 (time 1777594721.36 total 295.58 last 62.43) - open/close 19004 (time 1777594731.36 total 305.58 last 70.29) - open/close 20000 (time 1777594733.48 total 307.70 last 471.72) - open/close 24141 (time 1777594743.48 total 317.70 last 414.03) - open/close 28588 (time 1777594753.48 total 327.70 last 444.60) - open/close 30000 (time 1777594756.62 total 330.84 last 448.94) - open/close 34441 (time 1777594766.63 total 340.85 last 444.03) - open/close 38814 (time 1777594776.63 total 350.85 last 437.19) - open/close 40000 (time 1777594779.47 total 353.69 last 417.63) - open/close 44171 (time 1777594789.47 total 363.69 last 417.06) - open/close 48440 (time 1777594799.47 total 373.69 last 426.89) total: 50000 open/close in 377.28 seconds: 132.53 ops/second - unlinked 0 (time 1777594804 ; total 0 ; last 0) - unlinked 10000 (time 1777594820 ; total 16 ; last 16) - unlinked 20000 (time 1777594835 ; total 31 ; last 15) - unlinked 30000 (time 1777594851 ; total 47 ; last 16) - unlinked 40000 (time 1777594866 ; total 62 ; last 15) total: 50000 unlinks in 78 seconds: 641.025635 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg256-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg256-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg256-server stop mds service on oleg256-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg256-server unloading modules via unload_modules_local on: 'oleg256-server' oleg256-server: oleg256-server.virtnet: executing unload_modules_local oleg256-server: modules unloaded. pdsh@oleg256-client: oleg256-server: ssh exited with exit code 2 pdsh@oleg256-client: oleg256-client: ssh exited with exit code 2 pdsh@oleg256-client: oleg256-server: ssh exited with exit code 2 pdsh@oleg256-client: oleg256-client: ssh exited with exit code 2