== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 09:59:38 (1787839178) start mds service on oleg140-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1049: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg140-server' oleg140-server: oleg140-server.virtnet: executing load_modules_local oleg140-server: Loading modules from /home/green/git/lustre-release/lustre oleg140-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1049: echo: write error: Device or resource busy oleg140-server: detected 4 online CPUs by sysfs oleg140-server: MODOPTS_LIBCFS= oleg140-server: Force libcfs to create 2 CPU partitions oleg140-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg140-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg140-server: mdt/mdt options: 'mdt_enable_flr_ec=1' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg140-server: oleg140-server.virtnet: executing set_default_debug -1 all pdsh@oleg140-client: oleg140-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg140-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg140-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg140-server: oleg140-server.virtnet: executing set_default_debug -1 all pdsh@oleg140-client: oleg140-server: ssh exited with exit code 1 Started lustre-OST0000 oleg140-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg140-client.virtnet: -o user_xattr,flock 192.168.201.140@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 196 (time 1787839266.93 total 10.03 last 19.54) - open/close 449 (time 1787839277.01 total 20.10 last 25.11) - open/close 678 (time 1787839287.77 total 30.86 last 21.29) - open/close 923 (time 1787839297.80 total 40.90 last 24.41) - open/close 1156 (time 1787839308.58 total 51.67 last 21.62) - open/close 1313 (time 1787839318.95 total 62.05 last 15.14) - open/close 1479 (time 1787839329.95 total 73.05 last 15.09) - open/close 1649 (time 1787839339.99 total 83.08 last 16.93) - open/close 1816 (time 1787839350.56 total 93.65 last 15.80) - open/close 1982 (time 1787839361.13 total 104.22 last 15.71) - open/close 2058 (time 1787839371.45 total 114.55 last 7.36) - open/close 2197 (time 1787839381.46 total 124.55 last 13.89) - open/close 2308 (time 1787839391.50 total 134.60 last 11.05) - open/close 2448 (time 1787839401.53 total 144.63 last 13.96) - open/close 2567 (time 1787839412.89 total 155.99 last 10.48) - open/close 2737 (time 1787839423.12 total 166.22 last 16.61) - open/close 3023 (time 1787839433.13 total 176.23 last 28.57) - open/close 3101 (time 1787839443.14 total 186.23 last 7.80) - open/close 3208 (time 1787839453.20 total 196.29 last 10.63) - open/close 3431 (time 1787839463.21 total 206.31 last 22.27) - open/close 3616 (time 1787839473.23 total 216.33 last 18.46) - open/close 3729 (time 1787839484.04 total 227.14 last 10.45) - open/close 3893 (time 1787839494.22 total 237.31 last 16.12) - open/close 4122 (time 1787839505.14 total 248.23 last 20.97) - open/close 4235 (time 1787839515.16 total 258.26 last 11.27) - open/close 4373 (time 1787839525.46 total 268.55 last 13.40) - open/close 4497 (time 1787839537.68 total 280.78 last 10.15) - open/close 4629 (time 1787839547.79 total 290.88 last 13.06) - open/close 4761 (time 1787839557.79 total 300.88 last 13.20) - open/close 4873 (time 1787839567.79 total 310.89 last 11.19) - open/close 5072 (time 1787839578.92 total 322.01 last 17.89) - open/close 5303 (time 1787839588.94 total 332.04 last 23.03) - open/close 5528 (time 1787839599.13 total 342.22 last 22.10) - open/close 5605 (time 1787839609.19 total 352.29 last 7.65) - open/close 5738 (time 1787839619.19 total 362.29 last 13.30) - open/close 5887 (time 1787839629.21 total 372.31 last 14.88) - open/close 6026 (time 1787839639.21 total 382.31 last 13.90) - open/close 6107 (time 1787839649.68 total 392.77 last 7.74) - open/close 6238 (time 1787839659.70 total 402.79 last 13.08) - open/close 6509 (time 1787839670.35 total 413.44 last 25.45) - open/close 6640 (time 1787839680.36 total 423.45 last 13.08) - open/close 6754 (time 1787839690.36 total 433.46 last 11.40) - open/close 6956 (time 1787839700.88 total 443.97 last 19.21) - open/close 7060 (time 1787839710.88 total 453.97 last 10.40) - open/close 7107 (time 1787839722.48 total 465.57 last 4.05) - open/close 7296 (time 1787839733.52 total 476.61 last 17.12) - open/close 7435 (time 1787839744.17 total 487.26 last 13.05) - open/close 7566 (time 1787839754.17 total 497.27 last 13.09) - open/close 7661 (time 1787839765.24 total 508.34 last 8.58) - open/close 7787 (time 1787839775.27 total 518.37 last 12.57) - open/close 7929 (time 1787839785.29 total 528.38 last 14.17) - open/close 8014 (time 1787839795.30 total 538.39 last 8.49) - open/close 8136 (time 1787839805.31 total 548.40 last 12.19) - open/close 8252 (time 1787839815.34 total 558.44 last 11.56) - open/close 8403 (time 1787839825.37 total 568.46 last 15.07) - open/close 8504 (time 1787839835.74 total 578.84 last 9.73) - open/close 8713 (time 1787839846.43 total 589.52 last 19.56) - open/close 8946 (time 1787839856.46 total 599.55 last 23.22) - open/close 9136 (time 1787839866.90 total 609.99 last 18.20) - open/close 9337 (time 1787839877.20 total 620.30 last 19.50) - open/close 9582 (time 1787839888.03 total 631.12 last 22.64) - open/close 9799 (time 1787839898.09 total 641.19 last 21.56) - open/close 9945 (time 1787839908.37 total 651.47 last 14.20) - open/close 10000 (time 1787839910.77 total 653.87 last 22.92) - open/close 10105