== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 23:01:46 (1786935706) start mds service on oleg122-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg122-server' oleg122-server: oleg122-server.virtnet: executing load_modules_local oleg122-server: Loading modules from /home/green/git/lustre-release/lustre oleg122-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg122-server: detected 4 online CPUs by sysfs oleg122-server: MODOPTS_LIBCFS= oleg122-server: Force libcfs to create 2 CPU partitions oleg122-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg122-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg122-server: mdt/mdt options: 'mdt_enable_flr_ec=1' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg122-server: oleg122-server.virtnet: executing set_default_debug -1 all pdsh@oleg122-client: oleg122-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg122-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg122-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg122-server: oleg122-server.virtnet: executing set_default_debug -1 all pdsh@oleg122-client: oleg122-server: ssh exited with exit code 1 Started lustre-OST0000 oleg122-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg122-client.virtnet: -o user_xattr,flock 192.168.201.122@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 770 (time 1786935745.77 total 10.37 last 74.26) - open/close 1464 (time 1786935755.77 total 20.37 last 69.39) - open/close 2171 (time 1786935765.93 total 30.53 last 69.59) - open/close 2840 (time 1786935775.97 total 40.58 last 66.58) - open/close 3617 (time 1786935786.09 total 50.69 last 76.84) - open/close 4351 (time 1786935796.26 total 60.87 last 72.12) - open/close 5011 (time 1786935806.26 total 70.87 last 65.99) - open/close 5848 (time 1786935816.55 total 81.16 last 81.35) - open/close 6620 (time 1786935826.86 total 91.46 last 74.93) - open/close 7464 (time 1786935837.22 total 101.83 last 81.42) - open/close 8302 (time 1786935847.66 total 112.26 last 80.32) - open/close 9037 (time 1786935857.66 total 122.26 last 73.49) - open/close 9691 (time 1786935868.52 total 133.12 last 60.22) - open/close 10000 (time 1786935873.44 total 138.04 last 62.81) - open/close 10733 (time 1786935884.45 total 149.06 last 66.53) - open/close 11473 (time 1786935894.46 total 159.06 last 73.99) - open/close 12203 (time 1786935904.46 total 169.06 last 72.98) - open/close 12869 (time 1786935915.37 total 179.97 last 61.06) - open/close 13572 (time 1786935925.48 total 190.08 last 69.52) - open/close 14329 (time 1786935935.85 total 200.45 last 72.99) - open/close 14871 (time 1786935946.73 total 211.34 last 49.80) - open/close 15562 (time 1786935956.84 total 221.45 last 68.34) - open/close 16250 (time 1786935967.79 total 232.39 last 62.88) - open/close 16980 (time 1786935977.79 total 242.39 last 72.99) - open/close 17517 (time 1786935988.65 total 253.25 last 49.45) - open/close 18218 (time 1786935998.70 total 263.30 last 69.74) - open/close 18678 (time 1786936009.32 total 273.93 last 43.30) - open/close 19341 (time 1786936019.37 total 283.97 last 65.99) - open/close 20000 (time 1786936028.06 total 292.67 last 75.80) - open/close 20437 (time 1786936038.76 total 303.36 last 40.86) - open/close 21190 (time 1786936049.77 total 314.37 last 68.40) - open/close 22374 (time 1786936059.77 total 324.37 last 118.39) - open/close 26830 (time 1786936069.77 total 334.37 last 445.52) - open/close 30000 (time 1786936076.97 total 341.57 last 440.30) - open/close 34276 (time 1786936086.97 total 351.57 last 427.56) - open/close 38795 (time 1786936096.97 total 361.57 last 451.87) - open/close 40000 (time 1786936099.59 total 364.19 last 459.95) - open/close 44383 (time 1786936109.59 total 374.20 last 438.22) - open/close 48320 (time 1786936119.59 total 384.20 last 393.66) total: 50000 open/close in 388.21 seconds: 128.80 ops/second - unlinked 0 (time 1786936125 ; total 0 ; last 0) - unlinked 10000 (time 1786936141 ; total 16 ; last 16) - unlinked 20000 (time 1786936157 ; total 32 ; last 16) - unlinked 30000 (time 1786936172 ; total 47 ; last 15) - unlinked 40000 (time 1786936188 ; total 63 ; last 16) total: 50000 unlinks in 82 seconds: 609.756104 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg122-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg122-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg122-server stop mds service on oleg122-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg122-server unloading modules via unload_modules_local on: 'oleg122-server' oleg122-server: oleg122-server.virtnet: executing unload_modules_local oleg122-server: modules unloaded. pdsh@oleg122-client: oleg122-client: ssh exited with exit code 2 pdsh@oleg122-client: oleg122-server: ssh exited with exit code 2 pdsh@oleg122-client: oleg122-client: ssh exited with exit code 2 pdsh@oleg122-client: oleg122-server: ssh exited with exit code 2