== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 06:16:35 (1783073795)
start mds service on oleg145-server
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
ln: failed to create symbolic link '/sbin/.libs': Read-only file system
loading modules on: 'oleg145-server'
oleg145-server: oleg145-server.virtnet: executing load_modules_local
oleg145-server: Loading modules from /home/green/git/lustre-release/lustre
oleg145-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy
oleg145-server: detected 4 online CPUs by sysfs
oleg145-server: MODOPTS_LIBCFS=
oleg145-server: Force libcfs to create 2 CPU partitions
oleg145-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg145-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg145-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
Start mds1: mount -t lustre -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg145-server: oleg145-server.virtnet: executing set_default_debug -1 all
pdsh@oleg145-client: oleg145-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid
start ost1 service on oleg145-server
Start ost1: mount -t lustre -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg145-server: oleg145-server.virtnet: executing set_default_debug -1 all
pdsh@oleg145-client: oleg145-server: ssh exited with exit code 1
Started lustre-OST0000
oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid
mount lustre  on /mnt/lustre.....
Starting client: oleg145-client.virtnet:  -o user_xattr,flock 192.168.201.145@tcp:/lustre /mnt/lustre
seq.cli-lustre-OST0000-super.width=0x1ffffff
 - open/close 180 (time 1783073838.50 total 10.39 last 17.33)
 - open/close 302 (time 1783073849.50 total 21.39 last 11.09)
 - open/close 560 (time 1783073859.74 total 31.63 last 25.20)
 - open/close 811 (time 1783073869.74 total 41.63 last 25.09)
 - open/close 1295 (time 1783073880.03 total 51.92 last 47.06)
 - open/close 1523 (time 1783073890.59 total 62.48 last 21.58)
 - open/close 1751 (time 1783073903.39 total 75.28 last 17.81)
 - open/close 1912 (time 1783073913.39 total 85.28 last 16.10)
 - open/close 2210 (time 1783073925.28 total 97.17 last 25.07)
 - open/close 2616 (time 1783073936.22 total 108.11 last 37.10)
 - open/close 2867 (time 1783073946.66 total 118.55 last 24.04)
 - open/close 3057 (time 1783073956.95 total 128.85 last 18.46)
 - open/close 3418 (time 1783073966.96 total 138.85 last 36.09)
 - open/close 3664 (time 1783073977.51 total 149.40 last 23.32)
 - open/close 3907 (time 1783073988.06 total 159.95 last 23.02)
 - open/close 4119 (time 1783073998.63 total 170.52 last 20.07)
 - open/close 4332 (time 1783074009.38 total 181.27 last 19.81)
 - open/close 4569 (time 1783074020.45 total 192.34 last 21.42)
 - open/close 4863 (time 1783074030.94 total 202.83 last 28.01)
 - open/close 5077 (time 1783074041.82 total 213.72 last 19.67)
 - open/close 5219 (time 1783074052.13 total 224.02 last 13.78)
 - open/close 5408 (time 1783074062.69 total 234.58 last 17.90)
 - open/close 5609 (time 1783074073.38 total 245.27 last 18.80)
open(/mnt/lustre/d101a.conf-sanity/f101a.conf-sanity-5779) error: No space left on device
total: 5779 open/close in 254.75 seconds: 22.69 ops/second
/home/green/git/lustre-release/lustre/tests/conf-sanity.sh: line 8991: kill: (75001) - No such process
 - unlinked 0 (time 1783074085 ; total 0 ; last 0)
unlink(/mnt/lustre/d101a.conf-sanity/f101a.conf-sanity-5779) error: No such file or directory
total: 5779 unlinks in 16 seconds: 361.187500 unlinks/second
umount lustre on /mnt/lustre.....
Stopping client oleg145-client.virtnet /mnt/lustre (opts:)
stop ost1 service on oleg145-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg145-server
stop mds service on oleg145-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg145-server

unloading modules via unload_modules_local on: 'oleg145-server'
oleg145-server: oleg145-server.virtnet: executing unload_modules_local
oleg145-server: 
modules unloaded.
pdsh@oleg145-client: oleg145-client: ssh exited with exit code 2
pdsh@oleg145-client: oleg145-server: ssh exited with exit code 2
pdsh@oleg145-client: oleg145-server: ssh exited with exit code 2
pdsh@oleg145-client: oleg145-client: ssh exited with exit code 2