== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 00:05:44 (1772168744) start mds service on oleg206-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg206-server' oleg206-server: oleg206-server.virtnet: executing load_modules_local oleg206-server: Loading modules from /home/green/git/lustre-release/lustre oleg206-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy oleg206-server: detected 4 online CPUs by sysfs oleg206-server: MODOPTS_LIBCFS= oleg206-server: Force libcfs to create 2 CPU partitions oleg206-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg206-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg206-server: oleg206-server.virtnet: executing set_default_debug -1 all pdsh@oleg206-client: oleg206-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg206-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg206-server: oleg206-server.virtnet: executing set_default_debug -1 all pdsh@oleg206-client: oleg206-server: ssh exited with exit code 1 Started lustre-OST0000 oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg206-client.virtnet: -o user_xattr,flock 192.168.202.106@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 392 (time 1772168786.63 total 10.40 last 37.69) - open/close 820 (time 1772168796.64 total 20.40 last 42.79) - open/close 1262 (time 1772168806.73 total 30.50 last 43.79) - open/close 1761 (time 1772168816.84 total 40.61 last 49.35) - open/close 2056 (time 1772168827.78 total 51.55 last 26.96) - open/close 2303 (time 1772168837.92 total 61.69 last 24.37) - open/close 2480 (time 1772168849.63 total 73.40 last 15.11) - open/close 2733 (time 1772168859.72 total 83.49 last 25.08) - open/close 2898 (time 1772168870.73 total 94.50 last 14.99) - open/close 3409 (time 1772168880.73 total 104.50 last 51.10) - open/close 3785 (time 1772168891.15 total 114.92 last 36.08) - open/close 3981 (time 1772168901.52 total 125.28 last 18.91) - open/close 4225 (time 1772168911.88 total 135.65 last 23.55) - open/close 4793 (time 1772168921.93 total 145.70 last 56.51) - open/close 5356 (time 1772168932.04 total 155.81 last 55.69) - open/close 5861 (time 1772168942.86 total 166.62 last 46.68) - open/close 6238 (time 1772168953.80 total 177.57 last 34.45) - open/close 6437 (time 1772168964.30 total 188.07 last 18.95) - open/close 6576 (time 1772168975.24 total 199.01 last 12.70) - open/close 7022 (time 1772168985.24 total 209.01 last 44.59) - open/close 7199 (time 1772168995.85 total 219.62 last 16.69) - open/close 7579 (time 1772169005.85 total 229.62 last 38.00) - open/close 8025 (time 1772169016.45 total 240.22 last 42.06) - open/close 8643 (time 1772169027.21 total 250.97 last 57.47) - open/close 9238 (time 1772169037.90 total 261.66 last 55.67) - open/close 9782 (time 1772169048.45 total 272.22 last 51.52) - open/close 10000 (time 1772169051.00 total 274.77 last 85.57) - open/close 10424 (time 1772169061.32 total 285.09 last 41.09) open(/mnt/lustre/d101a.conf-sanity/f101a.conf-sanity-10514) error: No space left on device total: 10514 open/close in 286.31 seconds: 36.72 ops/second /home/green/git/lustre-release/lustre/tests/conf-sanity.sh: line 8955: kill: (74145) - No such process - unlinked 0 (time 1772169064 ; total 0 ; last 0) - unlinked 10000 (time 1772169076 ; total 12 ; last 12) unlink(/mnt/lustre/d101a.conf-sanity/f101a.conf-sanity-10514) error: No such file or directory total: 10514 unlinks in 12 seconds: 876.166687 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg206-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg206-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg206-server stop mds service on oleg206-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg206-server unloading modules via unload_modules_local on: 'oleg206-server' oleg206-server: oleg206-server.virtnet: executing unload_modules_local oleg206-server: modules unloaded. pdsh@oleg206-client: oleg206-client: ssh exited with exit code 2 pdsh@oleg206-client: oleg206-server: ssh exited with exit code 2 pdsh@oleg206-client: oleg206-client: ssh exited with exit code 2 pdsh@oleg206-client: oleg206-server: ssh exited with exit code 2