== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 14:32:16 (1777573936) start mds service on oleg222-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg222-server' oleg222-server: oleg222-server.virtnet: executing load_modules_local oleg222-server: Loading modules from /home/green/git/lustre-release/lustre oleg222-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy oleg222-server: detected 4 online CPUs by sysfs oleg222-server: MODOPTS_LIBCFS= oleg222-server: Force libcfs to create 2 CPU partitions oleg222-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg222-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg222-server: mdt/mdt options: 'mdt_enable_flr_ec=1' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg222-server: oleg222-server.virtnet: executing set_default_debug -1 all pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg222-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg222-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg222-server: oleg222-server.virtnet: executing set_default_debug -1 all pdsh@oleg222-client: oleg222-server: ssh exited with exit code 1 Started lustre-OST0000 oleg222-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg222-client.virtnet: -o user_xattr,flock 192.168.202.122@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 130 (time 1777573983.02 total 10.88 last 11.95) - open/close 285 (time 1777573994.03 total 21.89 last 14.08) - open/close 423 (time 1777574004.21 total 32.07 last 13.55) - open/close 609 (time 1777574015.08 total 42.94 last 17.11) - open/close 792 (time 1777574025.31 total 53.17 last 17.89) - open/close 1041 (time 1777574036.00 total 63.86 last 23.30) - open/close 1345 (time 1777574046.89 total 74.75 last 27.93) - open/close 1502 (time 1777574057.72 total 85.58 last 14.50) - open/close 1807 (time 1777574067.73 total 95.59 last 30.46) - open/close 2098 (time 1777574078.05 total 105.91 last 28.20) - open/close 2323 (time 1777574088.81 total 116.67 last 20.91) - open/close 2647 (time 1777574100.83 total 128.69 last 26.95) - open/close 3187 (time 1777574110.88 total 138.74 last 53.74) - open/close 3815 (time 1777574123.87 total 151.73 last 48.34) - open/close 4442 (time 1777574134.12 total 161.98 last 61.19) - open/close 5133 (time 1777574144.55 total 172.41 last 66.27) - open/close 5822 (time 1777574155.17 total 183.03 last 64.86) - open/close 6443 (time 1777574165.47 total 193.33 last 60.28) - open/close 6923 (time 1777574175.47 total 203.33 last 48.00) - open/close 7159 (time 1777574185.83 total 213.69 last 22.79) - open/close 7582 (time 1777574196.79 total 224.65 last 38.60) - open/close 7883 (time 1777574207.53 total 235.39 last 28.03) - open/close 7985 (time 1777574218.40 total 246.26 last 9.38) - open/close 8537 (time 1777574228.45 total 256.31 last 54.93) - open/close 9178 (time 1777574238.75 total 266.61 last 62.23) - open/close 9659 (time 1777574249.83 total 277.69 last 43.43) - open/close 9832 (time 1777574259.88 total 287.74 last 17.22) - open/close 10484 (time 1777574270.18 total 298.04 last 63.26) - open/close 11124 (time 1777574280.29 total 308.15 last 63.29) - open/close 11728 (time 1777574291.17 total 319.03 last 55.54) - open/close 12318 (time 1777574302.12 total 329.98 last 53.90) - open/close 12943 (time 1777574312.16 total 340.02 last 62.22) - open/close 13573 (time 1777574323.17 total 351.03 last 57.22) - open/close 14178 (time 1777574334.11 total 361.97 last 55.30) - open/close 15963 (time 1777574344.11 total 371.97 last 178.48) - open/close 20000 (time 1777574352.35 total 380.21 last 489.92) - open/close 24712 (time 1777574362.36 total 390.22 last 471.14) - open/close 29123 (time 1777574372.36 total 400.22 last 441.04) - open/close 34031 (time 1777574382.36 total 410.22 last 490.73) - open/close 38679 (time 1777574392.36 total 420.22 last 464.79) - open/close 40000 (time 1777574395.18 total 423.04 last 467.42) - open/close 44722 (time 1777574405.19 total 433.05 last 472.17) - open/close 49631 (time 1777574415.19 total 443.05 last 490.88) total: 50000 open/close in 443.75 seconds: 112.68 ops/second - unlinked 0 (time 1777574417 ; total 0 ; last 0) - unlinked 10000 (time 1777574431 ; total 14 ; last 14) - unlinked 20000 (time 1777574446 ; total 29 ; last 15) - unlinked 30000 (time 1777574461 ; total 44 ; last 15) - unlinked 40000 (time 1777574478 ; total 61 ; last 17) total: 50000 unlinks in 76 seconds: 657.894714 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg222-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg222-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg222-server stop mds service on oleg222-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg222-server unloading modules via unload_modules_local on: 'oleg222-server' oleg222-server: oleg222-server.virtnet: executing unload_modules_local oleg222-server: modules unloaded. pdsh@oleg222-client: oleg222-client: ssh exited with exit code 2 pdsh@oleg222-client: oleg222-server: ssh exited with exit code 2 pdsh@oleg222-client: oleg222-server: ssh exited with exit code 2 pdsh@oleg222-client: oleg222-client: ssh exited with exit code 2