== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 21:18:40 (1746148720) start mds service on oleg232-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg232-server' oleg232-server: oleg232-server.virtnet: executing load_modules_local oleg232-server: Loading modules from /home/green/git/lustre-release/lustre oleg232-server: detected 4 online CPUs by sysfs oleg232-server: MODOPTS_LIBCFS= oleg232-server: Force libcfs to create 2 CPU partitions oleg232-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg232-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg232-server: oleg232-server.virtnet: executing set_default_debug -1 all pdsh@oleg232-client: oleg232-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg232-server Starting mds2: -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg232-server: oleg232-server.virtnet: executing set_default_debug -1 all pdsh@oleg232-client: oleg232-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg232-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg232-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg232-server Starting ost1: -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg232-server: oleg232-server.virtnet: executing set_default_debug -1 all pdsh@oleg232-client: oleg232-server: ssh exited with exit code 1 Started lustre-OST0000 oleg232-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg232-client.virtnet: -o user_xattr,flock 192.168.202.132@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 191 (time 1746148782.50 total 10.17 last 18.78) - open/close 396 (time 1746148792.73 total 20.40 last 20.04) - open/close 653 (time 1746148803.15 total 30.82 last 24.66) - open/close 959 (time 1746148814.13 total 41.80 last 27.88) - open/close 1238 (time 1746148824.64 total 52.31 last 26.54) - open/close 1537 (time 1746148835.41 total 63.07 last 27.77) - open/close 1640 (time 1746148846.50 total 74.17 last 9.28) - open/close 1944 (time 1746148857.34 total 85.01 last 28.05) - open/close 2075 (time 1746148867.94 total 95.61 last 12.36) - open/close 2278 (time 1746148878.83 total 106.50 last 18.64) - open/close 2571 (time 1746148889.11 total 116.78 last 28.51) - open/close 2851 (time 1746148899.45 total 127.12 last 27.09) - open/close 3078 (time 1746148910.38 total 138.04 last 20.77) - open/close 3273 (time 1746148920.47 total 148.14 last 19.32) - open/close 3423 (time 1746148931.49 total 159.16 last 13.60) - open/close 3617 (time 1746148941.63 total 169.30 last 19.14) - open/close 3896 (time 1746148951.63 total 179.30 last 27.89) - open/close 4098 (time 1746148962.28 total 189.95 last 18.97) - open/close 4295 (time 1746148973.01 total 200.68 last 18.36) - open/close 4554 (time 1746148983.53 total 211.20 last 24.62) - open/close 4773 (time 1746148993.69 total 221.36 last 21.55) - open/close 5002 (time 1746149003.70 total 231.37 last 22.88) - open/close 5243 (time 1746149014.03 total 241.70 last 23.34) - open/close 5464 (time 1746149028.22 total 255.89 last 15.57) - open/close 5698 (time 1746149038.23 total 265.90 last 23.37) - open/close 5934 (time 1746149048.24 total 275.91 last 23.60) - open/close 6238 (time 1746149060.88 total 288.55 last 24.03) open(/mnt/lustre/d101a.conf-sanity/f101a.conf-sanity-6289) error: No space left on device total: 6289 open/close in 292.09 seconds: 21.53 ops/second /home/green/git/lustre-release/lustre/tests/conf-sanity.sh: line 8972: kill: (13083) - No such process - unlinked 0 (time 1746149069 ; total 0 ; last 0) unlink(/mnt/lustre/d101a.conf-sanity/f101a.conf-sanity-6289) error: No such file or directory total: 6289 unlinks in 29 seconds: 216.862076 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg232-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg232-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg232-server stop mds service on oleg232-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg232-server stop mds service on oleg232-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg232-server unloading modules via unload_modules_local on: 'oleg232-server' oleg232-server: oleg232-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg232-client: oleg232-client: ssh exited with exit code 2 pdsh@oleg232-client: oleg232-server: ssh exited with exit code 2 pdsh@oleg232-client: oleg232-server: ssh exited with exit code 2 pdsh@oleg232-client: oleg232-client: ssh exited with exit code 2