== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 20:25:19 (1746145519) start mds service on oleg101-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg101-server' oleg101-server: oleg101-server.virtnet: executing load_modules_local oleg101-server: Loading modules from /home/green/git/lustre-release/lustre oleg101-server: detected 4 online CPUs by sysfs oleg101-server: MODOPTS_LIBCFS= oleg101-server: Force libcfs to create 2 CPU partitions oleg101-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg101-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg101-server: oleg101-server.virtnet: executing set_default_debug -1 all pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg101-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg101-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg101-server: oleg101-server.virtnet: executing set_default_debug -1 all pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1 Started lustre-OST0000 oleg101-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg101-client.virtnet: -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 131 (time 1746145570.54 total 10.02 last 13.07) - open/close 274 (time 1746145580.54 total 20.03 last 14.29) - open/close 417 (time 1746145591.48 total 30.97 last 13.07) - open/close 594 (time 1746145601.91 total 41.39 last 16.98) - open/close 688 (time 1746145611.92 total 51.40 last 9.39) - open/close 974 (time 1746145622.47 total 61.95 last 27.10) - open/close 1209 (time 1746145632.55 total 72.03 last 23.32) - open/close 1444 (time 1746145643.05 total 82.53 last 22.38) - open/close 1586 (time 1746145653.57 total 93.06 last 13.49) - open/close 1805 (time 1746145666.25 total 105.73 last 17.28) - open/close 2048 (time 1746145676.74 total 116.22 last 23.17) - open/close 2144 (time 1746145687.28 total 126.76 last 9.11) - open/close 2281 (time 1746145697.97 total 137.45 last 12.82) - open/close 2485 (time 1746145708.42 total 147.91 last 19.51) - open/close 2660 (time 1746145718.80 total 158.28 last 16.87) - open/close 2820 (time 1746145729.73 total 169.21 last 14.63) - open/close 2923 (time 1746145740.05 total 179.54 last 9.98) - open/close 3026 (time 1746145750.57 total 190.06 last 9.79) - open/close 3140 (time 1746145761.13 total 200.61 last 10.80) - open/close 3291 (time 1746145771.46 total 210.94 last 14.62) - open/close 3448 (time 1746145782.26 total 221.74 last 14.53) - open/close 3525 (time 1746145793.18 total 232.66 last 7.05) - open/close 3637 (time 1746145803.68 total 243.17 last 10.66) - open/close 3717 (time 1746145813.69 total 253.17 last 7.99) - open/close 3896 (time 1746145823.70 total 263.19 last 17.88) - open/close 3975 (time 1746145835.94 total 275.43 last 6.45) - open/close 4204 (time 1746145846.56 total 286.04 last 21.58) - open/close 4361 (time 1746145856.57 total 296.05 last 15.68) - open/close 4509 (time 1746145866.57 total 306.06 last 14.79) - open/close 4613 (time 1746145877.07 total 316.55 last 9.91) - open/close 4775 (time 1746145887.08 total 326.56 last 16.19) - open/close 4914 (time 1746145897.98 total 337.46 last 12.75) - open/close 5112 (time 1746145907.98 total 347.47 last 19.79) - open/close 5217 (time 1746145918.54 total 358.03 last 9.94) - open/close 5343 (time 1746145928.55 total 368.03 last 12.60) - open/close 5498 (time 1746145938.96 total 378.44 last 14.89) - open/close 5690 (time 1746145949.95 total 389.44 last 17.46) - open/close 5864 (time 1746145960.92 total 400.41 last 15.86) - open/close 6029 (time 1746145971.10 total 410.58 last 16.22) - open/close 6213 (time 1746145981.97 total 421.45 last 16.93) - open/close 6354 (time 1746145991.97 total 431.45 last 14.10) - open/close 6511 (time 1746146002.31 total 441.80 last 15.18) - open/close 6661 (time 1746146012.55 total 452.04 last 14.65) - open/close 6811 (time 1746146023.36 total 462.84 last 13.88) - open/close 6886 (time 1746146033.45 total 472.94 last 7.43) - open/close 7023 (time 1746146043.54 total 483.02 last 13.59) - open/close 7186 (time 1746146053.55 total 493.03 last 16.28) - open/close 7330 (time 1746146065.81 total 505.30 last 11.74) - open/close 7406 (time 1746146075.82 total 515.30 last 7.60) - open/close 7614 (time 1746146091.59 total 531.07 last 13.19) - open/close 7781 (time 1746146101.59 total 541.08 last 16.70) - open/close 7908 (time 1746146112.07 total 551.56 last 12.12) - open/close 8161 (time 1746146122.48 total 561.96 last 24.31) open(/mnt/lustre/d101a.conf-sanity/f101a.conf-sanity-8241) error: No space left on device total: 8241 open/close in 564.03 seconds: 14.61 ops/second /home/green/git/lustre-release/lustre/tests/conf-sanity.sh: line 8972: kill: (8817) - No such process - unlinked 0 (time 1746146130 ; total 1 ; last 1) unlink(/mnt/lustre/d101a.conf-sanity/f101a.conf-sanity-8241) error: No such file or directory total: 8241 unlinks in 49 seconds: 168.183670 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg101-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg101-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg101-server stop mds service on oleg101-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg101-server unloading modules via unload_modules_local on: 'oleg101-server' oleg101-server: oleg101-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg101-client: oleg101-client: ssh exited with exit code 2 pdsh@oleg101-client: oleg101-server: ssh exited with exit code 2 pdsh@oleg101-client: oleg101-client: ssh exited with exit code 2 pdsh@oleg101-client: oleg101-server: ssh exited with exit code 2