== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 15:32:52 (1746127972) start mds service on oleg104-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg104-server' oleg104-server: oleg104-server.virtnet: executing load_modules_local oleg104-server: Loading modules from /home/green/git/lustre-release/lustre oleg104-server: detected 4 online CPUs by sysfs oleg104-server: MODOPTS_LIBCFS= oleg104-server: Force libcfs to create 2 CPU partitions oleg104-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg104-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg104-server: oleg104-server.virtnet: executing set_default_debug -1 all pdsh@oleg104-client: oleg104-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg104-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg104-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg104-server: oleg104-server.virtnet: executing set_default_debug -1 all pdsh@oleg104-client: oleg104-server: ssh exited with exit code 1 Started lustre-OST0000 oleg104-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg104-client.virtnet: -o user_xattr,flock 192.168.201.104@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 215 (time 1746128015.55 total 10.99 last 19.57) - open/close 467 (time 1746128026.70 total 22.13 last 22.61) - open/close 617 (time 1746128037.55 total 32.99 last 13.82) - open/close 976 (time 1746128050.56 total 45.99 last 27.61) - open/close 1304 (time 1746128062.57 total 58.01 last 27.29) - open/close 1428 (time 1746128075.08 total 70.51 last 9.92) - open/close 1522 (time 1746128085.14 total 80.58 last 9.34) - open/close 1694 (time 1746128096.10 total 91.54 last 15.69) - open/close 1846 (time 1746128106.34 total 101.78 last 14.84) - open/close 1985 (time 1746128117.36 total 112.80 last 12.62) - open/close 2148 (time 1746128127.66 total 123.10 last 15.82) - open/close 2288 (time 1746128138.53 total 133.96 last 12.88) - open/close 2456 (time 1746128149.12 total 144.56 last 15.86) - open/close 2605 (time 1746128159.70 total 155.14 last 14.08) - open/close 2721 (time 1746128170.40 total 165.84 last 10.84) - open/close 2814 (time 1746128180.73 total 176.17 last 9.00) - open/close 2935 (time 1746128191.43 total 186.87 last 11.31) - open/close 3076 (time 1746128201.67 total 197.11 last 13.77) - open/close 3203 (time 1746128211.68 total 207.12 last 12.68) - open/close 3313 (time 1746128222.35 total 217.79 last 10.31) - open/close 3450 (time 1746128232.48 total 227.92 last 13.52) - open/close 3747 (time 1746128242.50 total 237.94 last 29.66) - open/close 3873 (time 1746128253.12 total 248.55 last 11.87) - open/close 4036 (time 1746128264.00 total 259.44 last 14.97) - open/close 4151 (time 1746128274.60 total 270.04 last 10.85) - open/close 4295 (time 1746128284.61 total 280.05 last 14.39) - open/close 4477 (time 1746128294.61 total 290.05 last 18.19) - open/close 4665 (time 1746128305.42 total 300.86 last 17.40) - open/close 4838 (time 1746128316.38 total 311.82 last 15.79) - open/close 4979 (time 1746128326.40 total 321.84 last 14.06) - open/close 5174 (time 1746128337.26 total 332.70 last 17.96) - open/close 5290 (time 1746128348.03 total 343.47 last 10.77) - open/close 5439 (time 1746128358.13 total 353.57 last 14.75) - open/close 5640 (time 1746128368.14 total 363.58 last 20.08) - open/close 5830 (time 1746128378.18 total 373.62 last 18.94) - open/close 5971 (time 1746128388.27 total 383.70 last 13.97) - open/close 6094 (time 1746128399.26 total 394.70 last 11.18) - open/close 6292 (time 1746128410.20 total 405.64 last 18.10) - open/close 6487 (time 1746128420.21 total 415.65 last 19.48) - open/close 6667 (time 1746128430.22 total 425.66 last 17.98) - open/close 6875 (time 1746128441.23 total 436.67 last 18.90) - open/close 7071 (time 1746128451.72 total 447.16 last 18.67) - open/close 7266 (time 1746128462.49 total 457.93 last 18.11) - open/close 7459 (time 1746128473.02 total 468.46 last 18.33) - open/close 7700 (time 1746128483.76 total 479.20 last 22.44) - open/close 7956 (time 1746128494.01 total 489.45 last 24.96) - open/close 8159 (time 1746128504.02 total 499.46 last 20.28) - open/close 8330 (time 1746128514.02 total 509.46 last 17.10) - open/close 8598 (time 1746128524.48 total 519.92 last 25.62) - open/close 8807 (time 1746128535.04 total 530.48 last 19.80) - open/close 9009 (time 1746128545.62 total 541.06 last 19.09) - open/close 9236 (time 1746128555.63 total 551.07 last 22.69) - open/close 9477 (time 1746128565.63 total 561.07 last 24.08) - open/close 9675 (time 1746128576.44 total 571.88 last 18.32) - open/close 9862 (time 1746128586.87 total 582.31 last 17.92) - open/close 10000 (time 1746128593.46 total 588.89 last 20.96) - open/close 10236 (time 1746128604.04 total 599.48 last 22.29) - open/close 10447 (time 1746128614.49 total 609.93 last 20.20) open(/mnt/lustre/d101a.conf-sanity/f101a.conf-sanity-10546) error: No space left on device total: 10546 open/close in 612.50 seconds: 17.22 ops/second /home/green/git/lustre-release/lustre/tests/conf-sanity.sh: line 8972: kill: (8803) - No such process - unlinked 0 (time 1746128622 ; total 0 ; last 0) - unlinked 10000 (time 1746128692 ; total 70 ; last 70) unlink(/mnt/lustre/d101a.conf-sanity/f101a.conf-sanity-10546) error: No such file or directory total: 10546 unlinks in 74 seconds: 142.513519 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg104-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg104-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg104-server stop mds service on oleg104-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg104-server unloading modules via unload_modules_local on: 'oleg104-server' oleg104-server: oleg104-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg104-client: oleg104-client: ssh exited with exit code 2 pdsh@oleg104-client: oleg104-server: ssh exited with exit code 2 pdsh@oleg104-client: oleg104-client: ssh exited with exit code 2 pdsh@oleg104-client: oleg104-server: ssh exited with exit code 2