== conf-sanity test 69: replace an OST with the same index ========================================================== 00:05:06 (1746072306)
start mds service on oleg242-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
ln: failed to create symbolic link '/sbin/.libs': Read-only file system
loading modules on: 'oleg242-server'
oleg242-server: oleg242-server.virtnet: executing load_modules_local
oleg242-server: Loading modules from /home/green/git/lustre-release/lustre
oleg242-server: detected 4 online CPUs by sysfs
oleg242-server: MODOPTS_LIBCFS=
oleg242-server: Force libcfs to create 2 CPU partitions
oleg242-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg242-server: quota/lquota options: 'hash_lqs_cur_bits=3'
Starting mds1: -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg242-server: oleg242-server.virtnet: executing set_default_debug -1 all
pdsh@oleg242-client: oleg242-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid
start ost1 service on oleg242-server
Starting ost1: -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg242-server: oleg242-server.virtnet: executing set_default_debug -1 all
pdsh@oleg242-client: oleg242-server: ssh exited with exit code 1
Started lustre-OST0000
oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid
mount lustre  on /mnt/lustre.....
Starting client: oleg242-client.virtnet:  -o user_xattr,flock 192.168.202.142@tcp:/lustre /mnt/lustre
seq.cli-lustre-OST0000-super.width=0x1ffffff
On OST0, 11872 inodes available. Want 99872. rc=0
 - open/close 1782 (time 1746072347.68 total 10.00 last 178.16)
 - open/close 3988 (time 1746072357.68 total 20.00 last 220.60)
 - open/close 6227 (time 1746072367.68 total 30.00 last 223.90)
 - open/close 8408 (time 1746072377.68 total 40.00 last 218.09)
total: 10000 open/close in 46.39 seconds: 215.55 ops/second
 - unlinked 0 (time 1746072385 ; total 0 ; last 0)
total: 10000 unlinks in 25 seconds: 400.000000 unlinks/second
 - open/close 2087 (time 1746072421.74 total 10.00 last 208.67)
 - open/close 4391 (time 1746072431.75 total 20.00 last 230.34)
 - open/close 7388 (time 1746072441.75 total 30.01 last 299.64)
 - open/close 9428 (time 1746072451.75 total 40.01 last 203.99)
total: 10000 open/close in 42.85 seconds: 233.39 ops/second
 - unlinked 0 (time 1746072455 ; total 0 ; last 0)
total: 10000 unlinks in 22 seconds: 454.545441 unlinks/second
 - open/close 3393 (time 1746072488.43 total 10.00 last 339.24)
 - open/close 6189 (time 1746072498.43 total 20.00 last 279.56)
 - open/close 9462 (time 1746072508.43 total 30.00 last 327.27)
total: 10000 open/close in 31.60 seconds: 316.46 ops/second
 - unlinked 0 (time 1746072511 ; total 1 ; last 1)
total: 10000 unlinks in 15 seconds: 666.666687 unlinks/second
 - open/close 2865 (time 1746072536.29 total 10.00 last 286.47)
 - open/close 5358 (time 1746072546.29 total 20.00 last 249.28)
 - open/close 8453 (time 1746072556.29 total 30.00 last 309.49)
total: 10000 open/close in 34.80 seconds: 287.33 ops/second
 - unlinked 0 (time 1746072562 ; total 0 ; last 0)
total: 10000 unlinks in 20 seconds: 500.000000 unlinks/second
 - open/close 2346 (time 1746072593.98 total 10.00 last 234.60)
 - open/close 5241 (time 1746072603.98 total 20.00 last 289.40)
 - open/close 6775 (time 1746072613.98 total 30.00 last 153.39)
 - open/close 8178 (time 1746072623.98 total 40.01 last 140.23)
total: 10000 open/close in 48.73 seconds: 205.20 ops/second
 - unlinked 0 (time 1746072633 ; total 0 ; last 0)
total: 10000 unlinks in 22 seconds: 454.545441 unlinks/second
 - open/close 2241 (time 1746072666.30 total 10.00 last 224.09)
 - open/close 5030 (time 1746072676.30 total 20.00 last 278.88)
 - open/close 8022 (time 1746072686.30 total 30.00 last 299.15)
total: 10000 open/close in 34.65 seconds: 288.58 ops/second
 - unlinked 0 (time 1746072691 ; total 0 ; last 0)
total: 10000 unlinks in 25 seconds: 400.000000 unlinks/second
 - open/close 1824 (time 1746072727.60 total 10.01 last 182.28)
 - open/close 3156 (time 1746072737.60 total 20.01 last 133.15)
 - open/close 4385 (time 1746072747.61 total 30.02 last 122.80)
 - open/close 5861 (time 1746072757.61 total 40.02 last 147.57)
 - open/close 7647 (time 1746072767.62 total 50.02 last 178.55)
 - open/close 9354 (time 1746072777.62 total 60.03 last 170.63)
total: 10000 open/close in 63.69 seconds: 157.01 ops/second
 - unlinked 0 (time 1746072782 ; total 0 ; last 0)
total: 10000 unlinks in 34 seconds: 294.117645 unlinks/second
 - open/close 1810 (time 1746072828.17 total 10.00 last 180.95)
 - open/close 3367 (time 1746072838.17 total 20.00 last 155.70)
 - open/close 4975 (time 1746072848.17 total 30.00 last 160.77)
 - open/close 6684 (time 1746072858.17 total 40.01 last 170.86)
 - open/close 9151 (time 1746072868.18 total 50.02 last 246.43)
total: 10000 open/close in 54.06 seconds: 184.99 ops/second
 - unlinked 0 (time 1746072873 ; total 0 ; last 0)
total: 10000 unlinks in 33 seconds: 303.030304 unlinks/second
 - open/close 1859 (time 1746072917.20 total 10.00 last 185.87)
 - open/close 3356 (time 1746072927.20 total 20.01 last 149.65)
 - open/close 4784 (time 1746072937.21 total 30.01 last 142.73)
 - open/close 6182 (time 1746072947.21 total 40.02 last 139.69)
 - open/close 7890 (time 1746072957.21 total 50.02 last 170.79)
 - open/close 9809 (time 1746072967.21 total 60.02 last 191.88)
total: 10000 open/close in 61.03 seconds: 163.86 ops/second
 - unlinked 0 (time 1746072969 ; total 0 ; last 0)
total: 10000 unlinks in 33 seconds: 303.030304 unlinks/second
 - open/close 1516 (time 1746073013.92 total 10.00 last 151.57)
 - open/close 3080 (time 1746073023.93 total 20.01 last 156.34)
 - open/close 4553 (time 1746073033.93 total 30.01 last 147.23)
 - open/close 6316 (time 1746073043.93 total 40.01 last 176.30)
 - open/close 7609 (time 1746073053.93 total 50.01 last 129.26)
 - open/close 9247 (time 1746073063.94 total 60.01 last 163.79)
total: 10000 open/close in 64.23 seconds: 155.68 ops/second
 - unlinked 0 (time 1746073069 ; total 0 ; last 0)
total: 10000 unlinks in 37 seconds: 270.270264 unlinks/second
rm: missing operand
Try 'rm --help' for more information.
umount lustre on /mnt/lustre.....
Stopping client oleg242-client.virtnet /mnt/lustre (opts:)
stop ost1 service on oleg242-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg242-server

   Permanent disk data:
Target:     lustre-OST0000
Index:      0
Lustre FS:  lustre
Mount type: zfs
Flags:      0x42
              (OST update )
Persistent mount opts: 
Parameters: mgsnode=192.168.202.142@tcp sys.timeout=20 autodegrade=on
mkfs_cmd = zpool create -f -O canmount=off lustre-ost1 /dev/vde
mkfs_cmd = zfs create -o canmount=off  -o quota=409600000 lustre-ost1/ost1
  xattr=sa
  dnodesize=auto
  recordsize=1M
  compression=off
Writing lustre-ost1/ost1 properties
  lustre:mgsnode=192.168.202.142@tcp
  lustre:sys.timeout=20
  lustre:autodegrade=on
  lustre:version=1
  lustre:flags=66
  lustre:index=0
  lustre:fsname=lustre
  lustre:svname=lustre-OST0000
start ost1 service on oleg242-server
Starting ost1: -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg242-server: oleg242-server.virtnet: executing set_default_debug -1 all
pdsh@oleg242-client: oleg242-server: ssh exited with exit code 1
Started lustre-OST0000
oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid
oleg242-server: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50
oleg242-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 3 sec
mount lustre  on /mnt/lustre.....
Starting client: oleg242-client.virtnet:  -o user_xattr,flock 192.168.202.142@tcp:/lustre /mnt/lustre
On OST0, 12693 used inodes rc=0
umount lustre on /mnt/lustre.....
Stopping client oleg242-client.virtnet /mnt/lustre (opts:)
stop ost1 service on oleg242-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg242-server
stop mds service on oleg242-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg242-server
unloading modules via unload_modules_local on: 'oleg242-server'
oleg242-server: oleg242-server.virtnet: executing unload_modules_local
modules unloaded.
pdsh@oleg242-client: oleg242-client: ssh exited with exit code 2
pdsh@oleg242-client: oleg242-server: ssh exited with exit code 2
pdsh@oleg242-client: oleg242-client: ssh exited with exit code 2
pdsh@oleg242-client: oleg242-server: ssh exited with exit code 2